UniPat AI releases SPU, a 116-task benchmark for predicting wet-lab results

UniPat AI has led the release of a benchmark called Science Prediction Universe, or SPU, that tests whether AI systems can predict the results of wet-lab experiments before anyone runs them, according to coverage by Chinese tech outlet QbitAI.

The format is unusual. A model is handed a set of experimental conditions and asked to work out on its own which variant should yield more of a target product, or which antibody variant should express at a higher level. The answer key stays hidden until the model submits, and the predictions are then checked against measured results.

SPU contains 116 tasks and 4,207 wet-lab experiments across chemistry, materials science, biology and medicine, spanning 32 subfields. 100 public tasks were drawn from 85 recently published papers, with experts extracting candidates, conditions and measurements while keeping the full quantitative results for scoring; 16 private tasks come from four unpublished collaborations covering VHH nanobodies, reaction conditions, electrolytes and Magrolimab variants, so models cannot simply recall the literature.

Advisers and wet-lab partners include researchers from Stanford, Cornell and Harvard, along with other research institutes and biotech companies. Task selection is deliberately harsh: studies with incomplete measurements are dropped, as are problems that can be ordered along a single obvious trend or whose values sit too close together to separate models.

Each run is heavy by design: eight hours per task with eight CPU cores, 16 GiB of memory and a single H20 GPU, plus internet access and permission to install scientific software. The model decides how to gather evidence and which computational methods to use, then submits a low-to-high ranking of the comparison groups.

Scoring is sandboxed against contamination. Evaluation runs in isolated, offline containers, retrieval of the source papers is blocked, and tool output is scanned for DOIs and titles. Predictions are compared with measured results using Spearman correlation, and every task also generates 4,000 random rankings whose 95th percentile serves as the chance-level threshold.

The first evaluation covered 12 frontier models in 13 configurations. Claude Opus 5.5 posted the highest average score at 0.486, yet it beat the chance threshold on only about half of tasks; across the 13 configurations, the share of tasks above that threshold ranged from 34 to 52 percent. By discipline, Opus 5.5 led chemistry, Fable 5.1 led materials science, GPT-6 Astra Medium led biology and Opus 5 led medicine.

The most counterintuitive result concerns compute. Raising GPT-6 Astra from medium to maximum reasoning intensity multiplied median runtime by 4.8 and median API cost by 5.3, while scores barely moved: on the 79 tasks it completed, the model typically submitted an answer at around 26 percent of the median runtime and spent the rest on fine-tuning. Only 255 GPU runs succeeded across the suite, under a fifth of attempts, and average API cost per run ranged from $0.22 to $65.91.

The real contribution of SPU is not a leaderboard but the conversion of scientific judgement, previously hard to quantify, into numbers and trajectories that can be audited item by item. For teams considering AI in the lab, running a relevant task before committing to a real experiment may say more than a model’s rank on general benchmarks.