OpenAI test: AI agent scored 67 percent on laboratory data

OpenAI has introduced BioBasicsBench, a benchmark, meaning a standardized test, designed to show whether AI agents can interpret the raw data that comes out of everyday laboratory work. An AI agent is an AI system that works its way through a task on its own.

The test consists of 100 tasks drawn from real laboratory experiments. It was developed together with the biotechnology company Ligo, which works on enzyme design. The tasks cover five areas of routine wet lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods and molecular cloning.

The agents are given gel images, images of colony plates, microscopy stacks, sequencing data and videos of protocols, among other materials. They have to interpret the material, find errors and determine when data cannot support a conclusion.

Scientists created the tasks and the scoring criteria themselves. Each task was reviewed by another Ligo scientist, by an externally hired scientist with a PhD, and finally internally.

The scores

OpenAI tested 37 combinations of model and reasoning setting, with five attempts per task. That produced about 18,500 graded answers.

At the highest setting, GPT-6 Astra scored an average of 66.7 percent of the possible points. The margin of uncertainty (a 95 percent confidence interval) runs from 60.3 to 73.1. GPT-5.6 Sol scored 43.8 percent (37.6-50.0). According to Research & Development World, GPT-6.1 Sol scored 62.0 percent, GPT-6 Luna 38.3, GPT-5.5 35.4, GPT-5.4 33.2 and GPT-5.2 31.2 percent.

Astra also used about half as many output tokens, meaning chunks of text, per task as Sol: roughly 4,704 versus 9,099.

The scores come from OpenAI's own tests of its own models on a test that OpenAI itself developed together with Ligo.

Where does it go wrong?

The errors included the models ignoring inconsistencies in the labeling of control plates. Astra also overestimated the number of colonies by about 12 percent. The models could also sort data correctly and still draw the opposite conclusion, and they could misjudge colonies as successful based on the size of a PCR product.

In one example, GPT-5.6 Sol used empty negative control plates, while Astra discovered that the antibiotic controls were mislabeled.

OpenAI itself writes that a higher score on the test does not yet mean that models can make reliable assessments in the laboratory. Nicholas Larus-Stone, head of AI at Benchling, says that large language models can work in the wet lab with the right tooling setup and a human in the loop, meaning a person who follows along and checks the work.

The sources do not say whether the benchmark is publicly available.