Measuring an AI model’s performance helps us improve it and assess whether it is safe to use. That assessment needs to happen throughout training, as the model’s capabilities change.

Training an AI model involves incremental changes, with every modification providing a checkpoint at which to perform necessary evaluations to understand the development of the model. During training, a model is optimised to reduce its “loss”; a measure of how well the model predicts what it is trained on, i.e. how well it performs its training objectives. As long as the loss continues to decrease, we can confidently say that the model is learning. However, a falling loss does not always mean better performance on the tasks we care about. A model can improve at predicting biological sequences while becoming worse at a particular downstream task, and the final checkpoint is not necessarily the best for every capability. Evaluating checkpoints throughout training helps us understand what a model is learning, guide changes to training, and identify the right checkpoint for a given task.

In biology, this is particularly difficult, as unlike more established areas of AI, biology lacks a common and broad set of benchmarks to develop foundation models. Relevant tasks span nucleic acids, proteins, genomes, cells, whole organisms and clinical tasks, each with its own datasets and targets, and experimental validation in the lab is far too slow to run on every checkpoint. As a result, evaluations are usually largely fragmented and manual, making it difficult to judge whether the training is working, how much information the model is absorbing, and how well a model is performing. BaseBench is the tool we’re building to solve this problem in biology, bringing our evaluation work into a single benchmarking platform. BaseBench runs every checkpoint of every model, including publicly available bio models, through the same rigorous evaluations and testing, ensuring all models are assessed in the same way. Over time, we are expanding BaseBench to hundreds of evaluations across the entirety of biology and we plan to open it to public submissions.

BaseBench methodology

BaseBench is built on a foundation of three tools; evaluations, checkpoints, and adapters, tied together but kept apart to facilitate development of each component individually.

An evaluation consists of a dataset, the output it needs from a model, and the rules for grading that output. A checkpoint is a model caught at one moment of its training, including its weights and its configuration. Between the two live adapters, which allow a family of models to provide the outputs an evaluation requires, such as embeddings, likelihoods or a generation.

Because these three pieces are kept decoupled, each one can be reused. A new checkpoint can use its model family’s existing adapters. A new evaluation runs on every model that can give the output it needs. A new adapter connects a family of models to compatible evaluations. This structure lets teams add models and evaluations independently, without rebuilding the connections between them each time.

A sensor monitors the versioned data store for new checkpoints or changes to evaluation datasets. A compute worker runs the appropriate adapter, a grader scores the output, and the result is recorded in a table that feeds the leaderboard. Training does not need to pause during this process, meaning the next checkpoint can continue developing while the previous one is assessed.

Each leaderboard result records the conditions that produced it: the version of the data, the identity of the checkpoint, the configuration of the adapter, and the steps from input data to final score. Each checkpoint also records its model family and the types of sequence it accepts. Together, these records allow results to be traced and evaluations to be reproduced. When a model cannot provide the output an evaluation requires, BaseBench records that limitation and the reason.

Consistent procedures make comparisons easier, but interpreting a result still requires context. Performance depends on the dataset, how it is split and the scoring method, so each evaluation states what it measures and where its conclusions stop. Adding tasks, improving datasets and correcting graders are part of maintaining BaseBench as models and evaluation methods develop.

Applications spanning molecules, medicines and safety

The questions whose answers we try to track in BaseBench come from many of the places where biology is put to work, from DNA to the clinic. These are the areas that BaseBench addresses today, and the list is growing.

Safety and biosecurity are a priority in our work. In October 2026, Basecamp Research became a research partner of the UK AI Biosecurity Lab, alongside Google DeepMind, with backing from OpenAI and Anthropic. In this partnership, we will give the lab access to our AI-biology models, capabilities and technical expertise. The lab begins with two priorities: to detect new biological risks earlier, and to develop vaccines, treatments and other medical countermeasures faster. As part of our work with AIBL, we will provide the biosafety and biosecurity benchmarks that are already integrated in BaseBench and run with the same rigour and consistency as all other evaluations.

What comes next

Our target is hundreds of evaluations across many areas of biology, with safety and biosecurity a priority among them, run on every checkpoint of every model and gathered on one leaderboard that the community can add to.

BaseBench is already working on our own internal models and those available publicly, and we plan to open it to public submissions in the near term. A submission can be a biological question with a dataset and a way to score it, an adapter for a family of models, or a checkpoint to evaluate. Reach out for early access and to find out more.