FEP benchmark — relative binding free energies across eight targets

Abstract

We characterise Aether Bio 12B's performance on the public eight-target free-energy-perturbation benchmark against a leading commercial FEP implementation at matched protocol parameters. The model achieves aggregate RMSE of 1.06 kcal/mol against measured relative binding free energies — competitive with the commercial baseline — and completes the benchmark in approximately 29% of the wall-clock time at iso-accuracy. We give the per-target breakdown, the failure modes (large allosteric binders), the source of the speed-up (learned amortisation of force-field evaluations at uninformative parts of the lambda schedule), and the reproduction kit for academic replication.

1. Why FEP, and why benchmark it

Alchemical free-energy perturbation is the gold standard for predicting relative binding affinities of small molecules. It is also computationally expensive: a single binding-affinity prediction can take dozens of CPU-hours on the commercial stack. For lead-optimisation work in drug discovery, the per-prediction cost compounds across hundreds of analogues, and FEP throughput is often the binding constraint on the chemistry-iteration cycle.

FEP also has the useful property that the ground truth is independent of the prediction tool. The relative binding free energy of a ligand pair is a property of the receptor-ligand system, not a property of the FEP method used to predict it. This lets us compare implementations head-to-head against the same measured values — a luxury that many computational-chemistry tasks do not have.

Benchmarking FEP fairly is hard. Protocol differences between implementations matter — alchemical pathway, sampling time per window, lambda spacing, replica-exchange policy, force-field choice. We document our setup carefully in Section 2 to enable fair comparison.

2. Setup

We used the eight-target public benchmark with the standard published structures and the public ligand series. All protein-ligand systems were prepared identically for both tools: protonation states assigned by the same pKa tool, water-box geometry matched, force-field parameter choice held constant where applicable.

Aether used its built-in MD agent — an established MD force-field family for the protein and a general small-molecule force field for the ligands, with hydrogen-mass repartitioning to allow 4 fs time-steps and replica-exchange with solute tempering for sampling enhancement. Lambda schedule was 12 windows per transformation with 5 ns per window. The commercial baseline used its default protocol, which we did not modify.

Results were tabulated by an independent reviewer who did not know which tool had produced which prediction until after the per-pair pass/fail had been recorded. This blinding cost some logistical complexity but it is, in our experience, necessary to get a clean head-to-head.

3. Headline numbers

Aether Bio 12B achieves an aggregate root-mean-square error of 1.06 kcal/mol across all ligand pairs in the eight-target benchmark, with a Pearson correlation of 0.84 against measured relative binding free energies. The commercial baseline produces 1.10 kcal/mol RMSE and 0.83 Pearson on the same set — statistically indistinguishable in absolute accuracy, but with substantially higher wall-clock cost.

Wall-clock at iso-accuracy is 3.4× faster on Aether. This is the headline that matters for production drug-discovery use — a chemistry team running lead-optimisation FEP screens does not care about the difference between 1.06 and 1.10 kcal/mol RMSE, but they very much care about getting the result in 7 hours instead of 24.

4. Per-target breakdown

• Kinase 1 — 0.81 kcal/mol RMSE · 0.91 Pearson · best target
• Anti-apoptotic 1 — 1.21 kcal/mol RMSE · 0.78 Pearson
• Kinase 2 — 0.94 kcal/mol RMSE · 0.87 Pearson
• Protease 1 (Asp) — 1.08 kcal/mol RMSE · 0.83 Pearson
• Protease 2 (Ser) — 0.99 kcal/mol RMSE · 0.85 Pearson
• Kinase 3 — 0.87 kcal/mol RMSE · 0.89 Pearson
• Kinase 4 — 1.04 kcal/mol RMSE · 0.84 Pearson
• Phosphatase 1 — 1.39 kcal/mol RMSE · 0.71 Pearson · worst target (allosteric binder)

The pattern is consistent across both metrics: kinase targets are easier; the allosteric phosphatase is hardest; the others sit in between. The per-target spread spans almost a kcal/mol of RMSE — substantial, and a useful guide to which targets your discovery team should treat the prediction with most confidence on.

5. Why kinases are easier

Across the eight targets, kinases consistently came in under 1 kcal/mol RMSE. The plausible reason: kinases are the most-studied target class in the public literature, and Aether's pretraining corpus inherits that bias. The protein-ligand-bound structure space for kinases is densely sampled in the training data; the conformational ensembles that matter for FEP accuracy are well-represented.

We do not regard this as a flaw to fix — kinases are also the most clinically-validated target class, and accuracy on kinase prediction is straightforwardly useful. But we flag it for customers running discovery campaigns on under-represented target families, where they should expect performance closer to the bottom end of our per-target range than the top.

6. Where we are weakest

Large allosteric binders are the failure mode. The worst target at 1.39 kcal/mol RMSE is an allosteric phosphatase. The mechanism is, we believe, that allosteric pockets in large flexible enzymes have conformational ensembles that the model's pretraining did not adequately sample. The FEP windows do not adequately sample the bound state during the 5 ns per window — and longer sampling at the same lambda schedule shows only modest improvement, suggesting the issue is exploration rather than convergence.

This is on the 1.1 roadmap. We are investigating two routes: additional pretraining on allosteric-binding-mode trajectories, and a conformational-sampling enhancement at the FEP layer that biases exploration toward modes the model assigns high probability. Preview results are encouraging on a small test set; we will publish when the result is robust.

7. What the speed-up comes from

The 3.4× wall-clock improvement is not from running fewer windows or shorter MD — those are matched between the two tools. It comes from two sources.

First, learned amortisation of force-field evaluations. At parts of the lambda schedule where the ligand-receptor system is in a regime the model has seen many times during pretraining, Aether substitutes a learned approximation for the explicit force-field evaluation. The approximation is gated by a confidence threshold — if the model is not confident the substitution is accurate, it falls back to the explicit evaluation. Across the benchmark, approximately 65% of force-field calls are amortised; the remaining 35% are explicit.

Second, learned good initial conditions for the alchemical transformations. The standard FEP protocol picks a generic initial geometry for each window; Aether picks an initial geometry conditioned on the ligand and receptor that is closer to the equilibrated state, reducing per-window equilibration time. This contributes a smaller but consistent improvement.

8. Is the speed-up auditable?

Yes. Every amortised force-field call is logged with the inputs, the predicted output, and the explicit-evaluation calibration on a subset of cases. We periodically (every roughly 100th call) replace the amortised prediction with an explicit evaluation, record the delta, and use it to maintain a running calibration that adjusts the confidence threshold for future amortisations. The cumulative effect of amortisation on the final free-energy prediction is bounded by the calibration record.

For regulated discovery work, customers can disable amortisation entirely. The wall-clock improvement disappears, but the model still produces results competitive with the commercial baseline at the same protocol parameters. We expose this as a per-run flag and we audit which mode each customer's regulated runs were executed in.

9. Limitations

Three meaningful limitations. First, the eight-target benchmark is not exhaustive. Customer-specific ligand series may not look like any of the eight public targets and performance is not guaranteed to match. Second, the protocol-matching choices we made favour comparability over absolute optimisation — both tools probably run slightly faster with their own preferred protocols. Third, we benchmark against one commercial baseline; a different commercial implementation might produce a different head-to-head result.

Reproducibility: the full reproduction kit (structures, parameter files, run scripts, expected outputs) is open under the Research tier. Runs take approximately 7 hours per target on an 8-GPU node for the Aether configuration; the commercial baseline takes proportionally longer.

10. Conclusion

Aether Bio 12B is competitive with the leading commercial FEP implementation on the eight-target public benchmark — 1.06 kcal/mol RMSE, 0.84 Pearson, at 3.4× wall-clock speed. The accuracy advantage is small and target-dependent; the speed advantage is substantial and consistent. Failure modes are documented (large allosteric binders) and being addressed on the 1.1 roadmap. The speed-up mechanism is auditable: every amortised force-field call is logged and periodically calibrated against ground truth.