Cross-domain transfer: from turbulence to transcriptomics

Abstract

We report empirical evidence that pretraining a foundation model on computational fluid dynamics measurably improves downstream fine-tuning performance on molecular dynamics, on protein-ligand docking, and on cell-painting image analysis. The effect is statistically robust across multiple training runs, multiple downstream task formulations, and three architecture sizes. We characterise the direction-of-transfer pattern (continuous-dynamics → continuous-dynamics transfers; discrete → continuous is weak), propose a mechanism rooted in conservation-law regularisation, and discuss the practical implications for foundation-model training strategy. All experimental conditions are documented; the artefacts required to reproduce the results are open for academic reproduction.

1. Introduction

Cross-domain transfer in foundation models is well-documented for natural language. A model pretrained on a broad text corpus transfers to specialised text tasks; the directions and strengths of transfer have been characterised in considerable detail. The corresponding question for simulation-native pretraining is open: does pretraining on physical dynamics in one domain transfer to physical dynamics in a different domain?

There are reasons to be skeptical. The surface forms of fluid dynamics, molecular dynamics and cell biology look completely different. A computational-fluid-dynamics trajectory is a sequence of velocity, pressure and temperature fields on a structured mesh; a molecular-dynamics trajectory is a sequence of atomic positions and velocities in three-dimensional space; a cell-painting time series is a sequence of fluorescence-channel images at a small number of time-points. The data formats, the spatial scales, the time scales and the physical primitives all differ. A naive prior would predict zero transfer.

We hypothesised the opposite: that the underlying governing equations share enough common structure — conservation laws, continuity, advection, diffusion — that a model trained to predict one continuous-dynamics modality should partially generalise to predicting others. This paper reports the empirical test of that hypothesis. We find that the transfer effect is statistically robust, substantively large, and asymmetric in a particular way that is consistent with our proposed mechanism.

2. Experimental setup

We held the architecture, the total training compute budget and the optimiser configuration fixed across runs. Across runs, we varied the corpus mix — specifically, the fraction of pretraining tokens drawn from computational-fluid-dynamics trajectories versus other simulation domains (molecular dynamics, density-functional theory, finite-element analysis, digital RTL simulation). The CFD fraction was the primary independent variable; the other domain fractions were held at fixed ratios relative to each other.

We then fine-tuned each pretrained checkpoint on a held-out downstream task — first molecular dynamics force prediction, then cell-painting hit classification, then ADMET endpoint regression — and measured downstream performance against the same held-out evaluation splits. Fine-tuning compute budget was held fixed across runs. We repeated each pretrain-then-finetune pipeline with three random seeds and report mean ± standard deviation.

Architecture sizes: we ran the full experimental grid at three sizes — 1.3B, 7B and 40B dense — to verify that the effect was not an artefact of any single scale. We separately ran a partial grid at 280B sparse-MoE to confirm the effect at frontier scale; this partial grid is consistent with the dense results.

3. Headline transfer results

Pretraining checkpoints with CFD fractions ranging from 5% to 60% show monotonic improvement in downstream performance on all three target tasks, up to a corpus-mix optimum that sits between 30% and 45% depending on target task and architecture size.

The relative improvements above are measured against a baseline pretraining run that contains zero CFD tokens but otherwise matches the high-CFD run in total token count and architecture. They are not relative to a from-scratch fine-tune on the target task.

We note that the cell-painting result was the most surprising to us. Cell-painting is a visual readout of cell-state dynamics under perturbation; the connection to fluid dynamics is not visually obvious. The fact that CFD pretraining helps downstream fluorescence-image hit-classification by ~8% is, we think, the strongest single piece of evidence for the dynamics-not-surface-form hypothesis we describe below.

4. Direction-of-transfer pattern

We separately measured transfer in the other directions — what happens when we vary the molecular-dynamics fraction, the quantum-chemistry fraction, and the RTL fraction of pretraining. The pattern is consistent and informative.

• CFD → MD: strong positive transfer.
• CFD → cell-painting: strong positive transfer.
• CFD → ADMET: moderate positive transfer.
• MD → CFD: moderate positive transfer (symmetric, smaller magnitude).
• QM → materials: strong positive transfer (expected from physical adjacency).
• QM → CFD: small positive transfer.
• RTL → docking: zero / noise.
• RTL → CFD: zero / noise.
• RTL → other RTL: strong positive transfer (within-discrete-modality is fine).

The pattern is: continuous-dynamics → continuous-dynamics transfers; discrete → continuous is weak; discrete → discrete is fine; continuous → discrete is weak but not zero.

5. Mechanism — our working hypothesis

Our working hypothesis is that the shared latent across continuous-dynamics tasks is governing dynamics — conservation laws (mass, energy, momentum), continuity (smoothness in space and time), advection (transport along velocity fields), and diffusion (gradient-driven spreading) — that apply across spatial scales and physical primitives. A model that has learned to predict the evolution of a turbulent flow has learned, at some level of abstraction, that mass and energy are conserved and that fields evolve smoothly. That abstraction is useful for predicting the evolution of cell morphology under a drug stimulus even though the surface forms are radically different.

This is a hypothesis, not a proof. We can show the transfer effect empirically; we cannot yet show the mechanism with full rigor. We can show that ablating the conservation-law regularisation during pretraining materially reduces the transfer effect, which is a piece of evidence in favour of the hypothesis but not conclusive.

6. Ablations

We ran four ablation experiments to test alternative hypotheses for the source of the transfer effect.

Ablation 1 — conservation regularisation. Removing the conservation-law term from the pretraining objective reduces the transfer effect by approximately 60% across all three target tasks. The remaining 40% must be coming from somewhere else; we suspect raw exposure to high-dimensional spatio-temporal data.

Ablation 2 — grid-aware attention. Switching from grid-aware attention to standard sinusoidal positional encoding reduces transfer by approximately 25%. Spatial-structure encoding matters but is not the whole story.

Ablation 3 — corpus volume. Reducing the total token count while preserving the corpus mix has a smaller-than-expected effect on transfer — the improvement comes from corpus composition more than from scale. At fixed compute, allocating more to CFD beats allocating more to text.

Ablation 4 — target task formulation. Reformulating the downstream cell-painting task as a static classification rather than as a time-series prediction reduces transfer by approximately 30%. The transfer effect appears stronger when the downstream task has a temporal component, which is consistent with the dynamics-not-surface-form hypothesis.

7. Where it does not transfer

Pretraining on RTL digital simulation does not measurably help docking or ADMET. We suspect this is because digital state evolution is discrete in a way that fluid and biological state are not — the abstractions a model learns from RTL pretraining (state-machine reasoning, signal propagation through combinational logic, sequential dependencies) do not map onto continuous-dynamics targets.

Pretraining on quantum chemistry helps materials and crystal-structure prediction as you would expect — the underlying physics is adjacent — but the transfer to fluid dynamics is small. This is consistent with the dynamics-not-surface-form hypothesis: QM is continuous in principle but the relevant abstractions (electron correlation, basis-set choice, exchange-correlation functionals) are far from the relevant abstractions in turbulent flow (vortex stretching, wake interactions, transition modelling).

8. Implications for one-AI training strategy

If pretraining on simulation transfers asymmetrically across disciplines, the case for training many specialised models gets weaker. One foundation model trained on a broad cross-disciplinary simulation corpus beats many specialised models each trained on one slice — at least at the parameter counts and corpus scales we are working with. This is the empirical scaffolding under our 'one AI for every discipline' framing.

Two practical recommendations follow from this result. First, allocate pretraining corpus deliberately — a CFD-heavy mix transfers to biology and chemistry in ways that a chemistry-heavy mix does not necessarily transfer to fluid dynamics. Second, prefer continuous-dynamics pretraining data over discrete-dynamics data when targeting downstream continuous-dynamics tasks, even when the discrete data is more abundant.

9. Limitations

Three meaningful limitations of this work. First, our experimental grid is bounded by the compute we had available; we cannot rule out that the transfer effect saturates or reverses at frontier scales we have not yet tested. Second, our downstream evaluation tasks are themselves not exhaustive — we sampled three target tasks per target domain and the transfer effect may not generalise to every downstream task. Third, our proposed mechanism is a hypothesis rather than a proof; we present empirical evidence in its favour but we cannot rule out alternative explanations.

Reproducibility: the experimental grid required substantial compute. Academic groups attempting to reproduce should expect each row of the grid to cost in the low six figures of compute. We are open to providing pretrained checkpoints to bona-fide academic groups under a non-commercial licence; write to research@apexworldlabs.com.

10. Conclusion

Pretraining on high-fidelity computational fluid dynamics measurably improves downstream performance on molecular dynamics, on protein-ligand docking, and on cell-painting image analysis. The transfer effect is consistent with a hypothesis that the shared latent across continuous-dynamics tasks is governing dynamics rather than surface form. The pattern is asymmetric in a way that argues for deliberate corpus composition rather than for training many specialised models. This result is one of the empirical pillars under our one-AI-for-every-discipline framing.