Introducing Tahoe-x1 (Tx1) by @tahoe_ai. A 3-billion-parameter,...

The past years have been defined by scaling laws: more compute & data, better models. This has held true for language and protein modeling. But does it also hold true for systems biology; for models that learn how cells & genes function across contexts and drug perturbations?
Two barriers have stood in the way of answering that question: Lack of large, diverse single-cell data and Lack of compute-efficient models to enable search for better architectures and designs at billion-parameter scales.
With Tahoe-100M, we created the largest perturbation dataset ever assembled to address the first barrier. And we are proud that it has been downloaded nearly 200K times in the last few months, a first for a biological dataset of this scale.
Now we’re introducing Tahoe-x1 to tackle the second barrier. Tx1 is the first billion-parameter, compute-efficient foundation model trained on perturbation-rich single-cell data. And it is fully open-source with open weights.
To make it so, we borrowed the best tricks from LMs (FlashAttention v2, FSDP, Dataset streaming, Mixed-precision + multi-node scaling). But even cooler: We improved the attention operation at the heart of these models.
We want people to put Tx1 to test. We have built a tool to help you generate embeddings and visualize them for your own single-cell transcriptomic data.
Progress will come not from one model, but from trying hundreds of designs and iterating on how best to represent the cell. We built Tx1 for that. That’s why we are open sourcing it - checkpoints, training code, evaluation workflows. Let's build!
🤗huggingface.co/tahoebio/Tahoe…
Hats off to the superteam that made this happen, @shreshth_gandhi who led the work; Farnoosh Javadi, @vallens, and Umair Khan for their fantastic work, @MattG_Jones for his thoughtful contributions, @BoWang87 and @HAOTIANCUI1 for early conversations and @amerigo_81, @genophoria, @iamjohnnyyu for supervision!





