Introducing Tahoe-x1 (Tx1) by @tahoe_ai. A 3-billion-parameter,...

@nalidoust
Nima Alidoust@nalidoust
22 views Oct 25, 2025 ~3 min read
Advertisement
1
Introducing Tahoe-x1 (Tx1) by @tahoe_ai. A 3-billion-parameter, single-cell foundation model that learns unified representations of genes, cells, and drugs, achieving state-of-the-art performance across cancer-relevant cell biology benchmarks, open-sourced on @huggingface. 🧵
0:31
2
2/
The past years have been defined by scaling laws: more compute & data, better models. This has held true for language and protein modeling. But does it also hold true for systems biology; for models that learn how cells & genes function across contexts and drug perturbations?
3
3/
Two barriers have stood in the way of answering that question: Lack of large, diverse single-cell data and Lack of compute-efficient models to enable search for better architectures and designs at billion-parameter scales.
4
4/
With Tahoe-100M, we created the largest perturbation dataset ever assembled to address the first barrier. And we are proud that it has been downloaded nearly 200K times in the last few months, a first for a biological dataset of this scale.
5
5/
Now we’re introducing Tahoe-x1 to tackle the second barrier. Tx1 is the first billion-parameter, compute-efficient foundation model trained on perturbation-rich single-cell data. And it is fully open-source with open weights.
6
6/
Tx1 is a Transformer (scGPT-inspired self-supervised objective) that stays practical to train at 3B parameter scale, enabling empirical search for optimal architectures and hyperparameters for modeling cells. It is 3-30x more compute efficient than other cell state models.
Media image
7
7/
To make it so, we borrowed the best tricks from LMs (FlashAttention v2, FSDP, Dataset streaming, Mixed-precision + multi-node scaling). But even cooler: We improved the attention operation at the heart of these models.
8
8/ In earlier Tx1 models we built  a Triton-optimized bias-matrix trick for masking, but the final design goes further: leveraging fully dense attention with FlashAttention v2 , simpler, faster, and still cutting GPU memory use by ~10×.
Media image
9
9/ We built Tx1 both not only as a starting point for exploration but as a benchmark for the field. We designed new benchmarks to evaluate its performance in cancer-relevant discovery and translational tasks.
10
10/
Tx1 achieves SOTA on key discovery tasks. It outperforms other models (and for the first time matches with linear baselines) in predicting gene essentiality as measured by the landmark DepMap dataset, a key piece of data in identifying subtype-specific targets.
Media image
11
11/
Similarly, it does better than other models in inferring hallmark oncogenic programs, another benchmark we designed leveraging MSigDB, to evaluate the model’s ability to identify core signatures of tumor progression.
Media image
12
12/
Tx1 also moves us closer to in silico prediction of cell response to drug & other perturbations, with the aim to ultimately generalize to in silico clinical trials. Combined with post-training frameworks, it predicts drug responses in unseen cell types and patient contexts.
Media image
13
13/
It further provides evidence that better embeddings generated from Tx1 compared to other models improve zero-shot performance to generalize in entirely new contexts and perturbations, confirming earlier results
Media image
14
14/
We want people to put Tx1 to test. We have built a tool to help you generate embeddings and visualize them for your own single-cell transcriptomic data.
0:35
15
15/
Progress will come not from one model, but from trying hundreds of designs and iterating on how best to represent the cell. We built Tx1 for that. That’s why we are open sourcing it - checkpoints, training code, evaluation workflows. Let's build!

🤗huggingface.co/tahoebio/Tahoe…
16
Last/
Hats off to the superteam that made this happen, @shreshth_gandhi who led the work; Farnoosh Javadi, @vallens, and Umair Khan for their fantastic work, @MattG_Jones for his thoughtful contributions, @BoWang87 and @HAOTIANCUI1 for early conversations and @amerigo_81, @genophoria, @iamjohnnyyu for supervision!
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement