🚨In our NeurIPS paper, we bring encoder-decoders back.. for...

@mariannearr
Marianne Arriola@mariannearr
29 views Oct 29, 2025 ~1 min read
Advertisement
1
🚨In our NeurIPS paper, we bring encoder-decoders back.. for diffusion language models!

⚡️Encoder-decoders make diffusion sampling fast: a small (fast) decoder denoises tokens progressively and a large (slower) encoder represents clean context.
0:31
2
💡Our key insight is that diffusion LMs do two things: 1️⃣ represent clean tokens and 2️⃣ denoise corrupted ones.

🔄 By decoupling these tasks into separate modules, the encoder runs only periodically to refresh its contextual representation, while the decoder denoises in-between
Media image
3
🧱Block-wise decoding boosts quality: a decoder denoises each block and the encoder caches it before moving on

⏩ Encoder-decoders speed up block diffusion training! Decoder-only models process clean + noisy data, while we halve costs by splitting work across encoder and decoder
Media image
4
💪 Our efficient encoder-decoder block diffusion LM, E2D2 🤖, achieves faster inference and better performance on summarization, translation, and math compared to decoder-only diffusion LMs

💪 E2D2 is ~3x faster than MDLM while achieving better performance across tasks!
Media image
5
⚖️ Smaller decoder ➡️ faster inference, but lower quality.

💪 Across all decoder sizes, E2D2 outperforms decoder-only block diffusion LMs, achieving a strictly better speed-quality tradeoff.
Media image
6
Learn more here!
📄arxiv.org/abs/2510.22852
🌐 m-arriola.com/e2d2/
🧑‍💻github.com/kuleshov-group…
🤗huggingface.co/collections/ku…

This work was co-led with the talented @SchiffYair. Thanks to our amazing @cornell_tech team, @tienhaophung @SkyLi0n @volokuleshov

See you in San Diego🏝️!
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement