🕳️🐇Into the Rabbit Hull – Part I (Part II tomorrow) An...

@thomas_fel_
Thomas Fel@thomas_fel_
7 views Oct 17, 2025 ~2 min read
Advertisement
1
🕳️🐇Into the Rabbit Hull – Part I (Part II tomorrow)

An interpretability deep dive into DINOv2, one of vision’s most important foundation models.

And today is Part I, buckle up, we're exploring some of its most charming features.
0:30
2
Assuming the Linear Rep. Hypothesis, SAEs arise naturally as instruments for concept extraction, they will be our companions in this descent.
Archetypal SAE uncovered 32k concepts.

Our first observation: different tasks recruit distinct regions of this conceptual space.
Media image
3
Let's zoom in on classification.
For every class, we find two concepts: one fires on the object (e.g., "rabbit"), and another fires everywhere *except* the object -- but only when it's present!

We call them Elsewhere Concepts (credit: @davidbau).
Media image
4
This kind of concept breaks a key assumption in interpretability: that a concept is about the tokens where it fires. Here it is the opposite—the concept is defined by where it does not fire. An open question is how models form such concepts.
5
Another surprise here: the most important concepts are not object-centric at all, but boundary detectors. Remarkably, these concepts coalesce into a low-dimensional subspace within (see paper).
Media image
6
Now for depth estimation. How does DINO know depth?

It turns out it has discovered several human-like monocular depth cues: texture gradients resembling blurring or bokeh, shadow detectors, and projective cues.

Most units mix cues, but a few remain remarkably pure.
Media image
7
Curious tokens, the registers.
DINO seems to use them to encode global invariants: we find concepts (directions) that fire exclusively (!) on registers.

Example of such concepts include motion blur detector and style (game screenshots, drawings, paintings, warped images...)
Media image
8
Huge thanks to all collaborators who made this work possible, and especially to @WangBinxu. This work grew from a year of collaboration!

Tomorrow, Part II: geometry of concepts and Minkowski Representation Hypothesis.

🕹️ kempnerinstitute.github.io/dinovision
📄 arxiv.org/pdf/2510.08638
Media image
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement