Why do LLMs hallucinate—and can we fix it at the architectural...
Zhihu Frontier@ZhihuFrontier
40 views
Dec 08, 2025
~2 min read
Advertisement
1
Why do LLMs hallucinate—and can we fix it at the architectural level? 🤯
NeurIPS 2025 Paper Credal Transformer offers a bold answer.
One of the paper's authors 栀染, breaks down the core insight on Zhihu:
🧠 The Core Problem: Softmax = Forced Certainty
Current Transformers are architecturally predisposed to hallucinate.
Why? Because Softmax is a "belief-forcing machine."
Even when logits reflect deep uncertainty (flat distribution), Softmax must allocate 100% probability across choices—erasing ambiguity, hesitation, and boundary-of-knowledge signals.
Every layer, every head repeats this "uncertainty annihilation."
➡️ The model's confident output is built on flattened ruins of erased doubt—what the authors call Artificial Certainty.
🔍 Key Idea: Uncertainty Isn't Noise — It's a Signal
They reject the assumption that uncertainty should be suppressed. Instead, they treat it as a first-class, computable variable.
Their approach brings in:
• Evidence Theory
• Dirichlet distributions (a natural prior for distributions over distributions)
Rather than producing one attention distribution, the model represents a whole set of possible attention distributions—parameterized by Dirichlet α.
This gives us something we've never had before:
🔥 A differentiable, local, intrinsic uncertainty measure 👉 Fig 3
This scalar field U(context,token) becomes a lens on the model's epistemic boundary.
And once uncertainty is measurable?
➡️Hallucination becomes an engineering problem—not an inevitability.
⚙️ Two decoding strategies enabled by Credal Transformers
1️⃣ Uncertainty-Guided Sequence Sampling
When a token's uncertainty exceeds threshold τ → the system detects a potential hallucination zone.
It performs:
• controlled rollback
• constrained Monte Carlo resampling
• penalization of all high-uncertainty paths
2️⃣ Confidence-Driven Speculative Decoding
A small draft model generates a candidate sequence; the Credal Transformer evaluates the integral of uncertainty along the whole path.
• If ∫U(t) dt stays low → accept the entire sequence (major speedup)
• If not → cut at the first spike; main model takes over with careful decoding
A dynamic tradeoff between efficiency and reliability.
🏁 What's the contribution?
"We gave the model a way to reflect on its own ignorance."
Credal Transformers let LLMs sense the limits of their knowledge, not just make guesses with forced confidence.
A step toward AI systems that are more reliable, more transparent, and more honest about uncertainty
Just a small milestone on a much longer road—but a meaningful one.
🔗 Original Zhihu post:
zhihu.com/question/19586…
📄 Paper:
openreview.net/forum?id=XTM1B…
#AI #LLM #Tech #NeurIPS #Transformer #Research
NeurIPS 2025 Paper Credal Transformer offers a bold answer.
One of the paper's authors 栀染, breaks down the core insight on Zhihu:
🧠 The Core Problem: Softmax = Forced Certainty
Current Transformers are architecturally predisposed to hallucinate.
Why? Because Softmax is a "belief-forcing machine."
Even when logits reflect deep uncertainty (flat distribution), Softmax must allocate 100% probability across choices—erasing ambiguity, hesitation, and boundary-of-knowledge signals.
Every layer, every head repeats this "uncertainty annihilation."
➡️ The model's confident output is built on flattened ruins of erased doubt—what the authors call Artificial Certainty.
🔍 Key Idea: Uncertainty Isn't Noise — It's a Signal
They reject the assumption that uncertainty should be suppressed. Instead, they treat it as a first-class, computable variable.
Their approach brings in:
• Evidence Theory
• Dirichlet distributions (a natural prior for distributions over distributions)
Rather than producing one attention distribution, the model represents a whole set of possible attention distributions—parameterized by Dirichlet α.
This gives us something we've never had before:
🔥 A differentiable, local, intrinsic uncertainty measure 👉 Fig 3
This scalar field U(context,token) becomes a lens on the model's epistemic boundary.
And once uncertainty is measurable?
➡️Hallucination becomes an engineering problem—not an inevitability.
⚙️ Two decoding strategies enabled by Credal Transformers
1️⃣ Uncertainty-Guided Sequence Sampling
When a token's uncertainty exceeds threshold τ → the system detects a potential hallucination zone.
It performs:
• controlled rollback
• constrained Monte Carlo resampling
• penalization of all high-uncertainty paths
2️⃣ Confidence-Driven Speculative Decoding
A small draft model generates a candidate sequence; the Credal Transformer evaluates the integral of uncertainty along the whole path.
• If ∫U(t) dt stays low → accept the entire sequence (major speedup)
• If not → cut at the first spike; main model takes over with careful decoding
A dynamic tradeoff between efficiency and reliability.
🏁 What's the contribution?
"We gave the model a way to reflect on its own ignorance."
Credal Transformers let LLMs sense the limits of their knowledge, not just make guesses with forced confidence.
A step toward AI systems that are more reliable, more transparent, and more honest about uncertainty
Just a small milestone on a much longer road—but a meaningful one.
🔗 Original Zhihu post:
zhihu.com/question/19586…
📄 Paper:
openreview.net/forum?id=XTM1B…
#AI #LLM #Tech #NeurIPS #Transformer #Research
