Connectome suggests brain’s synaptic weights follow heavy-tailed...

@EvaYiXie
Eva Yi Xie@EvaYiXie
7 views Nov 01, 2025 ~2 min read
Advertisement
1
Connectome suggests brain’s synaptic weights follow heavy-tailed distributions, yet most analyses of RNNs assume Gaussian connectivity. 

🧵⬇️ Our @AllenInstitute #NeurIPS2025 paper shows heavy-tailed weights can strongly affect dynamics, trade off robustness + attractor dimension.

openreview.net/forum?id=J0SbY…
Media image
2
1/ Setup: With @Stefan_Mihalas and Lukasz Kusmierz, We study RNNs with weights drawn from biologically plausible Lévy alpha-stable distributions, generalizing the Gaussian distribution to heavy tails.
3
@Stefan_Mihalas 2/ 🔎Result 1: While mean-field theory for the infinite system predicts ubiquitous chaos, our analysis reveals *finite-size* RNNs have a sharp transition between quiescent & chaotic dynamics. 

We theoretically predict the gain of transition and validated it through simulations.
Media image
4
@Stefan_Mihalas 3/ 🔎Result 2: Compared to Gaussian networks, we found finite heavy-tailed RNNs exhibit a broader gain regime near the edge of chaos: a *slow* transition to chaos. 🐢
Media image
5
@Stefan_Mihalas 4/ (Side note: The computational benefit of being near the edge of chaos is well established for both feedforward and recurrent neural networks. We validate in Appendix L this indeed translates to improved info processing in simple reservoir-computing tasks. 🤖🧠)
6
@Stefan_Mihalas 5/ ‼️Result 3: However, this robustness of slow transition comes with a tradeoff ↔️: heavier tails reduce the Lyapunov dimension of the network attractor, indicating lower effective dimensionality.
Media image
7
@Stefan_Mihalas 6/ Conclusion: Our results reveal a biologically aligned tradeoff between the robustness of dynamics and the richness of neural activity. Our results provide a tractable framework for understanding dynamics in realistically sized, heavy-tailed neural circuits.
8
@Stefan_Mihalas 7/ For more details, implications of our results to neuroscience 🧠 and machine learning 🤖, + exciting future directions, please check out our full paper or visit our poster at #NeurIPS2025: 

🔗OpenReview: openreview.net/forum?id=J0SbY…
📍Code: github.com/AllenInstitute…
9
@Stefan_Mihalas 8/ @tyrell_turing’s group recently shows brain-like learning with exponentiated gradients naturally gives rise to log-normal connectivity distributions—our results offer a theoretical perspective that elucidates the dynamical consequences of these heavy-tailed structures.
10
9/9 Lastly, we thank the colleagues @AllenInstitute and @CosyneMeeting for their insightful feedback on an early version of this work! Happy to chat: evayixie@princeton.edu; lukasz.kusmierz@alleninstitute.org.
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement