Thread Navigator

I heard today that second order optimization algorithms are now...

@doodlestein
Jeffrey Emanuel@doodlestein
33 views Feb 08, 2025 ~1 min read
1
I heard today that second order optimization algorithms are now making a comeback in deep learning, with the Shampoo optimizer outperforming Adam in some evaluation. It brought back a flood of memories for me from 2012 when I became extremely excited about “Hessian Free” methods.
2
I looked through my old email messages and found a long correspondence from 2012 between me and James Martens (now at Google Brain), the inventor of the Hessian Free approach. It’s so mind blowing thinking of how absolutely quaint everything sounds— the number of params, layers.
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • LinkedIn & Instagram Carousel Maker
Create Free Account

Includes 7-day Premium trial

More from @doodlestein

Thinking Traces are Better for Understanding

25 views•2 months ago

The beauty and utility of linear algebra, courtesy of the new ChatGPT image mode...

28 views•3 months, 3 weeks ago
View all threads from @doodlestein

Save Threads to Notion

Upgrade to Premium to automatically save unrolled threads directly to your Notion workspace. Includes unlimited saves, visual editor, and ad-free experience.

Highlight Key Tweets

Upgrade to Premium to highlight individual tweets in any thread, then export just the highlights to Notion — like Readwise, for threads.

Save to Notion

Pick categories to organize this thread in your Notion database.

Sign Up to Highlight Tweets

Create a free account to get a 7-day Premium trial, including the ability to highlight key tweets in any thread.

Sign Up to Save to Notion

Create a free account to get a 7-day Premium trial, including saving unrolled threads directly to your Notion workspace.