✨ Visual Editor

close

palette Canvas & Background

Gradient:arrow_forward
Text Color:
135°

style Card Style

40px
16px

text_fields Typography

16px
Jeffrey Emanuel
@doodlestein
I heard today that second order optimization algorithms are now making a comeback in deep learning, with the Shampoo optimizer outperforming Adam in some evaluation. It brought back a flood of memories for me from 2012 when I became extremely excited about “Hessian Free” methods.
Jeffrey Emanuel
@doodlestein
I looked through my old email messages and found a long correspondence from 2012 between me and James Martens (now at Google Brain), the inventor of the Hessian Free approach. It’s so mind blowing thinking of how absolutely quaint everything sounds— the number of params, layers.
Generated by Thread Navigator
100%
view_carousel Carousel Studio NEW
Press + S to quick-export