Aarav Wattal (@aaravwattal)

As we scale to trillion parameter models and gigawatt-scale training, we don't do interesting work on just one chip anymore. Everything interesting happens across thousands of them, and moving data between those chips has become a fundamental bottleneck of both training and inference. I spent this ...