François Chollet (@fchollet)
The quantity of code that devs ship has roughly 10xed. But net developer productivity (value created by unit of time) is only up by a bit, if at all. Part of it is that the additional code is solving more incremental problems. A bigger part is that the new code is creating problems of its own....
The feed is the enemy of curiosity. It decides what you see. I like to decide myself where to go next, like with Wikipedia link surfing....
The keyword here isn't "understand". It's "novel". You "truly understand" a thing if your model of it lets you make sense of every possible instance of the thing, including those that are very far from what you've seen before (extreme generalization). You "somewhat understand" the thing if you can...
Big confusion in AI right now is folks believing that "reasoning" is a task, when it is actually a task-solving method....
When a model gives you the right answer to a reasoning question, you can't tell whether it was via memorization or via reasoning. A simple way to tell between the two is to tweak your question in a way that 1. changes the answer, 2. requires some reasoning to adapt to the change. If you still get t...
In general there are two different kinds of methodology to produce progress in any science or engineering field... both are important and can lead to transformative progress. There's the "Edison way" where you brute-force a large predefined design space and you keep what works, without necessarily ...
People ask me, "didn't you say before ChatGPT that deep learning had hit a wall and there would be no more progress?" I have never said this. I was saying the opposite (that scaling DL would deliver). You might be thinking of Gary Marcus. My pre-ChatGPT position (below) was that scaling up DL woul...
As a parent, you end up watching a fair amount of cartoons, and I've found that old-school, hand-drawn animation has soul, while CG animation, no matter how pretty, doesn't. I think this has to do with the fact that hand-drawn frames directly convey an accumulation of human-made choices. Each choic...