François Chollet (@fchollet)

To address the limits of deep learning and avoid stalling, the field of AI started by applying patch (1), which started being demoed 9 months later in December 2024 and has now become completely ubiquitous. However, long term, it is simply inevitable that AI will move to patch (2). <span class="quot...

The "aha" moment when I realized that curve-fitting was the wrong paradigm for achieving generalizable modeling of problems spaces that involve symbolic reasoning was in early 2016. I was trying every possible way to get a LSTM/GRU based model to classify first-order logic statements, and each new ...

There is an interesting disconnect between the ability of models to successfully execute precise instructions (improving incredibly fast) and their ability to make sound decisions when faced with something not covered by the instructions (stagnating for a while)....

The quantity of code that devs ship has roughly 10xed. But net developer productivity (value created by unit of time) is only up by a bit, if at all. Part of it is that the additional code is solving more incremental problems. A bigger part is that the new code is creating problems of its own....

The keyword here isn't "understand". It's "novel". You "truly understand" a thing if your model of it lets you make sense of every possible instance of the thing, including those that are very far from what you've seen before (extreme generalization). You "somewhat understand" the thing if you can...

Big confusion in AI right now is folks believing that "reasoning" is a task, when it is actually a task-solving method....

When a model gives you the right answer to a reasoning question, you can't tell whether it was via memorization or via reasoning. A simple way to tell between the two is to tweak your question in a way that 1. changes the answer, 2. requires some reasoning to adapt to the change. If you still get t...

In general there are two different kinds of methodology to produce progress in any science or engineering field... both are important and can lead to transformative progress. There's the "Edison way" where you brute-force a large predefined design space and you keep what works, without necessarily ...

People ask me, "didn't you say before ChatGPT that deep learning had hit a wall and there would be no more progress?" I have never said this. I was saying the opposite (that scaling DL would deliver). You might be thinking of Gary Marcus. My pre-ChatGPT position (below) was that scaling up DL woul...