People ask me, "didn't you say before ChatGPT that deep learning...
François Chollet@fchollet
137 views
Aug 22, 2025
~1 min read
Advertisement
1
People ask me, "didn't you say before ChatGPT that deep learning had hit a wall and there would be no more progress?"
I have never said this. I was saying the opposite (that scaling DL would deliver). You might be thinking of Gary Marcus.
My pre-ChatGPT position (below) was that scaling up DL would keep delivering better and better results, and *also* that it wasn't the way to AGI (as I defined it: human-level skill acquisition efficiency).
This was a deeply unpopular position at the time (neither AI skeptic nor AGI-via-DL-scaling prophet). It is now completely mainstream.
I have never said this. I was saying the opposite (that scaling DL would deliver). You might be thinking of Gary Marcus.
My pre-ChatGPT position (below) was that scaling up DL would keep delivering better and better results, and *also* that it wasn't the way to AGI (as I defined it: human-level skill acquisition efficiency).
This was a deeply unpopular position at the time (neither AI skeptic nor AGI-via-DL-scaling prophet). It is now completely mainstream.
François Chollet@fchollet
Two perfectly compatible messages I've been repeating for years:
1. Scaling up deep learning will keep paying off.
2. Scaling up deep learning won't lead to AGI, because deep learning on its own is missing key properties required for general intelligence.
1. Scaling up deep learning will keep paying off.
2. Scaling up deep learning won't lead to AGI, because deep learning on its own is missing key properties required for general intelligence.
2
People also ask, "didn't you say in 2023 that LLMs could not reason?"
I have also never said this. I am on the record across many channels (Twitter, podcasts...) saying that "can LLMs reason?" was not a relevant question, just semantics, and that the more interesting question was, "could they adapt to novel tasks beyond what they had been trained on?" -- and that the answer was no.
Also correct in retrospect, and a mainstream position today.
I have also never said this. I am on the record across many channels (Twitter, podcasts...) saying that "can LLMs reason?" was not a relevant question, just semantics, and that the more interesting question was, "could they adapt to novel tasks beyond what they had been trained on?" -- and that the answer was no.
Also correct in retrospect, and a mainstream position today.
3
I have been consistently bullish on deep learning since 2013, back when deep learning was maybe a couple thousands of people.
I have also been consistently bullish on scaling DL -- not as a way to achieve AGI, but as a way to create more useful models.
I have also been consistently bullish on scaling DL -- not as a way to achieve AGI, but as a way to create more useful models.
4
Back in 2017 my book had an entire chapter on generative AI, including language modeling and image generation. I wrote that content in 2016 and early 2017. This was some of the earliest textbook content that covered generative AI.
All the way back in 2014 I was convinced that AI would one day become a major source of cultural content creation -- which was a completely outlandish position at the time.
All the way back in 2014 I was convinced that AI would one day become a major source of cultural content creation -- which was a completely outlandish position at the time.