Jack Morris (@jxmnop)

"there's nothing interesting on arxiv these days!" - the words of an uncurious mind i have personally been blown away by the volume of interesting papers posted over the last few months, and eagerly following daily digests here are some papers i enjoyed the most: - Pre-training under infinite c...

curious about the training data of OpenAI's new gpt-oss models? i was too. so i generated 10M examples from gpt-oss-20b, ran some analysis, and the results were... pretty bizarre time for a deep dive 🧵 ...

new paper from our work at Meta! **GPT-style language models memorize 3.6 bits per param** we compute capacity by measuring total bits memorized, using some theory from Shannon (1953) shockingly, the memorization-datasize curves look like this: ___________ / / (🧵)...