Quantifying the algorithmic improvement from reasoning models
epochai.substack.comReasoning models were as big of an improvement as the Transformer, at least on some benchmarks
Evidence that Recent AI Gains are Mostly from Inference-Scaling — Toby Ord
In the last year or two, the most important trend in modern AI came to an end. The scaling-up of computational resources used to train ever-larger AI models through next-token prediction ( pre-training ) stalled out. Since late 2024, we’ve seen a new trend of using reinforcement learning (RL) in the