AGIMiddleLesson 295 min read

Why models suddenly got good

No single genius idea. Mostly a boring answer — more of everything — plus a few tricks that turned a text predictor into something that follows instructions.

🧪 Lab · 15 min📝 Lesson test
Lesson in motion

In 60 seconds

Why models suddenly got good

No single genius idea. Mostly a boring answer — more of everything — plus a few tricks that turned a text predictor into something that follows instructions.

1/4
In simple words
Nobody invented a magic brain. They just made the machine much bigger, fed it much more, and then taught it manners.

Part one: scaling

Researchers found that model quality improves smoothly and predictably as you increase three things together: parameters (size), data (how much text), and compute (how much training). These are the scaling laws.
Predictable is the important word. It meant you could spend a hundred million dollars and know roughly what you would get, which is what turned research into industry.
compute · data · parameters →capabilitywe are somewhere around hereif it keeps goingif it flattens
The whole timeline debate in one picture. Nobody knows which of those two lines we are on, and the honest position is that the data so far is consistent with both.

Part two: the tricks that made it usable

  1. 1

    Transformers (2017)

    An architecture where every word can pay attention to every other word, and which trains efficiently on modern hardware. This unlocked the scale.
  2. 2

    Instruction tuning

    Train on examples of instructions being followed well. Turns a text-continuer into something that answers your actual question.
  3. 3

    Learning from human feedback

    People rate outputs; the model is tuned toward the preferred ones. This is where "helpful, harmless, honest" behaviour is installed — and where a lot of the safety properties live.
  4. 4

    Reasoning at inference time

    Let the model think in steps, check itself, and try again before answering. Buying capability with runtime compute instead of training compute.
  5. 5

    Tools

    Give it a calculator, a search engine, a code runner. Every module in Track B follows from this one step.

What is genuinely surprising about it

  • Emergence. Some abilities appear fairly abruptly at scale — arithmetic, multi-step reasoning — rather than improving smoothly. How real and how sharp this effect is remains debated.
  • Generality was free. Nobody trained these models to write code, translate Tamil and explain photosynthesis. Those came along with predicting text well.
  • The recipe was simple. The core ideas fit on a page. The difficulty is engineering at enormous scale, not conceptual depth.
Watch out
This last point cuts both ways for safety. A simple recipe spreads fast and cheaply, which means capability is not going to stay inside a small number of careful organisations.

The limits people are hitting

LimitStatus
High-quality text dataIncreasingly scarce; synthetic data is the current bet
Compute cost and energyEnormous and growing; a real constraint on who can play
Diminishing returnsEach capability step costs more than the last
ReliabilityScaling improves averages more than it fixes tails

Watch and read more

Deep dive into LLMs like ChatGPTAndrej Karpathy · long form · video
Attention and transformers, visually3Blue1Brown · video
Scaling laws explainedYouTube search

Lab

A scaling curve you fitted yourself.

~15 min

The problem

Train the same tiny model at four sizes on the same data. Plot final loss against parameters on log-log axes. Fit a line and extrapolate to a size you did not train. Then train that size and check your prediction.
Starter codepython
sizes = [1e5, 3e5, 1e6, 3e6]
losses = [run_training(n) for n in sizes]
import numpy as np
a, b = np.polyfit(np.log10(sizes), np.log10(losses), 1)
predict = lambda n: 10 ** (a * np.log10(n) + b)
print(f"predicted loss at 1e7: {predict(1e7):.3f}")

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Your fit predicts a loss of 0.8 at 100B parameters. Give two reasons not to trust it.Reveal
One: you extrapolated four orders of magnitude beyond your data, and scaling laws are empirical fits with a validity range, not physical laws. Two: loss is not capability. A predicted loss says nothing about whether a specific ability appears, and the abilities you care about may not move smoothly with loss at all (Module 29).

Please sign in to continue.

Questions people ask

Will scaling alone reach AGI?

Genuinely contested. Some serious researchers think scale plus tools plus memory gets there. Others think something fundamental is missing — continual learning, causal models, grounding — that no amount of compute supplies. Neither camp can prove it yet.

Is it really "just" statistics?

Technically yes, and the phrase carries less weight than people intend. Predicting text well enough forces the model to build internal structure about the world. Whether that structure counts as understanding is philosophy, and it does not change what the system can do.

Why do bigger models hallucinate less but still hallucinate?

Because scale improves the average and does not remove the underlying mechanism. It is still a probability machine with no built-in access to truth. Grounding it in retrieved sources helps far more than size does.

Are open models a safety problem?

They are a trade-off with real weight on both sides: they enable independent safety research and reduce concentration of power, and they also remove the ability to un-release a capability. Reasonable people disagree, loudly.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in with your phone number to take the test and save your progress