I’ve been seeing “world models” come up a lot recently and finally spent a bit of time trying to understand what people actually mean by it. This is mostly me thinking out loud after watching a couple of talks and reading a bit.
The short version
A world model is an AI that learns how things change over time, so it can predict what happens next
The bit that helped me was realising there’s more going on than input -> output. It’s asking:
- what state are we in?
- what probably happens next?
- what happens if I do something?
If it can answer those well enough, it can simulate outcomes before doing anything.
Why it feels different
Most of the AI we use day to day is reactive. You give it something, it gives you something back.
A world model predicts what might happen, tries different options, then chooses an action. That’s why it feels closer to planning than autocomplete.
The mental model that helped
The thing that made it click for me was games. In chess you look at the current board and then think:
- if I do this
- they’ll probably do this
- then I can do this
You’re simulating a few steps ahead, and that’s basically the idea.
flowchart LR
A[Current state] --> B[World model]
B --> C[Predict possible futures]
C --> D[Choose action]
D --> E[New state]
E --> B
A more concrete example
Driving is a good one. You could build a system that maps camera input -> steering output directly, but that’s quite brittle.
A world model approach would learn how cars, roads and movement behave, simulate where the car will be in 2 seconds if it turns here, and pick the safest option.
What text alone misses
predicting the next word is not the same as understanding the world
If you only train on text, you never see objects, never interact with anything and never experience cause and effect. You end up with something that’s very good at language but isn’t grounded in reality.
The data comparison is wild
Large language models are trained on ~10¹⁴ bytes of text, while a 4-year-old has seen ~10¹⁴ bytes of visual data
That’s roughly the same amount of data. The model gets good at text and the child understands the physical world, so the difference comes down to what you learn from and how you learn it.
Another idea I found useful: you probably don’t want to predict every tiny detail of the future, just a simplified version that keeps the bits that matter.
Where this gets interesting (for me)
The obvious applications are robotics, self-driving and games, but the one I keep thinking about is coding. Right now AI coding tools mostly read code and predict the next change.
Imagine one that could simulate the system and ask “if I change this, what breaks?” or “if I move this, what improves?” That would be a pretty big shift from autocomplete.
Reality check
The full version of this, proper world models plus planning plus abstraction, doesn’t really exist yet. There are pieces of it, but the whole thing is still very much unsolved.
The talk
If you want to go deeper, this is Yann LeCun’s talk at Harvard CMSA on self-supervised learning, JEPA and world models: