AI world models: the interface after video

A video model gives you a clip. A world model tries to give you a place that responds.

By Ivan Flugelman · 11 min read · Updated 2026-08-18

Midjourney compressed image-making into a prompt box. Video models are compressing cinematic motion into a prompt box. World models may compress spatial imagination into an interactive system.

The interface changes from “make me an image” to “make me a place I can enter.” And eventually to “make me a world I can direct.”

This page is the plain version of what that means, why it matters for anyone who directs images for a living, and where the frontier actually is right now.

The definition

A world model is a generative system that maintains an internal representation of an environment and updates it in response to input, rather than producing a finished output in one pass.

That sounds academic. The practical version is shorter: a video model generates a clip, and a world model tries to generate an environment that can respond.

You do not just watch it. You move through it, direct it, alter it, or inhabit it. A video ends when the clip ends. A world continues, because it has state, space, memory, and consequences.

A video ends when the clip ends. A world continues, because it has state, space, memory, and consequences.

The four properties that make it a world

Whenever a new system gets announced as a world model, these are the four things worth checking. Most of the interesting failures are a missing one of these, not bad image quality.

• State. Something persists between one moment and the next. If you knock a thing over, it stays over.
• Action. Your input is a cause, not a re-roll. Pressing forward moves you forward rather than generating a different scene that happens to look forward-ish.
• Memory. Turn around, come back, and the room is the room you left. This is where most current systems visibly strain.
• Consequence. Actions change what is possible later. This is the hardest one, and the one that turns a walkable environment into something you can tell a story inside.

Where the frontier actually is

This is not speculation in the empty futurist sense. The research and product frontier has already moved from image and video into generated environments. A short map, by approach rather than by hype:

• DeepMind’s Genie line points toward interactive environments generated from prompts and images. Genie 3 has been described as a general-purpose world model generating navigable dynamic worlds with consistency across minutes and visual memory over time.
• GameNGen demonstrated a diffusion model acting like a real-time game engine for a DOOM-like environment. The engine is not simulating the game. The engine is the model.
• Decart’s Oasis explored a Minecraft-like neural world where user input changes the generated frames in real time, which made the latency and memory problems legible to everyone.
• World Labs works on spatially consistent worlds that can be moved through, edited, inhabited, expanded, and exported, which is the branch closest to a production pipeline.
• NVIDIA Cosmos comes at world models from physical simulation, robotics, autonomous vehicles, and synthetic data, where being right matters more than being beautiful.
• Alibaba’s Happy Oyster is framed explicitly around Directing and Wandering: guiding scenes, changing camera behavior, altering characters, and moving through generated worlds.
• Tencent’s HY-World 2.0 takes another path by generating persistent 3D assets, meshes, splats and point clouds, that can be imported into existing production environments.

There is also a second research lineage worth knowing, because it is where the interesting questions live once the rendering problem is solved: open-endedness. Novelty search, MAP-Elites, POET and XLand are all attempts to build systems that keep generating meaningfully new situations instead of converging on one. A world model that never surprises you is a very expensive screensaver.

These systems are early. The limits are obvious and they are all the same limits: duration, control, memory, physics, character consistency, text rendering, interaction, and persistence.

But the direction is clear. First we generated images. Then we generated video. Next we generate spaces. After that, we direct realities.

What this does not mean

It does not mean the craft resets. Every time a new interface arrives, the same story gets told: the tool will replace the skill. It has not happened once yet.

Photography did not end painting. Digital did not end photography. Midjourney did not end art direction, it removed the excuse of technical difficulty and left taste standing in the open.

A world model will do the same thing at a larger scale. If you ask a future world model for a cool cyberpunk city, it will give you a cool cyberpunk city. So will everybody else’s prompt. The default is free, and the default is worthless.

Sources and further reading

1. DeepMind Genie 3
2. GameNGen
3. Decart Oasis
4. World Labs
5. NVIDIA Cosmos
6. Alibaba Happy Oyster
7. Tencent HY-World 2.0

The method is free. The practice is the Academy.

Everything above is the whole method. The World Building Academy is the part a page cannot carry: the fundamentals on video, a full world built end to end, the Render Stack, motion, editing, and the case studies. Or start with the free Codex.

Read next