Almost every explanation of AI video generation starts in the wrong place. These models do not generate a picture, then another picture, then stitch them together. If they did, nothing would stay consistent between frames.
What they actually do is treat a video as a single object in space and time, compress it, chop it into blocks, and then learn to reverse the process — starting from pure noise and removing it, step by step, until a clip appears.
That architecture is worth understanding on its own terms, and it has become more interesting since OpenAI discontinued Sora in 2026, about six months after launching it. The technology did not fail. The product did, for reasons that say a lot about where generative video is heading.
First, What a Video Model Is Not
A text model like the one behind ChatGPT works by predicting tokens — fragments of words — one after another.
A naive video model would do the equivalent: generate frame one, then frame two conditioned on frame one, and so on. This approach exists, and it fails in a specific way. Small errors compound. A character's shirt changes colour, a hand gains a finger, a background object drifts. Nothing forces frame 90 to remember frame 1.
Modern video generation avoids the problem by never working frame-by-frame at all. The whole clip is generated at once, as one connected block of space and time.
Stage One: Compress the Video
Raw video is enormous. A few seconds at high resolution is millions of pixel values, and no model can attend across that directly.
So the first component is a video compression network, which squeezes the video into a lower-dimensional latent space — a compact numerical representation that preserves both spatial detail and temporal information, and discards what the model does not need. All the expensive work then happens in this smaller space, and only the final result is expanded back out.
This is the same insight that made image generation practical: don't operate on pixels, operate on a compressed representation of pixels.
Stage Two: Cut It into Spacetime Patches
This is the step that makes video generation work, and it is the genuinely clever part.
The compressed video is divided into spacetime patches — small blocks that each cover a patch of image area across several frames. Each patch therefore carries both appearance (what is here) and motion (how it is changing).
These patches are then fed to the model as tokens, exactly as a language model receives word fragments. OpenAI's own technical write-up makes the parallel explicit: patches are to video what tokens are to text.
The payoff is enormous. Once video is a sequence of tokens, it inherits everything already known about training transformers — including the scaling behaviour proven in language models. It also means a single model can be trained on video and still images, at variable durations, resolutions and aspect ratios, rather than needing one model per format.
Stage Three: Denoise from Pure Noise
Now the diffusion part, which runs in the opposite direction to intuition.
During training, the model is shown real video in latent form with varying amounts of random noise added, and learns to predict what noise was added. Do that enough times and it learns, in effect, what makes video look like video.
To generate, the process runs backwards. The model starts with a latent full of pure noise — the visual equivalent of the static on a dead television channel — and removes a little of it at each step, guided at every step by the text prompt. After many steps, what remains is a coherent latent video, which the decoder expands into viewable frames.
This is why the architecture is called a diffusion transformer: diffusion describes the noise-removal process, transformer describes the network doing the removing. It is a different family of model from a large language model, though both rest on the same attention mechanism described in our guide to machine learning.
What These Models Do Well — and What Breaks
By 2026 the strengths and failures have become predictable, which is more useful than any highlight reel.
The pattern behind the failures is consistent. These models have no explicit representation of physics. Nothing in the architecture encodes gravity, momentum, rigid-body dynamics, or the idea that an object continues to exist when the camera looks away.
Research on the current generation finds that both vision-language and video generation models lean heavily on surface-level visual cues and routinely violate basic constraints on balance, torque and inertia. In practice that produces the familiar artefacts: objects that float or slide rather than fall, liquids and cloth that move wrongly, and things that appear or vanish between shots.
This is a close cousin of the problem behind AI hallucinations in text models. A system trained to produce plausible output has learned what plausible looks like, not what is true.
The 2026 Landscape: Audio Became Standard
The most consequential capability change was not resolution or length. It was sound.
Sora 2, released on 30 September 2025, generated dialogue, sound effects and ambient audio together with the picture. That mattered because it moved a single generation from silent B-roll to something resembling finished footage.
Native audio is now standard across the leading models — Google's Veo 3.1, Kling, Seedance and LTXV all generate synchronised sound, with Veo's implementation extending to foley matched to on-screen action. The competitive frontier has moved from "can it make a clip" to controllability: consistent characters, camera direction, and editing across shots.
Why OpenAI Shut Sora Down
In late March 2026, OpenAI announced it was discontinuing Sora. The app and website closed on 26 April 2026, and the API is scheduled to end on 24 September 2026. The app had been live roughly six months.
The numbers explain most of it. Downloads peaked at 3,332,200 in November 2025 and had fallen to 1,128,700 by February 2026 — a decline of roughly two thirds. Lifetime in-app purchase revenue was around $2.1 million. For a company whose ChatGPT product has roughly 900 million weekly active users, that is not a rounding error so much as a different unit of measurement.
Set that against cost. Generating video is the most compute-intensive consumer AI workload in wide deployment — vastly more expensive per output than text or images. A product with falling engagement and a few million dollars of direct revenue cannot justify that allocation of GPUs.
There were also genuine liability problems. Sora's likeness feature — originally "cameos", renamed "characters" after Cameo Inc. brought a trademark suit — made realistic deepfakes trivial to produce. Unauthorised videos of public figures including Martin Luther King Jr., Robin Williams and Sam Altman circulated, prompting complaints from families. Users also generated copyrighted characters freely.
OpenAI's own farewell was brief: "We're saying goodbye to the Sora app. To everyone who created with Sora, shared it, and built community around it: thank you."
What Survives: World Models
Here is the part that most coverage missed, and it matters for anyone trying to read the direction of travel.
OpenAI did not abandon video generation research. It closed the consumer product and redirected compute toward enterprise tools and a more unified ChatGPT experience — while continuing Sora as a research programme focused on world models, with the stated ambition of automating parts of the physical economy.
That is a coherent strategy rather than a retreat, and the clue was in the title of OpenAI's original technical write-up: Video generation models as world simulators. If a model can predict how a scene evolves, the interesting application is not entertainment. It is robotics, simulation and planning — systems that need to anticipate what happens next in the physical world.
Which returns us to the physics failures above. A model that floats objects and forgets what is behind the camera is a serviceable video toy and a useless world simulator. Fixing that is the actual research agenda, and it is a much harder problem than making clips look better. Our overview of the AI landscape covers where this sits among the other frontier bets.
How to Tell If a Video Was Generated
Two mechanisms exist, and both have real limits worth knowing.
Content Credentials, under the C2PA specification, attach cryptographically signed provenance metadata to a media file, recording what created it and what was done to it afterwards.
Invisible watermarking, such as Google's SynthID, embeds an imperceptible signal into the output at the moment of generation, which survives some editing and re-encoding.
The limitation is subtler than "watermarks can be stripped". Researchers have shown that ordinary editing pipelines can produce authenticated fakes with no cryptographic compromise at all: a file can carry a valid C2PA manifest asserting only that a human edited it, while simultaneously carrying a watermark identifying it as synthetic. The signals contradict each other, and both are technically correct.
The practical conclusion is that provenance metadata is useful evidence and not proof. Absence of a credential means nothing, because most real footage has none either.
The Bottom Line
AI video generation compresses video into a latent space, cuts it into spacetime patches that function as tokens, and uses a diffusion transformer to denoise its way from static to a clip. That design is what let video inherit the scaling results from language models, and it is why quality improved so quickly.
What has not been solved is physics. These systems learned what video looks like, not how the world works, and every characteristic failure — floating objects, vanishing props, impossible liquids — traces back to that gap.
OpenAI's decision to close Sora while keeping the research is the clearest statement anyone has made about where the value is thought to lie: not in generating clips for consumers, but in models that can predict the physical world well enough to act in it. Follow the story through our OpenAI hub.
Frequently Asked Questions
How does AI video generation actually work?
In three steps. A compression network squeezes video into a lower-dimensional latent space that preserves space and time. That latent video is divided into spacetime patches — blocks covering an image area across several frames — which act as tokens. A diffusion transformer then starts from pure noise and removes it step by step, guided by the text prompt, before a decoder expands the result back into frames.
What are spacetime patches?
Small blocks of a compressed video that span both an area of the frame and a short span of time, so each one encodes appearance and motion together. They serve the same role in video models that word tokens serve in language models. This is what allows a single model to handle video and images at varying durations, resolutions and aspect ratios, and to benefit from transformer scaling.
Why does AI video get physics wrong?
Because nothing in the architecture represents physics. The models learn statistical patterns in how video looks, with no explicit encoding of gravity, momentum, rigid-body dynamics or object permanence. Research on current models finds they rely on superficial visual cues and routinely violate constraints on balance, torque and inertia — which is why objects float, slide, or disappear between shots.
Is Sora still available?
No. OpenAI announced the discontinuation in late March 2026. The Sora app and website closed on 26 April 2026, and the API is scheduled to shut down on 24 September 2026. OpenAI advised users to export their content before the deadlines. Sora continues internally as a research programme on world models rather than as a consumer product.
Why did OpenAI shut down Sora?
A combination of weak economics and real liability. App downloads fell from a peak of about 3.3 million in November 2025 to roughly 1.1 million by February 2026, while lifetime in-app purchase revenue was around $2.1 million — negligible against the cost of the most compute-intensive consumer AI workload in deployment. The likeness feature also enabled realistic deepfakes of public figures, and users generated copyrighted characters freely.
Which AI video models are worth knowing about in 2026?
The leading models all now generate synchronised audio alongside the picture, which is the capability that changed most recently. Google's Veo 3.1, Kling, Seedance and LTXV are the significant players, with Veo notable for generating dialogue, ambience and foley matched to on-screen action. Competition has shifted from raw clip quality to controllability — consistent characters, camera direction, and continuity across shots.
Can you detect AI-generated video?
Only partially. C2PA Content Credentials attach signed provenance metadata to a file, and watermarks such as SynthID embed an imperceptible signal at generation. But researchers have demonstrated authenticated fakes produced through ordinary editing, where a file carries both a valid manifest claiming human editing and a watermark marking it synthetic. Treat provenance data as evidence rather than proof, and remember that most genuine footage carries no credential either.
Sources
- OpenAI — Video generation models as world simulators (Sora technical report)
- OpenAI — What to know about the Sora discontinuation
- TechCrunch — OpenAI's Sora was the creepiest app on your phone; now it's shutting down
- NBC News — OpenAI is shutting down its Sora video-creation app
- The Decoder — OpenAI sets two-stage Sora shutdown, app closing April 2026 and API in September
- arXiv — Sora: a review on background, technology, limitations and opportunities
- C2PA — Content Credentials specification explainer
