Ask an AI assistant something forty messages into a conversation and it may answer as if the first twenty never happened. Paste a long document and it might summarise the beginning and end while missing the part in the middle you cared about.
Both behaviours come from the same limit: the context window.
It is the single most useful concept for predicting what these systems will do, and it is almost always explained as "the model's memory" — which is wrong, and is the reason the behaviour stays confusing.
The One-Sentence Definition
A context window is the maximum amount of text a model can consider in a single request.
Not per conversation. Not per day. Per request. Everything the model takes into account — your question, the system instructions, the entire conversation so far, any files you attached, and the answer it is about to write — has to fit inside that one budget.
Think of it as a desk, not a filing cabinet. However large the desk, you can only work on what is physically on it right now.
It Is Measured in Tokens, Not Words
Models do not read words. They read tokens — chunks of text that are often a whole word but can be a fragment. "Cat" is one token; "unbelievable" might split into "un", "believ" and "able".
A workable rule of thumb for English: one token is about ¾ of a word, so 1,000 tokens is roughly 750 words. Code, unusual names and non-English text tokenise less efficiently and consume more tokens per character.
This matters because every limit and every price is quoted in tokens. If you want a proper look at how text becomes tokens and what the model does with them, see how ChatGPT actually works and how LLMs are trained.
How Big They Got
Windows have grown enormously. Early production models worked with a few thousand tokens; the frontier has converged on a million.
A million tokens is around 750,000 words, or a shelf of books. In practice almost no chat conversation approaches it; the workloads that do are long documents, large codebases and agents that run for hours.
Note the word advertised. The number a vendor publishes is the size of the container, not a promise about how well the model uses all of it — a distinction we come back to below.
The Thing Everyone Gets Wrong: It Is Not Memory
This is the misconception worth spending a minute on, because everything else follows from it.
When you send message forty in a chat, the model is not remembering messages one through thirty-nine. The application re-sends all of them, every time. The underlying API is stateless — it has no record of your previous request.
Three consequences follow, and they explain most confusing AI behaviour:
Nothing is learned. The model's weights do not change because you told it your name. Correct it, and the correction lasts exactly as long as it stays inside the window being re-sent.
Long conversations get more expensive per message, not less. Every turn re-sends a longer history. That compounding is worked through in detail in our explainer on how the agent loop works, where a 20-turn task bills far more than the content it carries.
"Memory" features are built on top, not inside. When an assistant appears to remember you between sessions, ordinary software is storing notes and pasting relevant ones back into the window. Useful, but a different mechanism from the model itself.
What Actually Happens When You Run Out
Nothing dramatic, which is exactly the problem.
Most consumer chat products handle overflow silently. Older turns are summarised or dropped so the request still fits, and you receive a confident answer built on a partial record. The model cannot tell you it has forgotten something, because from its point of view that something was never there.
The practical tell: if an assistant starts contradicting a decision you made much earlier in a long conversation, you have probably fallen out of the window. Restating the constraint fixes it — not because the model is being reminded, but because you are putting the fact back on the desk.
Why a Bigger Window Is Not Automatically Better
Here is the part vendor announcements leave out: models do not use every position in their window equally well.
The reference result is "Lost in the Middle" by Liu and colleagues at Stanford and the University of Washington, published in Transactions of the ACL. Testing multi-document question answering and key-value retrieval across contexts of 10, 20 and 30 documents, they found a U-shaped performance curve: accuracy is highest when the relevant information sits at the beginning or the end of the input, and drops substantially when it sits in the middle.
The degradation is not marginal. Moving the needed information from the edges to the middle cost more than 30% of performance — and it affected models explicitly built for long contexts.
The cause is architectural rather than a bug: attention mechanisms combined with positional encodings such as RoPE produce a long-term decay effect that emphasises the start and end of a sequence at the expense of the middle.
Two practical rules follow:
- Put the important material at the start or the end of a long prompt. Not buried in the middle.
- A larger window is permission to include more, not an instruction to. Filling a million tokens with marginally relevant text can make an answer worse, not better.
What It Costs to Fill One
Windows are billed per token, so the price of "just paste everything in" is calculable rather than theoretical.
Using Anthropic's published input rates, filling a model's window once costs:
| Model | Window | Cost to fill it once |
|---|---|---|
| Claude Haiku 4.5 | 200K | $0.20 |
| Claude Sonnet 5 | 1M | $2.00 |
| Claude Opus 5 | 1M | $5.00 |
| Claude Fable 5 | 1M | $10.00 |
Now recall that the API is stateless. In a long agent run, a near-full window is re-sent on every turn, so those are per-request figures, not per-task. This is why prompt caching exists: cached input tokens are billed at roughly a tenth of the normal rate, which turns an unaffordable pattern into a routine one.
Three Different Ways Systems Cope
"Running out of context" has three distinct engineering answers, and they are frequently confused with one another.
Compaction — summarise. When a conversation approaches the limit, the system replaces older turns with a summary and carries that forward instead. Anthropic offers it both server-side and in its SDKs, with a configurable token threshold that decides how early it kicks in. You keep the gist and lose the detail.
Context editing — delete. Rather than summarising, this clears material that has served its purpose — typically old tool results or intermediate reasoning. It is the right tool when the bulk is machine output nobody needs again, and it is not the same thing as compaction.
Retrieval — fetch only what is needed. Instead of loading everything and hoping, store the corpus externally and pull in only the passages relevant to the current question. This is retrieval-augmented generation, and it remains the standard answer for bodies of knowledge far larger than any window.
The deciding question is what the bulk is. Conversation history compacts well. Tool output is usually better deleted. A document library belongs in retrieval.
Output Length Is a Separate Limit
A common and expensive confusion: a million-token context window does not mean the model can write a million tokens.
Input and output are governed by different caps. Current frontier Claude models pair their 1M-token context window with a maximum output of 128K tokens — large, but roughly an eighth of the input capacity. Ask for a book-length document in one response and you will hit the output ceiling long before the context limit, usually as an answer that stops mid-sentence.
If you are checking a specific model's limits programmatically, the Models API exposes the context window as max_input_tokens and the output cap as max_tokens — two separate fields, which is the clearest statement that they are two separate things.
The Bottom Line
A context window is the working space for one request, measured in tokens, containing everything the model sees — and it is refilled from scratch every single turn.
Three things follow that are worth carrying around:
- It is not memory. Nothing persists unless software re-sends it, which is also why long conversations get more expensive rather than cheaper.
- Bigger is not automatically better. Information in the middle of a long context is measurably less likely to be used, so position matters as much as inclusion.
- Running out is silent. No error appears; the answer just gets built from less.
Understand the desk, and most of the confusing behaviour stops being mysterious. For more on how these systems work underneath, follow our LLM hub.
Frequently Asked Questions
What is a context window in simple terms?
It is the maximum amount of text a model can consider in a single request, measured in tokens. It has to hold everything at once — the system instructions, your question, the whole conversation so far, any attached files, and the response being generated. Think of it as a desk rather than a filing cabinet: however large it is, the model can only work with what is on it at that moment.
Is a context window the same as memory?
No, and this is the most common misconception. The API is stateless, meaning the model keeps no record of previous requests. When you send a new message, the application re-sends the entire conversation. Nothing is learned and nothing persists on its own. Assistant features branded as "memory" are separate systems that store notes externally and paste relevant ones back into the window.
How many words is a million tokens?
Roughly 750,000 words, or about 1,500 pages — a shelf of books. The rough conversion for English is that one token is about three-quarters of a word. Code, unusual proper nouns and non-English text tokenise less efficiently and use more tokens for the same visible length.
Does a bigger context window mean better answers?
Not automatically. Research from Stanford and the University of Washington, published in Transactions of the ACL, found a U-shaped performance curve: models use information best at the beginning and end of a long input and noticeably worse in the middle, with degradation exceeding 30% when key material moves to the middle. A bigger window is permission to include more, not a guarantee the model will use it all well.
What happens when a conversation exceeds the context window?
Usually nothing visible. Most products silently summarise or drop the oldest turns so the request still fits, then return a confident answer built on a partial record. There is no error message. The practical symptom is an assistant contradicting a decision made much earlier in a long conversation — restating the constraint puts it back in the window and resolves it.
Why do long AI conversations get more expensive?
Because the whole history is re-sent on every turn. Message forty costs more than message four, since it carries thirty-nine previous messages along with it. Prompt caching is the standard mitigation — cached input is billed at roughly a tenth of the normal rate — which is what makes long agent runs economically viable.
Can a model output as much text as its context window holds?
No. Input and output have separate limits. Current frontier Claude models combine a 1M-token context window with a 128K-token maximum output, so the output ceiling is roughly an eighth of the input capacity. Requests for extremely long documents typically stop mid-sentence at the output cap, not the context limit.
Sources
- Lost in the Middle: How Language Models Use Long Contexts — Transactions of the ACL
- Lost in the Middle (Liu et al., Stanford) — full paper PDF
- Anthropic — Models overview and context window reference
- Anthropic — Prompt caching
- Anthropic — Context editing and compaction
- Elvex — AI model context window comparison 2026: advertised vs real
