Three effects people lump together
Context drift is a label, not a single mechanism. The phrase covers three effects, each with a different cause and a different test.
The first effect is position inside one long prompt, often called “lost in the middle.” The second is quality change across the turns of a simulated multi-turn conversation. The third is growth: each turn adds to the transcript that the model reads next time.
| Effect | What was measured | Limit |
|---|---|---|
| Position inside one prompt | Accuracy by where the relevant text sat in one long input, on 2023 models | Not chat turns |
| Multi-turn degradation | Conversations split across turns, compared with one full message, using a simulated LLM user | A simulation; reported averages differ (39% and 35%) |
| Transcript growth | Tokens re-sent with each turn in a chat that keeps its history | Token counts are estimates |
Position inside one long prompt: lost in the middle
Liu et al. measured where a relevant passage sits inside one long input. Their abstract says that “performance is often highest when relevant information occurs at the beginning or end of the input context,” and that performance “significantly degrades when models must access relevant information in the middle of long contexts.” The paper calls the resulting pattern “a U-shaped performance curve.” (Liu et al., Lost in the Middle)
The models were MPT-30B-Instruct, LongChat-13B (16K), GPT-3.5-Turbo and Claude-1.3, tested on multi-document question answering and key-value retrieval. For GPT-3.5-Turbo, closed-book accuracy with no documents was 56.1%. Placing the relevant document in the middle of the context fell below that figure. Claude-1.3 is an older model, so this result does not describe current Claude models.
What this tells you: where a fact sits inside one prompt can change how well a model uses it. What it does not tell you: that an instruction you typed at turn 12 of a chat was lost because it sat in the middle. The study measured one prompt, not turns, and it used 2023 models.
Quality across turns: the Laban et al. simulation
Laban et al. compared conversations where the task arrived in pieces over several turns with conversations that gave the full task in one message. The study reports more than 200,000 simulated conversations, 15 LLMs and six generation tasks. The simulated user was itself an LLM, and the authors state that “the conversations we simulate are not representative of human-AI conversations.” (Laban et al., LLMs Get Lost In Multi-Turn Conversation)
The headline figure is “an average drop of 39% across six generation tasks.” The same paper’s Figure 1 caption gives −35% for the same 15 LLMs. The text checked for this guide does not reconcile the two numbers. When you cite it, give both and say they differ.
The paper splits the loss in two. The abstract names it as “a minor loss in aptitude and a significant increase in unreliability.” In the sharded setting, the reported averages are a 16% drop in aptitude and a 112% increase in unreliability. This guide does not restate the paper’s formal definitions. Read aptitude as the best a model can do and unreliability as how much its results vary, then check the paper before you quote either term.
The authors give a mechanism: “LLMs often make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely.” In their experiments, providing all the information at once was “the only effective method to improve reliability.” That finding describes the simulated setting. It does not guarantee the same result in your chat.
Plain growth: the transcript is sent again each turn
Each request to a model API is separate, so a chat sends its earlier messages again with every new one. OpenAI’s documentation says each text generation request is “independent and stateless,” and that “all previous input tokens for responses in the chain are billed as input tokens in the API.” (OpenAI, Conversation state)
Anthropic’s documentation describes the same accumulation: each user message and assistant response adds to the context window, and “previous turns are preserved completely.” (Anthropic, Context windows) Chroma’s technical report tested 18 LLMs and found that “model performance consistently degrades with increasing input length” in its controlled tasks. (Chroma, Context Rot) Anthropic calls this pattern context rot.
Three limits matter here. First, the window holds output and reasoning as well as input: OpenAI says the limit “includes input, output, and reasoning tokens,” and output beyond it “may be truncated.” Second, chat apps can manage history differently. Anthropic’s documentation notes that claude.ai “can also manage the context window on a rolling ‘first in, first out’ basis.” Third, a token count before sending is an estimate. Anthropic’s token-counting page says “The token count is an estimate.” (Anthropic, Token counting)
Spot drift yourself with three checks
Run these checks in the same chat and model, and note what you see. Each one gives a warning sign, not proof of a cause. The Context Drift Heatmap automates the rule check and the length watch from the three checks below. The restatement check, and the re-ask check that compares two answers, stay manual.
- Instruction canary. Early in the chat, set a rule you can check, such as “answer in under 80 words” or “no emojis.” Near the end, ask: “Restate every standing rule I gave you, word for word, then answer my next question.” Then count the words in the answer and look for emojis. A missing or changed rule in the restatement is a warning. So is a broken rule in the answer that follows, even when the restatement was correct.
- Re-ask test. Ask a question you asked early, in new words, about the same fact. Compare the two answers line by line. A changed date, price or name, or a direct contradiction, means at least one answer is wrong or the fact changed during the chat.
- Length watch. Note the word count of each answer to a similar kind of question. Read any answer that is several times longer than the earlier ones, because it may repeat earlier text.
Invented example, not measured data: a rule “no emojis” holds from turn 2 to turn 6. In turn 9, an answer contains an emoji, and the restatement you ask for in turn 14 leaves the rule out. Answer lengths run 60, 72, 58, 64 and 61 words, then 310 words in turn 11. Two signs point to turns 8 to 11, and the turn-14 restatement that leaves out the rule shows the lapse was not recovered, so read from turn 8.
What the Context Drift Heatmap automates
Paste the chat into the Context Drift Heatmap and it reads the turns, estimates the tokens in each one and draws a heatmap with one row per turn. A ledger lists the standing instructions it found in user and system turns, and it marks which later answers kept each checkable one.
Only instructions that a program can check get a pass or fail. Word limits, banned words, required phrases, emoji use, a required citation pattern and JSON-only output are examples. Soft instructions such as “be friendly” appear in the ledger as not machine-checkable. The checks follow the kinds of instruction in IFEval, which notes that “very few instructions are 100% verifiable objectively and automatically.” (Zhou et al., IFEval)
The tool also flags answers that are at least 2.5 times the median length of earlier answers. The first three answers are compared with the median of all answers. Contradiction signals are heuristics, such as a changed number for the same subject. They quote the text that triggered them, so read the quote first. Treat a flag as a place to read, not a verdict.
Two limits apply. The heatmap reads only the visible transcript, so its percentage of the window shows only the visible part of the load. The real load can be higher, because a real request can include hidden system prompts and tool definitions, or lower if the app drops older turns. The count here is an estimate. A middle-band overlay marks turns between 33% and 67% of the transcript, a different setting from the position study.
For a live count of a prompt, use the LLM Token Counter. To see how a transcript fills a window size, use the Context Window Visualizer.
Common mistakes, and what this cannot tell you
- Treating lost in the middle as chat drift. That study measured position inside one prompt, on 2023 models.
- Quoting 39% as your chat’s rate. The figure comes from simulated conversations with an LLM user.
- Trusting a restatement as proof. A correct list of rules can sit beside a wrong answer.
- Converting characters to tokens with a fixed ratio. OpenAI’s token-counting guide says “estimates like characters / 4 are inaccurate” in a list of the limits of local tokenizers. Use the provider’s counting method for that model.
This guide cannot tell you whether a particular model in a particular app will drift on your task. It cannot see hidden instructions, and it cannot judge whether an answer is correct. The studies cited cover named models tested from 2023 to 2025. If you want to try a restart, the handoff checklist explains one practical method, but none of these studies tests it.
Common questions
Why does an AI forget my instructions in a long chat?
Each turn adds to the input, and controlled tests have found that model performance degrades as input length grows.
Is lost in the middle the same as context drift?
No. Liu et al. measured where relevant text sat inside one long prompt, using 2023 models. Drift across turns is a different setting, and the Laban et al. simulation studied that.
Where does the 39% figure come from?
It is the average drop that Laban et al. report for 15 LLMs across six generation tasks, in simulated multi-turn conversations with an LLM user. The paper’s Figure 1 caption gives −35% for the same models. Cite both numbers and say the conversations were simulated.
How many tokens is my chat?
Treat any count you make yourself as an estimate. Anthropic’s token-counting page says “The token count is an estimate.” To estimate a request before you send it, Anthropic recommends its token counting API for the model you plan to use.
Does summarizing a chat fix drift?
The research cited here does not test restarting a chat with a summary, so no study shows that it fixes drift. Anthropic describes compaction as replacing older turns with a summary. OpenAI offers server-side compaction, which runs when the rendered token count crosses a threshold you set. Anthropic’s engineering team warns that “overly aggressive compaction can result in the loss of subtle but critical context.” (Anthropic, Effective context engineering)
Can a tool prove that the AI broke a rule?
Only for rules a program can check, such as a word limit or a banned word, and even those checks can fail on edge cases. Soft rules are not machine-checkable. The Context Drift Heatmap marks pass or fail for checkable rules and lists contradictions as leads to read.
Sources and further reading
- Liu et al., Lost in the Middle, arXiv, Nov 2023
- Laban et al., LLMs Get Lost In Multi-Turn Conversation, arXiv, May 2025
- Chroma, Context Rot, technical report, July 2025
- Anthropic, Effective context engineering for AI agents, Sep 2025
- Anthropic, Context windows, Claude Platform docs
- Anthropic, Token counting, Claude Platform docs
- OpenAI, Conversation state, API docs
- OpenAI, Counting tokens, API docs
- Zhou et al., IFEval, arXiv, Nov 2023