What Is Context Engineering? Techniques for Reliable AI Agents

Context engineering is the practice of deciding what an AI model sees at every step: the system prompt, tool definitions, examples, retrieved documents, conversation history, memory and tool results. Anthropic defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference." The goal is the smallest set of high-signal tokens that makes the right outcome likely. It matters most for agents, which run many steps and can fill their own context with noise.

Quick answer

• What it is: managing the whole context window, not just writing the prompt.
• Where the term came from: popularized in June 2025 by Tobi Lütke and Andrej Karpathy, then formalized by LangChain (July 2025) and Anthropic (September 2025).
• Why it matters: models get less reliable as context grows ("context rot"), and every token costs money and time.
• The four strategies: write (save outside the window), select (pull in what is relevant), compress (summarize or trim), isolate (split across sub-agents).
• Core techniques: a right-sized system prompt, a small set of clear tools, canonical examples, just-in-time retrieval, compaction, structured notes and sub-agents.
• Relationship to prompt engineering: prompt engineering is one part of it. See context vs prompt engineering.

Where the term came from

"Prompt engineering" came to mean typing clever instructions into a chatbot. By 2025, people building real systems were spending most of their effort on something else: what to retrieve, which tools to expose, how to keep a long agent run on track.

In June 2025, Shopify CEO Tobi Lütke wrote that he really liked the term "context engineering" over prompt engineering because "it describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy replied with "+1 for 'context engineering' over 'prompt engineering'", describing it as "the delicate art and science of filling the context window with just the right information for the next step."

LangChain's Context Engineering for Agents (2 July 2025) organized the field into four strategies. Anthropic's Effective context engineering for AI agents (29 September 2025) became the most-cited guide, and it is still the top result for the term.

Why context engineering matters

Context rot and the attention budget

Long context windows invite people to paste everything in. Research says that backfires. Chroma's Context Rot report (July 2025) tested 18 models and found performance is not consistent across input lengths: reliability tends to decline as inputs get longer, even on simple tasks, and worsens when the relevant information looks less like the question.

Anthropic frames this as an attention budget. Every token you add draws it down, so context should be treated "as a finite resource with diminishing marginal returns." The model can technically read 200,000 tokens; that does not mean it attends to all of them equally well.

How contexts fail

Drew Breunig's June 2025 post How Contexts Fail named four failure modes that are now common vocabulary:

Cost and speed

Every token is billed and processed. Anthropic's code execution with MCP post describes an agent workflow that dropped from about 150,000 tokens to about 2,000 by keeping tool definitions and intermediate data out of the model's context. That is context engineering applied to tools.

What goes into the context

Everything the model sees on a given call:

1. System prompt: role, rules, output format.
2. Tool definitions: names, descriptions and schemas, often from MCP servers.
3. Examples: few-shot demonstrations of the behavior you want.
4. Retrieved knowledge: passages from RAG, files, search results.
5. Conversation history: user turns and the model's own previous replies.
6. Tool results: outputs from every call so far, often the biggest and noisiest part.
7. Memory and state: notes, to-do lists, user preferences loaded from outside the window.

Prompt engineering mostly works on item 1. Context engineering manages all seven, every turn.

The four strategies: write, select, compress, isolate

LangChain's framework is the answer to "what are the four pillars of context engineering?":

Context engineering techniques that work

These come from Anthropic's guide and its prompting documentation, with practical notes on applying them.

1. Write the system prompt at the right altitude

Anthropic warns against two extremes: brittle, hard-coded if-else logic, and vague guidance that assumes shared context. Aim for the "right altitude": specific enough to guide behavior, flexible enough to let the model use judgment. Aim for "the minimal set of information that fully outlines your expected behavior", organized into clear sections. Start small with a capable model, test, and add instructions only for failures you actually see.

2. Design a small, clear tool set

Tools are context too. Anthropic advises tool sets with minimal overlap in functionality, and puts the test bluntly: "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better." Return compact, token-efficient results. If you connect many MCP servers, use the client's on-demand loading: Claude Code, for example, uses tool search by default so tool definitions load only when needed. The 2026-07-28 MCP spec also asks servers to list tools in a deterministic order to improve prompt cache hits.

3. Use a few canonical examples, not a list of edge cases

Rather than stuffing a prompt with every rule, Anthropic recommends a set of "diverse, canonical examples" that show expected behavior. For a model, a good example is worth many sentences of instructions.

4. Retrieve just in time

Instead of loading everything up front, give the agent lightweight references (file paths, queries, links) and tools to fetch details when needed. This mirrors how people work: you remember where a file is, not its contents. Anthropic notes many systems use a hybrid: load a little up front (a README or a project rules file) and explore the rest with tools.

5. Order long inputs well

From Anthropic's prompting docs: put long documents near the top of the prompt, above the instructions and the question. Its docs say queries at the end "can improve response quality by up to 30 percent in tests, especially with complex, multidocument inputs."

6. Compact long runs

When the window is nearly full, summarize the conversation, keeping decisions, open problems and key details, and start a fresh window with the summary. A light version is tool result clearing: once a tool's output has been used, drop the raw output from history.

7. Keep structured notes outside the window

Have the agent maintain a progress file, to-do list or notes store, and reload it after compaction or in a new session. Anthropic calls this structured note-taking, or agentic memory. It is the "write" strategy in practice, and it guards against context poisoning when notes are checked against reality.

8. Isolate work in sub-agents

Give focused tasks to sub-agents with clean context windows, and have them return condensed summaries to the lead agent. The lead agent keeps a small, high-level context while sub-agents explore deeply. This is how most serious coding and research agents are built; see how to build an AI agent.

Context engineering in practice: a coding agent

Coding agents are where these ideas are most visible, because a repository is far bigger than any context window. A well-engineered coding agent typically:

1. Starts with a small, stable prefix: a system prompt, the tool list and a short project rules file (stack, test command, conventions). This stays identical across turns so prompt caching works.
2. Explores just in time: it lists directories, searches with grep or a code index, and opens only the files it needs, rather than loading the repository.
3. Keeps tool output short: test runs return the failing tests and the relevant stack trace, not thousands of lines of logs.
4. Writes a plan and progress notes to a file, so a compaction or a new session can pick up where it left off.
5. Delegates exploration to sub-agents: "find every caller of this function" runs in a separate window and returns a short list.
6. Compacts when the window fills, keeping decisions and open problems and dropping raw output.

The same agent with the same model will behave very differently if it instead pastes whole files into history, keeps every test log, and has thirty overlapping tools. Nothing about the prompt wording changed; the context did.

How to measure your context engineering

You cannot improve what you do not measure. Useful signals:

• Tokens per completed task: total input tokens across all calls. Falling numbers with steady quality mean less waste.
• Cache hit rate: if your provider reports cached input tokens, a low rate usually means the prefix changes every turn.
• Turn at which quality drops: run long tasks and note when the agent starts repeating itself or forgetting constraints. That is your practical context limit.
• Tool selection errors: how often the agent calls the wrong tool or the right tool with wrong arguments. High rates point at tool descriptions or overlap.
• Retrieval precision: of the passages you load, how many were actually used in the answer.

A practical checklist

Before shipping an agent, check each call it makes:

• Is the system prompt as short as it can be while still covering observed failures?
• Does every tool have a distinct job and a description a new engineer would understand?
• Are tool results trimmed to what the next step needs?
• Is retrieval returning a few relevant chunks, not whole documents?
• Do long documents come before the question?
• Is there a plan for when the window fills: compaction, clearing or a fresh sub-agent?
• Are notes and memory written outside the window and reloaded deliberately?
• Are stable parts (system prompt, tool list) kept at the start and unchanged so prompt caching works?

Context engineering vs prompt engineering

Prompt engineering is not obsolete. Clear instructions still matter, and they are part of the context. But for agents, the wording of the prompt is rarely what breaks; what breaks is everything else that piled up around it. The full comparison is in context vs prompt engineering, and the related terms are in our glossary under context engineering and context window.