Context Engineering for AI Agents: Building Reliable Enterprise Systems

Updated October 2026

Context engineering for AI agents is the practice of deciding what an agent sees at every step of a task. It covers instructions, tools, retrieved data, memory, and the tool results that pile up as the agent works. Done well, it keeps the window small and relevant, which makes long running enterprise agents more accurate and cheaper to run.

This guide covers the runtime practice. For how it relates to the shared business meaning agents draw on, read context engineering vs enterprise context management.

What is context engineering for AI agents?

Context engineering is the work of curating every token a model sees during inference, not just the prompt. Anthropic defines it as strategies for curating and maintaining the optimal set of tokens, in its September 2025 engineering guide. For an agent, that set changes after every tool call.

The discipline matters more for agents than for chat. Anthropic reports that agents typically use about four times the tokens of chat, and multi agent systems about 15 times, per its multi agent research write up. More tokens mean more chances to bury the facts that matter.

Why a bigger window is not the answer

Models get less reliable as inputs grow, even on simple tasks. Chroma tested 18 models from four providers and found that one distractor lowered accuracy, per its context rot study. Earlier work found accuracy is highest when relevant facts sit at the start or end of the input, per the Lost in the Middle paper.

So the target is not the most context. It is the smallest set of high signal tokens that makes the right outcome likely. Every section below is a way to get closer to that set.

What goes into an agent's context window?

An agent's window holds six kinds of material, and each needs its own rules. LangChain groups the responses into four strategies, which are write, select, compress, and isolate, in its July 2025 overview. Write saves context outside the window. Select pulls the right pieces in. Compress trims what is there, and isolate splits work across separate windows.

The rest of this guide walks through each component in the order an agent meets them. Start with the parts that stay fixed, then handle the parts that grow.

How do you write a system prompt at the right altitude?

Write the minimal set of instructions that fully defines the behavior you want, then add detail only when tests show a failure. Anthropic calls this finding the right altitude, between brittle hard coded logic and guidance too vague to act on. Start small and grow the prompt from observed mistakes.

Organize the prompt into clear sections, such as background, instructions, tool guidance, and output format. Use a few diverse, canonical examples instead of a long list of edge cases. A model learns more from three well chosen examples than from 30 rules it must reconcile.

Keep business facts out of the prompt

A system prompt is the wrong home for metric formulas, customer lists, or org charts. Those facts change, and copies drift between agents. Point the agent at a tool that returns the governed definition instead. The section on business meaning below explains why this matters for accuracy.

How should you design tools for agents?

Design a small set of distinct tools whose outputs are compact and readable. Anthropic's guide to writing tools for agents recommends consolidating multi step workflows into single tools, namespacing tools by service, and avoiding overlap. If a person cannot tell which tool fits a task, the model cannot either.

Tool outputs are often the largest thing in the window, so limit them. Use pagination, filtering, and truncation. Claude Code caps tool responses at 25,000 tokens by default, according to the same guide. Return names rather than opaque IDs, and write error messages that say how to fix the call.

Offer a concise mode

A response format parameter lets the agent ask for a short or detailed answer. In Anthropic's example, the concise response used about a third of the tokens of the detailed one. Most steps only need the short version, and the agent can ask for more when it must.

When should an agent retrieve context up front versus just in time?

Retrieve most context just in time, and load only a few always needed facts up front. Anthropic recommends that agents hold lightweight identifiers such as file paths, saved queries, or links and fetch content when a step requires it. The agent then explores the way a person uses folders and bookmarks.

This keeps the window lean and lets the agent find context in layers. A file name, folder path, or timestamp already says a lot before the content loads. The tradeoff is speed, because runtime lookups take longer than reading data that is already present.

For choosing the retrieval backend itself, see RAG vs knowledge graph vs semantic layer.

How should agents handle memory on long tasks?

Agents should write progress to storage outside the window and read it back when needed. Anthropic calls this structured note taking. A notes file or todo list survives compaction and lets the agent resume a task across many context resets.

Manus, which runs tasks averaging about 50 tool calls, uses the file system as external memory. It rewrites a todo file as it works, which pushes the current plan to the end of the context where the model attends best, per its lessons from building Manus. It also keeps compression restorable by holding a URL or file path whenever it drops content.

Use a memory tool where one exists

Anthropic's memory tool stores files in a directory the developer controls, so notes persist across conversations. Combining the memory tool with context editing improved an internal agentic search evaluation by 39 percent, per Anthropic's context management announcement. Treat memory as data with access rules, because stored notes can leak into unrelated tasks.

How do you clear and compact context during a long run?

Clear old tool results first, then compact the whole history when the window nears its limit. Clearing is cheap and loses little, because a tool result has usually served its purpose a few turns later. Compaction rewrites the history into a summary and starts a fresh window from it.

Tool result clearing

Claude's context editing feature, in beta, clears the oldest tool results once input passes a threshold. The default threshold is 100,000 input tokens, and the three most recent tool uses are kept. You can exclude tools whose results must stay, such as a definitions lookup.

On a 100 turn web search task, context editing let agents finish work that would otherwise fail and cut token use by 84 percent, per Anthropic. Context editing alone improved the agentic search evaluation by 29 percent.

Compaction

Compaction summarizes the conversation and continues from the summary. Anthropic advises tuning the summary prompt to keep everything important first, then trimming for precision. Claude's docs now name server side compaction as the primary strategy for most long conversations. Claude Code compacts automatically after the window passes 95 percent, according to LangChain.

Keep errors in the summary. Manus found that leaving failed actions visible helps the model avoid repeating them. A summary that hides mistakes invites the same mistake again.

When do sub agents help?

Sub agents help when a task splits into independent parts that each need deep exploration. A lead agent hands each part to a sub agent with a clean window. Each sub agent returns a short distilled summary, often 1,000 to 2,000 tokens, according to Anthropic's context engineering guide.

The gains are real and so are the costs. Anthropic's multi agent setup, with Claude Opus 4 leading Claude Sonnet 4 sub agents, beat a single Opus 4 agent by 90.2 percent on its internal research eval. It also used about 15 times the tokens of chat. Use sub agents for broad research and avoid them for tightly coupled work.

How does prompt caching change context design?

Caching rewards a stable prefix and an append only history. On most Claude models a cache read costs one tenth of the base input price, and the default cache lifetime is five minutes, per the prompt caching docs. Any change near the top of the prompt invalidates everything after it.

Manus calls KV cache hit rate the most important metric for a production agent. Its average input to output token ratio is about 100 to 1, so input cost dominates. It avoids timestamps at the start of the prompt, serializes data in a fixed order, and masks unavailable tools instead of removing their definitions mid task.

Where should business meaning come from?

Business meaning should come from a governed source that every agent queries, not from text pasted into prompts. A revenue formula in a system prompt works for one agent. A second agent gets a slightly different copy, and soon two agents report different numbers with no way to tell which is right.

Explicit meaning moves accuracy more than most prompt changes. On the data.world benchmark, GPT-4 answered 16 percent of enterprise questions correctly over raw SQL and 54 percent over a knowledge graph of the same data, per Sequeda, Allemang, and Jacob. A 2026 study found explicit business semantics added 17 to 23 points across Claude Opus 4.7, Sonnet 4.6, and GPT-5.4 (arXiv 2604.25149).

Serve definitions through tools

Expose the semantic layer, ontology, and resolved entities as tools, often through MCP, which now sits under the Linux Foundation's Agentic AI Foundation. The agent fetches only the definitions a step needs. Exclude that tool from clearing so definitions survive long runs. Our guide to enterprise context management for AI agents covers how to build that shared layer.

Own the meaning you serve

Those definitions are institutional knowledge, and they outlast any single model or agent framework. Keeping them in open formats on infrastructure you control means the next agent can use them on day one. Galaxy is one partner that builds semantic layers, ontologies, and entity resolution on a team's existing stack, and it builds in the open with Filament.

How do you test context engineering?

Test it with an evaluation set of realistic multi step tasks that have verifiable outcomes. Anthropic's tool writing guide recommends reviewing transcripts and tracking tool calls, runtime, and errors alongside success. Rerun the set after every change to prompts, tools, retrieval, or models.

Track tokens per task as a first class metric. On the BrowseComp evaluation, token usage alone explained 80 percent of performance variance, per Anthropic's multi agent write up. For agents that touch regulated data, also log which sources fed each answer. Our piece on provenance and lineage for AI context covers that audit trail.

What are the most common context engineering mistakes?

Most failures come from adding context rather than curating it. The table lists the patterns teams hit most often and the fix for each.

For the controls that keep agent access safe, see data governance for AI agents.

Frequently asked questions

What is context engineering for AI agents?

Context engineering is the practice of choosing and maintaining the tokens an agent sees at each step of a task. It covers system instructions, tool definitions, retrieved data, message history, tool results, and memory. Anthropic describes it as curating the optimal set of tokens during inference. The goal is a small window that holds exactly what the next decision needs.

How is context engineering different from prompt engineering?

Prompt engineering focuses on wording one set of instructions. Context engineering manages everything else that enters the window across many steps, including tool outputs, retrieved documents, and notes. Agents change their own context every turn, so the work becomes deciding what to load, what to keep, and what to clear as the task runs.

Why do long agent runs get less accurate?

Models do not use long inputs evenly. Chroma tested 18 models in July 2025 and found reliability drops as input grows, and even one distractor lowers accuracy. Earlier research found models do best when key facts sit at the start or end of the input. Long runs pile up stale tool output that pushes useful facts into the middle.

What is compaction in context engineering?

Compaction summarizes a conversation that is nearing the context limit and starts a fresh window from that summary. Anthropic recommends tuning the summary for recall first, then trimming for precision. Claude now offers server side compaction through its API, and Claude Code compacts automatically once the window is about 95 percent full, according to LangChain.

What is tool result clearing?

Tool result clearing removes old tool outputs from the history once the agent has used them. Anthropic calls it one of the lightest forms of compaction. Its API strategy clears the oldest results after a threshold, which defaults to 100,000 input tokens, keeps the three most recent tool uses, and can exclude chosen tools from clearing.

Should an agent load all relevant data at the start?

Usually not. Loading everything up front fills the window with material the agent may never use. Anthropic recommends keeping lightweight identifiers such as file paths, links, or saved queries and fetching content only when a step needs it. A hybrid works well when a few facts are always needed, such as core definitions loaded up front.

When are sub agents worth the cost?

Sub agents help when work can split into independent parts, such as broad research across many sources. Anthropic's multi agent system beat a single agent by 90.2 percent on an internal research eval, but used about 15 times the tokens of chat. They are a poor fit for tightly coupled tasks where every step depends on the last.

How does prompt caching affect context design?

Caching rewards a stable prefix. On most Claude models a cache read costs one tenth of the base input price, and the default cache lasts five minutes. Manus reports an input to output token ratio near 100 to 1, so cache hits dominate cost. Avoid timestamps at the top of prompts and keep history append only.

Where should business definitions come from in an agent's context?

They should come from a governed source, such as a semantic layer, ontology, or resolved entity store, rather than text copied into prompts. Copied definitions drift between agents. When meaning is explicit, accuracy rises, with one 2026 study finding business semantics added 17 to 23 points across Claude Opus 4.7, Sonnet 4.6, and GPT-5.4.

How do you measure whether context engineering is working?

Build an evaluation set of realistic multi step tasks with verifiable outcomes, then track success rate, tokens per task, tool calls, and errors. Anthropic found token usage alone explained 80 percent of performance variance on BrowseComp. Rerun the set after every prompt, tool, or model change so improvements and regressions show up as numbers.