Enterprise Context Management for AI Agents: Architecture & Patterns

Updated October 2026

Enterprise context management for AI agents is the architecture that gives agents governed business knowledge at the moment they act. It combines context sources, retrieval patterns such as RAG, GraphRAG, and semantic layers, delivery through MCP, memory, permissions, and evaluation. Teams that own these layers get agents that answer consistently and stay within access rules.

This guide covers each layer and the patterns that hold up in production. For a side by side comparison of the three main retrieval approaches, read RAG vs knowledge graph vs semantic layer.

What is enterprise context management for AI agents?

Enterprise context management is how a company decides what knowledge agents can use, keeps it correct, and delivers it with the right access rules. It treats context as shared infrastructure rather than something each agent team rebuilds. The output is one governed source of meaning that many agents can call.

The idea overlaps with context engineering, which Anthropic describes as curating the best set of tokens during inference in its guide to context engineering for agents. Context engineering happens inside one agent. Enterprise context management happens across the company, so every agent starts from the same definitions. Our piece on context engineering for AI agents covers the single agent side.

Why should a company own its context layer?

A company should own its context layer because the definitions, relationships, and decisions inside it are institutional knowledge. That knowledge is one of the most valuable assets a business has. If it only lives inside one vendor's proprietary model, switching tools means rebuilding it from scratch.

Ownership in practice means three things. Core models sit in open formats such as SQL, YAML metric definitions, or RDF. They run on infrastructure you control. Any agent host can reach them through an open protocol. The metric spec that started as Open Semantic Interchange is now Apache Ossie (Incubating), and MCP was donated to the Agentic AI Foundation under the Linux Foundation. Both make open formats a practical default.

Which context sources do AI agents need?

Agents need the same sources your people use, grouped by the kind of question they answer. Structured data answers how much and how many. Documents answer why and how. Metadata answers where data came from and who owns it. Memory answers what happened in earlier sessions.

Structured data and metrics

Warehouses and lakehouses hold most of the numbers agents are asked about. Raw tables are hard for models to query well. On the Spider 2.0 benchmark of real enterprise text to SQL tasks, the best agent solved 21.3 percent of problems, according to the Spider 2.0 paper.

Documents and conversations

Policies, contracts, tickets, and wiki pages carry the reasoning behind the numbers. They need chunking, metadata, and source permissions carried along with every chunk.

Metadata and lineage

Catalog metadata tells an agent which table is certified, who owns it, and where it came from. Without it, an agent can pick a deprecated table that looks correct and return a confident wrong answer.

Which retrieval patterns work for enterprise agents?

No single retrieval pattern wins on every question. Vector RAG works for finding passages in documents. GraphRAG helps with questions that span many entities. A semantic layer gives governed answers for metrics. Tools called through MCP let the agent fetch live data or take actions.

Vector and hybrid RAG

Plain vector search is the default for documents, and adding context improves it. Anthropic reported that contextual embeddings plus contextual BM25 and reranking cut top 20 retrieval failures by 67 percent in its contextual retrieval study. Hybrid search, which Weaviate documents as fusing vector and BM25F results, helps with product codes and names.

GraphRAG

GraphRAG builds an entity graph from text and summarizes communities of related entities. Microsoft's GraphRAG paper reported 72 to 83 percent comprehensiveness win rates over vector RAG on global questions. Indexing is expensive, and the GraphRAG-Bench study found it often underperforms vanilla RAG on many real tasks.

Semantic layer

A semantic layer defines each metric once and compiles queries from that definition. In dbt Labs' April 2026 vendor benchmark of 11 questions, the semantic layer scored 98 to 100 percent versus 84 to 90 percent for text to SQL, per dbt's write up. The same post found the semantic layer fails openly on questions it cannot model, while text to SQL fails with plausible wrong answers.

Ontology and knowledge graph

An ontology names business entities, their relationships, and the rules between them. In the data.world benchmark, GPT-4 answered 16 percent of enterprise questions correctly over raw SQL and 54 percent over a knowledge graph of the same data (Sequeda, Allemang, and Jacob). Our enterprise ontology playbook walks through building one.

How does MCP deliver context to agents?

MCP gives every context source one standard interface that any compatible agent host can call. A server exposes resources for data, tools for functions the model can run, and prompts for templated workflows, as set out in the current MCP specification. Build one server per context domain, not one integration per agent.

What changed in the July 2026 revision

The July 28, 2026 revision made MCP stateless. It removed the initialize handshake and protocol level sessions, so each request carries its own version and capabilities, per the specification changelog. It also moved tasks into an official extension and deprecated Roots, Sampling, and Logging. List and read results now carry cache hints, which helps context servers that serve slow moving definitions.

Authorization for context servers

For HTTP servers, MCP authorization is based on OAuth 2.1 with resource indicators. Servers must only accept tokens issued for them and must not pass other tokens through. That rule matters for context layers, because a server that forwards a broad token can expose every system behind it.

Vendor context servers

Several vendors now position MCP as the way to ship governed context. Atlan describes its hosted MCP server as delivering certified definitions, lineage, and access rules to agents, with 29 tools including write actions. Glean's MCP server exposes search, chat, and document reading to over 20 tested hosts. The dbt MCP server and Snowflake's managed MCP server expose semantic layer queries.

These are useful if you already run the product. The limit is that each vendor's server only reaches context stored in that vendor. A company with definitions spread across a catalog, a warehouse, and a wiki still needs a plan for which source owns each concept.

How should agent memory be managed?

Agent memory should be treated as governed context with an owner, a retention rule, and a way to expire old facts. Without that, memory becomes a second, unreviewed knowledge base. The risk is an agent that confidently repeats a fact that changed months ago.

Two patterns work well together. Short term memory keeps a working set within one task, and Anthropic recommends compaction and structured notes for long tasks in its context engineering guide. Long term memory stores facts across sessions. Graphiti, an Apache 2.0 project, keeps a validity window on each fact, so old facts are invalidated rather than deleted.

The Zep paper behind Graphiti reported 94.8 percent on the DMR benchmark versus 93.4 percent for MemGPT, per Rasmussen and coauthors. That is a vendor authored paper, so test memory designs on your own tasks.

How do you enforce permissions on agent context?

Enforce permissions at the source and pass the user's identity through every retrieval and tool call. The agent should see exactly what the requesting person could see, no more. Application level filtering after retrieval is fragile, because one missed path leaks data.

Glean's documentation shows the pattern. Its MCP server runs tools as the specific user and states that items hidden from a user in Glean stay hidden through MCP, per its security page. Apply the same rule to your own servers with row level security in the warehouse and scoped OAuth tokens on each MCP server.

Retrieved content is also an attack path. OWASP ranks prompt injection first in its 2025 LLM risk list and covers indirect injection through websites and files. Its main mitigation is least privilege, which means giving each agent only the tools and scopes its task needs.

How do you evaluate an enterprise context layer?

Evaluate it with a fixed set of real business questions that have known answers, run before and after every change. Score accuracy, honest abstentions, and confident wrong answers as separate numbers. A context change that raises accuracy but also raises confident errors is a regression.

Research shows how much the context layer moves these numbers. An ontology based query check on the data.world benchmark raised accuracy to 72.55 percent and turned some errors into explicit abstentions, per Allemang and Sequeda. A 2026 study found explicit business semantics added 17 to 23 points across Claude Opus 4.7, Sonnet 4.6, and GPT-5.4 (arXiv 2604.25149).

Evaluation also protects the budget. Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027, citing costs, unclear value, and weak risk controls (Gartner). A measured context layer gives you evidence before that decision gets made for you.

How do you build an enterprise context layer step by step?

Start with one domain, one agent, and one question set, then expand. Building the whole company's context before any agent uses it is the pattern that stalls. Each step below produces something an agent can use right away.

1. Pick a domain with frequent questions and a clear owner, such as revenue or support.

2. Write 20 to 50 real questions with known answers. These become your test set.

3. Define the metrics and entities those questions need, and resolve duplicate records for them.

4. Expose the definitions and data through an MCP server with user level authorization.

5. Run the test set, fix the gaps, and add memory only where tasks span sessions.

6. Add the next domain and reuse the same entities, so meaning stays shared.

Galaxy works in that last row. It builds semantic layers, ontologies, and entity resolution on the stack a team already runs, using internal tooling, and builds in the open with Filament while planning to open source more. For platform options on the graph side, see our list of knowledge graph platforms.

Frequently asked questions

What is enterprise context management for AI agents?

It is the practice of deciding what business knowledge agents can see, keeping it accurate, and delivering it at the moment of a task. That knowledge includes documents, governed metric definitions, entity relationships, metadata, and memory of past work. The goal is that agents reason over the same meaning your people use, with the same access rules.

How is enterprise context management different from RAG?

RAG is one retrieval pattern inside a context architecture. It finds relevant text chunks and adds them to the prompt. Enterprise context management also covers structured definitions, entity resolution, relationships, memory, permissions, and evaluation. RAG works well for documents, but numeric questions are better served by a semantic layer, and multi hop questions by a graph.

What does MCP add to a context architecture?

MCP standardizes how agents connect to context sources. A server exposes resources for data, tools for functions, and prompts for templated workflows. One server per context source can then serve many different agent hosts. Over HTTP, MCP authorization builds on OAuth 2.1, so each connection can carry scoped, user level access instead of a shared key.

What changed in the July 2026 MCP specification?

The July 28, 2026 revision made the protocol stateless by removing the initialize handshake and protocol level sessions. It moved tasks into an official extension and deprecated the Roots, Sampling, and Logging features. It also added cache hints to list and read results. Servers built for older revisions need review before upgrading to the new version.

Should AI agents use a semantic layer or a knowledge graph?

Most enterprise agents need both. A semantic layer gives one governed answer for metrics like revenue or churn. A knowledge graph captures entities and relationships, which helps with questions that span customers, contracts, and products. Route each question to the source that holds its meaning, and keep both aligned through shared business definitions.

How should permissions work for AI agents?

Enforce permissions where the data lives and pass the user's identity through every call. The agent should only retrieve what the requesting person could already see. Glean's MCP server, for example, runs tools as the specific user. Treat retrieved content as untrusted, since OWASP lists indirect prompt injection through external content as the top LLM risk.

What is agent memory in an enterprise setting?

Agent memory is context the agent writes and reads across sessions, such as user preferences, decisions, and facts learned during work. It needs the same governance as other context. Temporal graph engines like Graphiti track when each fact became true and when it was replaced, so agents can tell current facts from old ones.

How do you evaluate an enterprise context layer?

Build a fixed set of real business questions with known answers and run it before and after each change. Track accuracy, honest abstentions, and confident wrong answers separately. Research on the data.world benchmark showed GPT-4 accuracy rose from 16 percent over raw SQL to 54 percent over a knowledge graph of the same data.

Do we need a vendor context layer product?

Not necessarily. Catalog and search vendors such as Atlan and Glean now ship hosted MCP servers that deliver their governed context to agents. That is useful if you already run them. The tradeoff is where your definitions live. Keeping core models in open formats on your own stack keeps them portable across vendors and agent hosts.

Why do enterprise AI agent projects fail?

Many fail on cost, unclear value, or weak risk controls rather than model quality. Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027. Missing or inconsistent business context makes all three problems worse, because agents give plausible wrong answers that erode trust and require human checking.