Knowledge Graphs for Enterprise AI: GraphRAG, Agents, and Grounded Answers

Updated October 2026

A GraphRAG implementation has five parts. You chunk and index source content, build a knowledge graph from documents or structured data, choose query patterns that fit your questions, control indexing and query costs, and evaluate against a vector RAG baseline. Most failures come from the graph construction and evaluation steps, not from the retrieval library.

This guide is the hands-on companion to our other knowledge graph articles. For how graphs support reasoning and which platforms run GraphRAG, see knowledge graph platforms for AI reasoning. For when to pick a graph, plain RAG, or a semantic layer, see RAG vs. knowledge graph vs. semantic layer.

What does a GraphRAG system consist of?

A GraphRAG system consists of an indexing pipeline, a graph store with a vector index, a set of retrievers, and a generation step. The indexing pipeline turns raw content into entities, relationships, and summaries. The retrievers decide which slice of the graph reaches the model for each question.

Microsoft Research popularized the pattern in its 2024 GraphRAG paper, which showed that vector RAG struggles with questions about a whole corpus. The paper extracted an entity graph from documents, grouped entities into communities, and summarized each community before any question arrived. Today the label covers a wider family of designs, including retrieval from graphs that people modeled by hand.

Two open source toolkits cover most hands-on work. Microsoft GraphRAG is an MIT licensed Python library that runs the full document pipeline. The Neo4j GraphRAG package for Python builds a graph inside Neo4j and ships retrievers for it. Managed options exist too, such as Amazon Bedrock Knowledge Bases GraphRAG, which became generally available in March 2025.

How does the GraphRAG indexing pipeline work?

The indexing pipeline splits documents into chunks, extracts entities and relationships from each chunk with an LLM, merges and summarizes them, clusters them into communities, and embeds the results. Each stage adds cost and each one can introduce errors. Knowing the defaults helps you decide where to spend.

Microsoft documents its default indexing dataflow in six phases. The table below follows that sequence and uses the library's published defaults.

Tune the prompts before you index

The default entity types are generic. A contract corpus needs types like party, clause, and obligation, and a support corpus needs product, issue, and fix. Microsoft's prompt tuning guide strongly encourages running auto tuning, which adapts the extraction prompts to your data before the first full run.

Gleanings trade cost for recall

A gleaning is an extra LLM pass over the same chunk that asks what the first pass missed. One gleaning is the default. Each added pass can catch more entities, but it also adds another full set of extraction calls across the corpus.

Should you build the graph from documents or from structured data?

Build from structured data wherever the facts already exist in systems of record, and extract from documents only for knowledge that lives in text. Mapping a customer table is cheap and exact. Asking an LLM to rediscover the same customers from emails is expensive and introduces errors.

Building from documents

Document graphs suit research archives, policies, incident reports, and other narrative content. The Neo4j pipeline lets you pass a schema of node types, relationship types, and allowed patterns, such as a person who works for a company. It then prunes nodes and relationships that break the schema, which keeps extraction from drifting into hundreds of ad hoc labels.

A schema matters more than the model choice. Free extraction produces labels like Company, Organization, and Firm for the same thing. A fixed schema with short descriptions for each type gives the LLM a smaller target and gives you a graph that queries can rely on.

Building from structured data

Structured graphs start from tables you already trust. Standards like R2RML, a W3C Recommendation since 2012, express mappings from relational databases to RDF. Property graph teams often write the same mapping as load scripts. Our guide to SQL to RDF mapping tools walks through that workflow.

Combining both

Most enterprise systems need both. The structured graph supplies the backbone of customers, products, accounts, and owners. Extracted entities from documents then attach to that backbone, so a support ticket links to the real account node instead of a free text name.

The join point is entity resolution. The Neo4j pipeline merges nodes with the same label and name by default and offers fuzzy and semantic resolvers. Real enterprise data needs more than exact name matches, as our guide to entity resolution techniques explains.

Which GraphRAG query pattern fits which question?

Match the query pattern to the question type. Entity questions need local context, corpus-wide questions need summaries, and exact counts need a generated query against a curated schema. Most production systems route between two or three patterns rather than relying on one.

Document graph patterns

Microsoft GraphRAG offers four search methods. Local search combines graph neighbors with raw text chunks for questions about specific entities. Global search runs a map-reduce pass over community reports. DRIFT adds community information to local search, and basic search is a plain vector RAG baseline for comparison.

Curated graph patterns

The Neo4j package provides retrievers for a modeled graph. VectorCypherRetriever finds entry nodes by similarity, then runs a Cypher query to pull neighbors. Text2CypherRetriever asks an LLM to write the query itself. ToolsRetriever lets the LLM choose among several retrievers, which is a simple way to route by question type.

Text to query patterns need guardrails. Give the model the schema, a few example queries, and read only credentials. Our article on knowledge graphs for agentic AI covers how to check generated queries before they run.

What does GraphRAG cost to run?

GraphRAG costs more to index than vector RAG and can cost more per query, depending on the search method. Microsoft estimates that graph extraction is roughly 75 percent of indexing cost. Global search adds further cost at query time because it reads many community reports per question.

A quick way to estimate indexing calls

Estimate calls before you spend. With the defaults, chunks advance about 1,100 new tokens each, so a 10 million token corpus yields roughly 9,100 chunks. One extraction pass plus one gleaning means about 18,200 extraction calls. Description summaries and community reports add more calls on top.

Cheaper variants

FastGraphRAG extracts noun phrases with NLTK or spaCy and links entities that appear together. Microsoft says the result is quite a bit noisier and best for summary questions with global search. LazyGraphRAG matches vector RAG indexing cost, about 0.1 percent of full GraphRAG, and reports over 700 times lower query cost than global search at comparable quality.

How do you evaluate a GraphRAG system?

Evaluate GraphRAG against a plain vector RAG baseline on your own questions, and score retrieval and generation separately. Published results disagree on when graphs help. A test set built from real user questions is the only reliable way to know whether the extra cost pays off.

The evidence is mixed. The original GraphRAG paper reported strong wins on corpus-wide questions. GraphRAG-Bench notes that GraphRAG often underperforms vanilla RAG on many real tasks. A 2025 systematic comparison found each approach has distinct strengths and that combining them gives consistent gains.

Build the test set first

Collect 50 to 200 real questions and label each by type, such as lookup, multi-hop, aggregate, or corpus summary. Write expected answers or the source facts for questions that have one. Label types matter because GraphRAG may win on summaries and lose on lookups, and a single average score hides that.

Pick metrics per layer

Ragas offers context precision, context recall, faithfulness, and response relevancy, which cover retrieval and generation. Microsoft's BenchmarkQED generates local and global queries and uses an LLM judge for pairwise win rates. It is a vendor benchmark, so treat its LazyGraphRAG results as a starting hypothesis.

Check the graph itself

Retrieval metrics cannot fix a bad graph. Sample 100 extracted entities and relationships and have a domain expert mark each one right or wrong. Count duplicate entities and off-schema labels. These numbers predict answer quality better than any end-to-end score.

How do you keep a GraphRAG index fresh?

Keep document graphs fresh with incremental indexing and keep structured graphs fresh by syncing changed rows from the source. A graph that lags the business gives confident answers about last quarter. Track the source version of every node so stale facts are easy to find.

Microsoft GraphRAG 1.0 added an update command that computes deltas against an existing index. The same post notes that LLM caching makes reruns cheaper than the first run. It also warns that large additions can shift community structure enough to force broad recomputation. LightRAG was designed around incremental updates as a core feature.

Structured graphs refresh differently. Load changed rows on a schedule or through change data capture, then reapply mappings and entity resolution to those rows only. This path avoids LLM cost entirely for facts that already live in databases.

Where do GraphRAG projects go wrong?

GraphRAG projects usually fail on graph quality, scope, or cost, not on the retrieval library. The same few mistakes appear across most implementations. Each one has a simple guard.

Another risk is treating the extracted graph as the source of truth. LLM extraction is probabilistic. Keep a clear line between facts mapped from systems of record and facts extracted from text, and store provenance on both so reviewers can trace any answer.

Who should own the knowledge graph behind GraphRAG?

Your company should own the graph, its schema, and the mappings that feed it. The entity model encodes how your business works, which makes it one of your most valuable assets. Keep it in open formats on infrastructure you control, so you can swap models and retrieval libraries without rebuilding it.

Galaxy works with teams to build this layer on the stack they already run, using internal tooling for ontologies, semantic models, and entity resolution. It builds in the open with Filament, an Apache 2.0 data movement engine, and plans to open source more of that tooling. For the wider design question, see knowledge graphs vs. embeddings.

Frequently asked questions

What is GraphRAG in simple terms?

GraphRAG is retrieval augmented generation that uses a knowledge graph to choose context for a language model. Instead of only fetching text chunks that look similar to a question, it follows entities and relationships, or reads summaries of connected groups. That helps with questions that span many documents or need facts joined across several hops.

How long does it take to implement GraphRAG?

A prototype on a few hundred documents can run in days with Microsoft GraphRAG or the Neo4j GraphRAG package. A production system takes longer because schema design, entity resolution, evaluation, and refresh jobs carry most of the work. Plan the timeline around building a test set and a clean entity model, not around installing the library.

What chunk size should I use for GraphRAG?

Microsoft GraphRAG defaults to 1,200 token chunks with 100 tokens of overlap, and that is a sensible starting point. Smaller chunks usually surface more entities per document but raise the number of LLM calls and the indexing bill. Test two or three sizes on a sample, then compare extraction quality and cost before indexing the full corpus.

Should I build my knowledge graph from documents or from databases?

Build the backbone from databases when the facts already live there, such as customers, products, contracts, and owners. Mapping is cheaper and far more accurate than asking an LLM to rediscover them. Use LLM extraction for knowledge that only exists in text, then link those extracted entities to the structured backbone through entity resolution.

How much does GraphRAG indexing cost?

Cost scales with corpus size, chunk size, model price, and how many LLM passes you run. Microsoft estimates graph extraction is about 75 percent of indexing cost. LazyGraphRAG cuts indexing to the level of vector RAG, about 0.1 percent of full GraphRAG. Index a small sample first and multiply out before committing a large corpus.

What is the difference between local and global search in GraphRAG?

Local search answers questions about specific entities by combining graph neighbors with the raw text chunks that mention them. Global search answers corpus-wide questions by running a map-reduce pass over community summaries. DRIFT search starts from community information and then drills into local detail, which suits questions that mix broad themes with specifics.

How do I evaluate a GraphRAG system?

Build a test set of real questions with expected answers, labeled by type such as lookup, multi-hop, aggregate, and summary. Score retrieval and generation separately with metrics like context recall and faithfulness. For open-ended summary questions, use pairwise LLM judging against a vector RAG baseline, and spot check the judge with human reviewers.

Does GraphRAG replace vector search?

No. Most production GraphRAG systems still use vectors to find entry points, then use the graph to expand context. Microsoft GraphRAG embeds entity descriptions, chunks, and community reports by default. Neo4j retrievers pair vector search with Cypher traversal. Research also finds that plain RAG wins some task types, so keep it as a baseline.

How do I keep a GraphRAG index up to date?

For document graphs, use incremental indexing. Microsoft GraphRAG added an update command in version 1.0 that merges deltas and reuses cached LLM results. For structured data, sync source tables on a schedule or through change data capture and remap only changed rows. Either way, track which source version each node came from.

When should I not use GraphRAG?

Skip GraphRAG when questions are simple lookups answered by one passage, when the corpus is small enough to fit in context, or when answers are metrics better served by a semantic layer. GraphRAG-Bench reports that graph retrieval often trails vanilla RAG on many real tasks. Prove the gain on your own questions before paying for indexing.