Standard retrieval-augmented generation does one thing, and it does it well: given a question, it finds the passages that most resemble the question and hands them to the model. Ask "which clause covers early termination?" and somewhere in the contract there is a paragraph about early termination. Similarity search finds it. Everyone goes home.
Now ask: "what themes recur across all five hundred of these contracts?"
There is no paragraph that answers that. The answer is a property of the collection, not of any passage inside it. You can improve the embedding model, tune the chunk size, rerank harder — and you will still be ranking passages by resemblance to a question whose answer was never written down. The failure is structural.
That gap is the entire reason GraphRAG exists. It is not a better retriever; it is a different operation. And because it is a different operation, it costs differently — which is why the honest comparison is not "which is better" but "which shape of question do you actually have, and can you afford the index?"
The Two Question Shapes
The distinction has a name in the literature. Microsoft Research's team put it plainly in the paper that introduced GraphRAG: RAG "fails on global questions directed at an entire text corpus, such as What are the main themes in the dataset?, since this is inherently a query-focused summarization (QFS) task, rather than an explicit retrieval task."
Note the second half of that sentence, because it explains why the answer wasn't simply "use a summarisation tool." Query-focused summarisation was a well-studied problem before RAG existed. The catch, as the paper says, is that prior methods "do not scale to the quantities of text indexed by typical RAG systems." You could summarise a document. You could not summarise a million-token corpus for every question a user might ask.
GraphRAG is an attempt to get summarisation's coverage at retrieval's scale.
What GraphRAG Does Instead
The trick is to do the expensive reading once, at index time, and store the result as structure.
Stage one — build the index. An LLM reads the source documents and extracts the entities in them along with the relationships between those entities, producing a knowledge graph. Community-detection then groups closely related entities into clusters, and the system pregenerates a summary for each community. This is done once, and it is the part that costs real money, because an LLM has to read everything.
Stage two — answer a question. Rather than retrieving the top few chunks, the question is put to every community summary. Each produces a partial response, and all the partial responses are then summarised into a single final answer.
If that sounds less like search and more like map-reduce, that's exactly right. Nothing is being ranked by similarity at query time. Every part of the corpus gets a vote, which is precisely how the system can answer a question whose answer is spread across the whole collection.
If retrieval basics are new to you, our guide to retrieval-augmented generation covers the vector-search half of this comparison in detail; the rest of our model coverage lives in the LLMs topic hub.
What the Evidence Actually Says
The original paper — Edge, Trinh and colleagues, first posted April 2024 and revised in February 2025 — evaluated the approach on "global sensemaking questions over datasets in the 1 million token range" and reports "substantial improvements over a conventional RAG baseline for both the comprehensiveness and diversity of generated answers."
Two things about that result deserve to be stated clearly, because summaries of this paper routinely skip both.
The win is on a specific question class, not in general. Global, corpus-wide questions are where the improvement shows up. On specific lookup queries, conventional semantic search performs better — and cheaper. Anyone presenting GraphRAG as a straight upgrade to vector RAG is describing something the research does not claim.
The metrics are judgement calls. Comprehensiveness and diversity were assessed in head-to-head comparisons scored by a language model, not against a gold-standard answer key. That is a reasonable methodology for open-ended questions that have no single correct answer — there isn't much alternative — but LLM-as-judge evaluation carries known biases, and Microsoft has since published BenchmarkQED as tooling to make this kind of comparison more systematic. Treat the direction of the finding as solid and the magnitude as approximate.
The Objection That Nearly Killed It
For most of 2024 and 2025, the practical answer to "should we use GraphRAG?" was "we costed the index and stopped there."
The reason is visible in the pipeline. Vector RAG's index cost is one embedding pass over your documents — cheap, and roughly linear. GraphRAG's index cost is an LLM reading every document and writing structured output about it, and then summarising every community it finds. On a large corpus that is a bill that needs approving.
Microsoft's own response to this was LazyGraphRAG, which restructures where the work happens. Rather than pregenerating the entire graph and every community summary up front, it combines vector and graph search on the fly and defers the heavy LLM analysis to query time. The reported effect on cost is dramatic: LazyGraphRAG's data indexing costs are "identical to vector RAG and 0.1% of the costs of full GraphRAG".
That reframes the decision. When indexing was a five-figure commitment, graph retrieval was a strategic bet. When indexing costs the same as embedding, it becomes something you can try on a Tuesday — which is why the deferred approach is a sensible default for exploratory work, one-off questions and streaming data where you'd never amortise a full index.
Query-time cost is the trade you're making instead. Consulting many community summaries and merging their answers uses far more tokens per question than fetching five chunks. You have moved the expense, not removed it — and where you want it depends entirely on your ratio of queries to documents.
Side by Side
| Vector RAG | GraphRAG (full) | Lazy / hybrid | |
|---|---|---|---|
| Core operation | Rank passages by similarity | Map-reduce over community summaries | Vector search, escalating to graph on demand |
| Index cost | Low — one embedding pass | High — an LLM reads everything | Reported as identical to vector RAG |
| Query cost | Low — a handful of chunks | High — long prompts, many partials | Paid only when a question needs it |
| Wins on | Lookups, specific facts, citations | Themes, coverage, relationships, "what's across all of this" | Mixed workloads and exploration |
| Loses on | Anything corpus-wide | Simple lookups it answers expensively | Nothing obvious — the usual starting point |
| Multi-hop questions | Weak — needs the hops in one chunk | Strong — relationships are explicit in the graph | Depends on escalation |
The multi-hop row is worth dwelling on, because it's the most common real-world case that quietly breaks vector search. "Which of our suppliers is exposed to the same shipping route as our largest customer?" requires connecting a supplier to a route, a route to a customer, and a customer to a revenue ranking. No single chunk contains that chain. A graph has the edges; an embedding space only has proximity.
How to Choose
A workable order of operations:
- Write down twenty real questions your users ask. Not hypotheticals — actual queries, from logs if you have them.
- Sort them into lookup and corpus-wide. If nearly all are lookups, stop. You need better chunking and reranking, not a graph.
- If a meaningful share are corpus-wide, start with the deferred approach. The index costs what embedding costs, so the experiment is nearly free and the result tells you whether graph structure helps your data.
- Only commit to a full pregenerated index when the same corpus-wide questions get asked repeatedly by many users, so pregeneration amortises.
- Route rather than replace. Classify the incoming question and send it down the cheaper path when the cheaper path suffices. This is unglamorous and it is what production systems converge on.
Retrieval rarely sits on its own, either. Inside an agent loop a retriever is simply one more tool the model can request, and MCP is increasingly how that tool gets connected to the data in the first place — neither of which changes the choice above, but both determine how the retrieved text reaches the model.
One more consideration that has nothing to do with quality: a knowledge graph is inspectable. You can look at the entities and edges the extraction produced, spot that it merged two different people with the same name, and fix it. An embedding index offers no comparable surface. For regulated or high-stakes work, auditability can matter more than the win rate.
Common Misconceptions
- "GraphRAG replaces vector RAG." The research says the opposite: semantic search wins on specific lookups. They answer different question shapes.
- "A knowledge graph is more accurate." It is more structured. Accuracy still depends on an LLM having extracted entities and relationships correctly from your documents — and it will make mistakes, which is exactly why inspectability is useful.
- "Graph retrieval is too expensive to consider." That was true of the original pregenerated approach. The deferred variant reports indexing at parity with vector RAG.
- "You need a graph database." You need graph structure. Whether it lives in a dedicated graph store, a relational schema or a document store is an implementation choice, not a prerequisite.
- "Better embeddings would fix the corpus-wide case." No embedding can retrieve a passage that does not exist. The problem is the operation, not the representation.
The Bottom Line
Vector RAG answers "where is this written down?" GraphRAG answers "what does all of this add up to?" Both are legitimate; only one of them is what most teams have actually built.
The decision is therefore not a technology preference. Look at the questions your users ask. If the answers exist in passages, similarity search is the right tool and a graph is expensive overhead. If the answers are distributed across the corpus — themes, patterns, relationships spanning documents — no amount of retrieval tuning will produce them, because retrieval is not the operation you need.
And the cost objection that made this an either/or has largely dissolved. With deferred indexing reported at vector-RAG cost, the sensible posture is to keep the cheap path for the cheap questions, add the expensive path for the questions that need it, and route between them.
Frequently Asked Questions
What is GraphRAG in one sentence?
It is a retrieval approach that has an LLM read your documents once to build a knowledge graph of entities and their relationships, pregenerates a summary for each cluster of related entities, and then answers a question by collecting a partial answer from every cluster summary and merging them. It was introduced by a Microsoft Research team in the 2024 paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization."
When does vector RAG genuinely beat GraphRAG?
On specific lookup queries — the kind where a passage in your corpus contains the answer. It is faster, dramatically cheaper to index, cheaper per query, and the research itself reports semantic search performing better on this class. If your users mostly ask "what does the policy say about X," you do not have a GraphRAG problem.
Isn't GraphRAG prohibitively expensive?
Full pregenerated GraphRAG has a high index cost, because an LLM must read every document and summarise every community it finds. That objection stalled a lot of projects. Microsoft's LazyGraphRAG defers the heavy analysis to query time and reports indexing costs identical to vector RAG — 0.1% of full GraphRAG — which makes trying it cheap. You then pay more per query instead.
What is a "sensemaking" or "global" question?
One whose answer is a property of the whole collection rather than of any passage in it: main themes, recurring patterns, how two things relate across many documents, what changed over a set of reports. The formal framing is query-focused summarisation. The tell is simple — if you cannot point to a paragraph that would answer it, retrieval alone won't.
Does GraphRAG help with multi-hop questions?
Yes, and this is one of its stronger practical arguments. A question requiring you to connect A to B and B to C fails under vector search unless one chunk happens to contain the whole chain. A graph stores those relationships as explicit edges, so the connection is traversable rather than something the retriever has to get lucky about.
Do I need a graph database to use it?
No. What you need is the graph structure — entities, edges and cluster summaries. Where you store it is an engineering decision, and plenty of implementations use ordinary relational or document stores. Microsoft's GraphRAG implementation is open source, which is the usual starting point for evaluating it against your own corpus.
How do I decide without building both?
Collect twenty real user questions and sort them into lookup versus corpus-wide. That single exercise resolves most cases in an afternoon. If the split is meaningful, prototype with the deferred approach — because indexing is at parity with vector RAG, the experiment costs little, and it tells you whether graph structure helps your data specifically rather than the paper's benchmark corpora.
Sources
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization — Edge, Trinh et al., arXiv:2404.16130
- GraphRAG: Unlocking LLM discovery on narrative private data — Microsoft Research
- LazyGraphRAG: Setting a new standard for quality and cost — Microsoft Research
- BenchmarkQED: Automated benchmarking of RAG systems — Microsoft Research
- microsoft/graphrag — reference implementation on GitHub
