mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-14 00:16:20 +00:00
feat(graphrag): seed the walk from what entities are, and rank with passages and vector hits
Graph retrieval tied plain vector search at best and never beat it. Measured
across five corpora, the bottleneck was seeding, not the graph: the walk
started from nodes whose embeddings were computed from bare entity names, and
a whole question shares almost nothing with a name like "Quill".
Extraction now embeds each node from "name (type): description" and each
relationship as the fact it asserts ("Alder streams_to Quill: ..."), stored on
a new nullable graph_edges.fact_embedding column that ensure_vector_schema adds
in place. Entity names are canonicalised (case, punctuation, word breaks and a
cautious plural) so "VECTOR_STORE" and "vector stores" land on one node. Extraction calls run
concurrently (GRAPHRAG_EXTRACTION_WORKERS, default 8) while embedding and graph
writes stay serial on the task thread, so ordering and idempotency are
unchanged; that measured 8.4x faster with identical output.
Retrieval gains per-source options, stored under retrieval.graph and read live
at query time:
- seed_strategy: start from matching entities (default) or matching
relationships, which can reach an entity the question never names;
- passage_nodes (on): walk the source's passages alongside entities, with
PageRank damping 0.5 instead of 0.85;
- blend_vector (on): fuse the graph ranking with the source's vector ranking
by reciprocal rank.
The defaults are the measured-best configuration. Through GraphRAGRetriever,
the new seeding moved recall@4 from 0.41 to 0.68 on a multi-hop corpus and
from 0.50 to 1.00 on the docs corpus, and regressed none of the corpora
measured. Existing graphs keep name-only embeddings until rebuilt.
This commit is contained in:
1 parent
b7e7872bf7
commit
a83e1dc0af
13 files changed
+1577
-79
No files matched your search
@@ -12,7 +12,7 @@ reproduces today's chunking byte-for-byte.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Optional
|
||||
from typing import Literal, Optional
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, field_validator, model_validator
|
||||
|
||||
@@ -90,6 +90,27 @@ class ChunkingConfig(BaseModel):
|
||||
duplicate_headers: bool = False
|
||||
|
||||
|
||||
class GraphRetrievalConfig(BaseModel):
|
||||
"""How the graph retriever walks a graphrag source (live; no re-ingest).
|
||||
|
||||
The defaults are the configuration that measured best across the corpora
|
||||
tested rather than a neutral starting point: seed from entity matches, put
|
||||
the passages in the walk, and blend with the source's own vector ranking.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
# Where the walk starts: entities whose descriptions match the question, or
|
||||
# relationships ("A streams_to B") that do. Relationships can start the
|
||||
# walk on an entity the question never names.
|
||||
seed_strategy: Literal["entities", "relationships"] = "entities"
|
||||
# Chunks join the walk as nodes, so a passage is reachable both by being
|
||||
# about the question and by being connected to what is.
|
||||
passage_nodes: bool = True
|
||||
# Fuse the graph ranking with plain vector search by reciprocal rank.
|
||||
blend_vector: bool = True
|
||||
|
||||
|
||||
class RetrievalConfig(BaseModel):
|
||||
"""Query-time retrieval knobs (live; no re-ingest needed)."""
|
||||
|
||||
@@ -102,6 +123,7 @@ class RetrievalConfig(BaseModel):
|
||||
rephrase_query: bool = True # toggle ClassicRAG._rephrase_query side-call
|
||||
reranker: Optional[dict] = None # reserved: future cross-encoder/LLM reorder
|
||||
prescreen: Optional[dict] = None # None = off; else PreScreenConfig dict (D12)
|
||||
graph: GraphRetrievalConfig = GraphRetrievalConfig() # graphrag retriever only
|
||||
|
||||
@field_validator("chunks")
|
||||
@classmethod
|
||||
|
||||
Reference in new issue
Block a user