mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-17 18:13:36 +00:00
fix(graphrag): review fixes — replay-safe writes, strict count, zero weights
Three findings from review, all on code this branch introduced.
apply_chunk was not replay-safe. commit() can report connection loss *after*
Postgres committed, and the reconnect retry then replays the write: _upsert_node
bumps doc_freq a second time and _add_edge inserts another row, since
graph_edges has no uniqueness constraint for a logical edge. The chunk's
graph_ingest_progress row is now written in the same transaction as the rows it
describes, and a replay that finds it already "done" returns (0, 0) without
touching the graph. Extraction drops its separate mark_chunk("done"): the
checkpoint and the graph can no longer disagree.
count_nodes swallows every query failure and answers 0, so extraction's
"fall back to the write count" handler could never run — a failed count after a
successful build reported an empty graph. count_nodes grows a strict mode that
re-raises; retrieval keeps the swallow, which is what routes a source to
ClassicRAG.
A zero edge weight was read as a full-strength link: `or 1.0` rewrote an
explicit 0 before the <= 0 filter. Only missing and null weights default now.
The same coercion sat in _ppr_scores, where it would have kept the ranker's
rule unreachable from the product path, so it is fixed there too.
This commit is contained in:
1 parent
9e9f130ed0
commit
3f774d813c
6 files changed
+274
-9
No files matched your search
@@ -290,7 +290,9 @@ def extract_graph_for_source(
|
||||
)
|
||||
node_upserts += chunk_nodes
|
||||
edges += chunk_edges
|
||||
store.mark_chunk(source_id, chunk_id, "done")
|
||||
# ``apply_chunk`` marks the chunk done inside the transaction that
|
||||
# writes its rows, so the checkpoint cannot disagree with the graph
|
||||
# and a replayed write cannot apply the chunk twice.
|
||||
chunks_processed += 1
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
@@ -311,9 +313,12 @@ def extract_graph_for_source(
|
||||
# upserts and a single node, so the old count overstated every graph whose
|
||||
# entities recur. Report what the graph holds, falling back to the write
|
||||
# count only if the count query itself fails.
|
||||
# ``strict`` is what makes the fallback below reachable: the default
|
||||
# count swallows query failures and answers 0, which would report a
|
||||
# successful build as an empty graph.
|
||||
nodes = node_upserts
|
||||
try:
|
||||
nodes = store.count_nodes(source_id)
|
||||
nodes = store.count_nodes(source_id, strict=True)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"count_nodes failed for source %s; reporting upserts instead: %s",
|
||||
|
||||
Reference in new issue
Block a user