fix(tasks): a deferred duplicate records no result, rather than success

Returning a deferred marker traded one wrong signal for a worse one. A
redelivery reuses the original task id — Context.as_execution_options carries
task_id into the retry — so the duplicate's return marked the very id the
client polls as SUCCESS. /api/task_status reports celery's state verbatim and
the UI maps SUCCESS to "done", so the GraphRAG enable modal would announce a
finished build, rendered from a payload with no counts, while the run holding
the lease was still extracting.

Raise Ignore instead: celery records no state for the duplicate, so the task
id keeps whatever the holder sets and the poller keeps waiting. The autoretry
wrapper re-raises Ignore ahead of autoretry_for, so the wider
autoretry_for=(Exception,) on these tasks cannot turn it back into a retry.
This commit is contained in:
Alex committed 2026-09-17 16:51:06 +01:00
1 parent 3f774d813c
commit b7e7872bf7
2 files changed
+21 -15

No files matched your search

+12 -10
View File
@@ -9,7 +9,7 @@ import threading
import uuid
from typing import Any, Callable, Optional
from celery.exceptions import MaxRetriesExceededError
from celery.exceptions import Ignore, MaxRetriesExceededError
from docsgpt.storage.db.repositories.idempotency import IdempotencyRepository
from docsgpt.storage.db.session import db_readonly, db_session
@@ -90,21 +90,23 @@ def with_idempotency(
)
except MaxRetriesExceededError:
# The holder is simply slower than LEASE_RETRY_MAX
# deferrals — a task that outruns the broker's visibility
# deferrals: a task that outruns the broker's visibility
# timeout is redelivered while its first run is still
# going. Standing down is the correct end state for the
# duplicate; raising here would report a failure for a
# task that is running normally somewhere else.
# going. Letting the exhaustion propagate would report a
# failure for a task that is running normally — but so
# would returning a value, only less visibly. A redelivery
# reuses the original task id (``Context`` carries
# ``task_id`` into the retry), so a return marks the very
# id the client polls SUCCESS, and ``/api/task_status``
# hands that to the UI as a finished build. ``Ignore``
# records no state at all, leaving the outcome to the run
# that actually holds the lease.
logger.info(
"idempotency: lease still held after %s deferrals; "
"leaving task=%s key=%s to its holder",
LEASE_RETRY_MAX, task_name, key,
)
return {
"status": "deferred",
"reason": "another worker holds the lease",
"idempotency_key": key,
}
raise Ignore() from None
if attempt > MAX_TASK_ATTEMPTS:
logger.error(