feat(webhooks): lease heartbeat to prevent duplicate async processing (#1275)

The async webhook worker double-processed long-running agent calls: the
90s stale-running sweep reclaimed rows whose agent ran longer than 90s
(no heartbeat), re-running the agent and duplicating MCP side-effects.

Add a last_heartbeat_at column + a Heartbeat lease-renewal method (CAS on
lease_token). While an agent runs, the worker renews the lease every 30s;
ReclaimStale now keys off last_heartbeat_at instead of started_at, so a
live run is never reclaimed while a dead worker is still recovered within
the stale window. If a run loses its lease (reclaimed), the heartbeat
cancels the run context so the agent stops immediately and writes no
further side-effects.

- migrations/000085 (PG) + SQLite schema v52 + RequiredSchemaVersion 85
- WebhookCallStore.Heartbeat (PG + SQLite impls); ClaimNext/ReclaimStale
  switched to last_heartbeat_at
- worker heartbeat goroutine + cancel-on-lease-loss; invokeAgent honors ctx
- store + worker tests
This commit is contained in:
thotam authored and GitHub committed 2026-06-24 16:10:38 +07:00
1 parent ce0472b580
commit cd5ad845f5
15 files changed
+327 -21

No files matched your search

+1
View File
@@ -2072,6 +2072,7 @@ CREATE TABLE IF NOT EXISTS webhook_calls (
delivery_id TEXT NOT NULL,
next_attempt_at TEXT,
started_at TEXT,
last_heartbeat_at TEXT,
lease_token TEXT,
request_payload TEXT,
response TEXT,