feat(webhooks): stream provider responses for server-side runs to enable prompt caching (#1273)

Server-side webhook agent runs (sync, async, and admin test) used Stream:false,
so OpenAI-compatible routers that only cache streaming requests never populated
or served their prompt cache. Webhook runs paid full input-token price on every
turn even with a stable session and an identical multi-turn prefix, while WS chat
(Stream:true) got cache hits from the 3rd turn.

Add gateway.webhook_stream (default true, env GOCLAW_WEBHOOK_STREAM) and apply it
to all three run sites. ChatStream returns the fully assembled response, so the
payload returned to the caller is unchanged. Set to false to restore non-streaming.
This commit is contained in:
thotam authored and GitHub committed 2026-06-24 13:42:20 +07:00
1 parent 68ad5953d7
commit e2ec3370dd
11 files changed
+108 -3

No files matched your search

+1
View File
@@ -234,6 +234,7 @@ func (d *gatewayDeps) wireHTTPHandlersOnServer(
sharedWebhookLimiter, // K10: shared limiter
nil, // lane: nil → internal default (4-slot); configurable in future via cfg
webhooks.ResolveTimeoutSec(d.cfg.Gateway.WebhookSyncTimeoutSec),
webhooks.ResolveStream(d.cfg.Gateway.WebhookStream),
)
llmH.SetEncKey(webhookEncKey) // K6: decrypt secret at HMAC verify time
d.server.SetWebhookLLMHandler(llmH)
+1
View File
@@ -173,6 +173,7 @@ func (d *gatewayDeps) runLifecycle(
WorkerConcurrency: workerConcurrency,
PerTenantConcurrency: 4,
AsyncAgentTimeout: webhooks.ResolveTimeoutSec(d.cfg.Gateway.WebhookAsyncTimeoutSec),
Stream: webhooks.ResolveStream(d.cfg.Gateway.WebhookStream),
},
)
// K6: decrypt raw secret for outbound HMAC signing using the same key as inbound verify.