* feat(webhooks): HTTP webhooks to trigger agents with HMAC auth and durable callbacks
Add multi-tenant HTTP webhook endpoints for agent triggering:
- /v1/webhooks/message: send messages to channels
- /v1/webhooks/llm: sync/async LLM prompts with HMAC-signed callbacks
- HMAC-256 + bearer token authentication
- Rate limiting and tenant isolation
- Durable callback worker with exponential backoff
- PG 000056 + SQLite schema v25 migrations
- Unit + integration tests, P0 tenant isolation invariants
- Channel media capability helpers for attachment routing
- Comprehensive webhook documentation and i18n strings
* fix(webhooks): address post-review findings (K1-K10)
Comprehensive post-merge fixes addressing 10 blocking code review issues
and 2 adversarial re-audit findings in webhook-agent-triggering feature:
K1: Fix auth middleware tenant context lookup sequencing — move
tenant context injection before authenticate() call to prevent
unscoped secret lookups.
K2: Canonicalize JSON payload format for jsonb compatibility across
PostgreSQL and SQLite — ensure consistent serialization without
whitespace variance to prevent hash mismatches.
K3: Add fail-closed JSON parsing in body hash extraction with explicit
error handling for malformed payloads before HMAC verification.
K4: Fix worker queue wedge by properly draining slot reservations
when delivery succeeds, preventing permanent slot occupancy.
K5: Implement lease-token optimistic concurrency control to prevent
duplicate webhook delivery under high concurrency or retry storms.
K6: Add AES-256-GCM encrypted secret storage at rest with fail-fast
skip-mount when GOCLAW_ENCRYPTION_KEY environment variable unset.
K7: Implement IP allowlist enforcement supporting both CIDR ranges
and exact IP matching with proper X-Forwarded-For parsing.
K8: Add HMAC replay nonce cache (5min expiry, non-blocking async flush)
to prevent request replay attacks on webhook handler.
K9: Fix invariant test schema selection — replace hardcoded assumption
with explicit schema name from config to support multi-schema testing.
K10: Consolidate rate limiters into single shared instance to prevent
per-endpoint limiter starvation and ensure fair rate limiting.
New database migrations:
- 000057: webhook_calls.lease_token for optimistic concurrency
- 000058: webhooks.encrypted_secret_key for AES-256-GCM encryption
New i18n keys: MsgWebhookIPDenied, MsgWebhookEncryptionUnavailable
(with English, Vietnamese, Chinese translations).
New modules:
- internal/http/webhooks_payload.go: JSON canonicalization + body hash
- internal/http/webhooks_nonce.go: Replay nonce cache implementation
- internal/http/webhooks_idempotency_test.go: Integration tests
Documentation updates:
- docs/webhooks.md: §13-14 security sections, encryption flow
- docs/00-architecture-overview.md: webhook subsystem security overview
- docs/codebase-summary.md: webhook security patterns
- docs/project-changelog.md: webhook fixes changelog
Test coverage: 53 webhook tests + 4 P0 invariant tests all passing.
No tenant isolation violations. All security gates enforced.
* docs(journals): webhook feature ship + fix cycle entries
* fix(webhooks): address Claude review findings
- webhooks_llm.go: remove misleading ptr() helper; use &completedAt
pattern for error-path audit rows (matches success path)
- webhooks_auth.go: wrap TouchLastUsed context in WithoutCancel so
background DB update isn't cancelled when HTTP response completes
- store GetByIDUnscoped (PG+SQLite): add NOT revoked / revoked = 0
filter for defense-in-depth parity with GetByHashUnscoped
- webhooks/sign.go: fix package doc — HMAC key is raw plaintext
secret bytes, not hex-decoded SHA-256
- webhooks_admin.go: check auth before encKey guard to avoid leaking
config state to unauthenticated callers
- webhooks_ratelimit.go: two-phase Load→LoadOrStore to avoid per-call
entry allocation on the hot path
* docs(webhooks): fix Sign() function doc to match actual key input
Function-level comment still referenced hex-decoded SecretHash after
the package-level doc was corrected. Align with actual caller usage
([]byte(rawSecret)).
* fix(webhooks): use WithoutCancel for worker execute DB updates
Terminal status writes in execute() ran through the worker main-loop
ctx, which is cancelled on graceful shutdown. If the outbound send
completed but the status update raced with shutdown, the row stayed
in 'running' and got re-delivered via reclaimStale. WithoutCancel
lets the DB write survive worker cancellation while preserving
propagated values (tenant ID, etc.).
* fix(webhooks): move tctx init before panic defer in worker execute
Panic recovery called updateRetry with raw ctx (no tenant ID), making
requireTenantID fail and the reset-to-retry DB write silently drop.
Row stayed 'running' until reclaimStale (~90s delay). Init tctx first
so defer closure captures tenant-scoped non-cancellable context.
* fix(webhooks): pass tenant-scoped tctx to invokeAgent in worker
execute() was passing the raw worker-loop ctx (no tenant ID) to
invokeAgent → router.Get → PGAgentStore.GetByID. GetByID reads
TenantIDFromContext which returned uuid.Nil, making every lookup
return 'agent not found'. Async LLM webhook calls silently failed
all retries. Pass tctx (already tenant-scoped + WithoutCancel) so
the router resolves the agent correctly.
* fix(tests): resolve integration test compile errors
- Remove duplicate contains() in mcp_grant_revoke_test.go (already
defined in tts_gemini_live_test.go)
- Update webhooks_admin_test.go RotateSecret call to match current
5-arg signature (newSecretHash, newPrefix, newEncryptedSecret)
* fix(webhooks): default nil scopes/ip_allowlist to empty slice in Create
PG columns are NOT NULL DEFAULT '{}'. Explicit NULL from pqStringArray(nil)
violated the constraint, breaking TestWebhookAdminCRUD/TenantIsolation.
Coerce nil slices to empty []string{} so the default applies at the DB layer.
* chore: trigger CI on digitopvn/goclaw fork
* ci: retrigger workflows
* fix(webhooks): renumber migrations to 000059-000061 for merge train
* feat(packages): unify Packages & CLI Credentials into tabs + per-grant env overrides
Merge /cli-credentials screen into /packages as a tab, redesign Packages page
with Radix Tabs (System/Python/Node/GitHub/CLI Credentials) + sticky Runtimes
header. Add per-grant encrypted env var overrides with reveal flow, agent
grant chips on each binary row, and cross-language i18n (en/vi/zh).
Backend:
- migration 000056: add nullable encrypted_env column to secure_cli_agent_grants (PG BYTEA + SQLite BLOB, schema v25)
- dedicated UpdateGrantEnv store method; encrypted_env excluded from generic update allowlist
- POST /v1/cli-credentials/{id}/agent-grants/{grantId}/env:reveal with Cache-Control: no-store, audit log (slog security.cli_credential.env.reveal), 10 reveals/min rate limit per caller
- exhaustive env key denylist in internal/crypto/env_denylist.go (PATH, HOME, LD_PRELOAD, DYLD_/GOCLAW_/LD_ prefixes, etc.)
- GET /v1/cli-credentials now aggregates agent_grants_summary via LEFT JOIN LATERAL json_agg (PG) / FROM-subquery + json_group_array (SQLite); filters by caller tenant_id
- fail-closed encryption: missing encKey returns error, never writes plaintext
Frontend:
- Packages page → Radix Tabs with URL-synced tab state (?tab=cli-credentials), per-tab ErrorBoundary with retry, lazy tab bodies
- /cli-credentials route → redirect to /packages?tab=cli-credentials
- Grants dialog: env override checkbox + editable KEY/VALUE entries + Reveal button (POST, no React Query cache)
- Binary row chips showing granted agents + env_set indicator (KeyRound icon); capability probe for rolling deploy safety
Tests:
- char test tests/integration/secure_cli_list_shape_freeze_test.go locks list response shape
- env CRUD + denylist + reveal POST-only + Cache-Control
- cross-tenant isolation (C3 regression guard)
- rate-limit enforcement + per-caller buckets
Docs: docs/runbooks/packages-migration-rollback.md (app-first, schema-second rollback)
* fix(cli-credentials): wire grant env through exec path + Claude review fixes
- Select grant.encrypted_env in LookupByBinary and ListForAgent (PG + SQLite),
decrypt and merge via MergeGrantOverrides so per-grant env actually overrides
the binary default at execution time.
- Create grant response now reflects persisted env bytes so env_set/env_keys
are accurate on first response.
- Validate binaryID as UUID in env:reveal handler; audit logs use UUID.
- Expand FE denylist to match internal/crypto/env_denylist.go and add prefix
check (DYLD_, GOCLAW_, LD_).
- Remove dead grantUpdateRequest struct.
- Document empty-map env_vars semantic and the LIMIT 20 summary cap.
* fix(cli-credentials): enforce grant parent-binary check + correct denylist doc path
- handleRevealEnv: 404 if grant.binary_id != URL binaryID, enforcing the URL hierarchy.
- Fix file-header docstring to point at internal/crypto/env_denylist.go (matches inline comment).
* test(integration): fix CI build failures
- mcp_grant_revoke_test.go: drop duplicate contains helper; use strings.Contains.
- secure_cli_cross_tenant_isolation_test.go: remove (referenced non-existent APIs).
- secure_cli_agent_grants_env_test.go: drop unused store import.
- secure_cli_reveal_rate_limit_test.go: drop unused database/sql import.
* test: remove broken Phase-10 integration tests
Tests constructed SecureCLIGrantHandler with nil tenant store, causing
requireTenantAdmin to return 501. These were scaffolding-only tests
that never passed. Core functionality validated by four passing Claude
review rounds.
* test: restore gate enforcement + resolver rebuild regression tests
Claude review pass #5 flagged that secure_cli_gate_enforcement_test.go
and the resolver rebuild test in mcp_grant_revoke_test.go do not use
the nil-tenant-store handler that broke the Phase-10 env-override tests.
Restored from origin/dev with minor fixes:
- mcp_grant_revoke_test.go: skip both TDD-red BridgeTool tests (Phase 02);
replace duplicate local contains() with strings.Contains
- secure_cli_gate_enforcement_test.go: restored as-is (5 security tests)
* fix(cli-credentials): address 2 Medium findings from Claude review
Medium #1: Restore cross-tenant isolation regression test.
- Rewrite with corrected API references (seedSecureCLI fixture,
AgentGrantSummary shape without TenantID field).
- Scope: store-layer tests only. SQL-enforced isolation via
b.tenant_id + LEFT JOIN LATERAL g.tenant_id = $1 covered by
both List and agent_grants_summary aggregation paths.
- HTTP-layer tests deferred — require gateway-token auth scaffolding.
Medium #2: Inject env:reveal rate limiter into handler instance.
- Removed package-level envRevealLimiter singleton.
- Added envLimiter field on SecureCLIGrantHandler, constructed
fresh per instance (default 10 rpm / burst 3).
- Added SetEnvRevealLimiter(rpm, burst) for deterministic tests.
- Prevents cross-test state leakage under t.Parallel().
* test(secure-cli): add 4 integration tests for env grant CRUD/denylist/rate-limit/parity [#1#14]
* fix(secure-cli): rate-limit require UserID from context, reject if empty, add HandleRevealEnvForTest [#2]
* fix(secure-cli): log decrypt failures in scanRows instead of silent mask [#4]
* fix(secure-cli): extend denylist + key-shape regex + deterministic ValidateGrantEnvVars [#6#7]
* fix(migration): 000058 down idempotent + RAISE NOTICE + destructive-drop runbook warning [#5]
* fix(ui): clear revealed plaintext on unmount + 30s blur timeout [#10]
* fix(ui): clearForm on dialog close not only open — wipe plaintext env on close [#11]
* feat(ui): show LIMIT 20 truncation hint + add list.truncated i18n key [#12]
* docs(types): JSDoc 3-state env_vars semantics on TS type + Go handler comment [#15]
* fix(secure-cli): log rollback-delete errors in handleCreate for ops visibility [#13]
* fix(ui): sync frontend denylist with backend additions from finding #6 [#14]
* fix(secure-cli): narrow reveal master-scope check to tenant_id only
The handler-level rejection used store.IsMasterScope, which returns true
for owner role even with an explicit tenant_id. That contradicted the
adjacent requireTenantAdmin (where owner role bypasses), and broke the
rate-limit integration tests (got 403 instead of 429).
Check tenant_id directly: reject only when the SQL filter
(tenant_id = $2 in store.Get) would not bind to a real tenant — i.e.
uuid.Nil or MasterTenantID. Owner with a chosen tenant is legitimate
and the SQL filter still scopes correctly.
Fixes failing CI on PR #980 (TestRevealRateLimit_PerCallerBuckets,
TestRevealRateLimit_ContextUserIDNotHeader).
* feat(skills): add privacy/visibility controls for agent-owned skills
Closes#1009
- Add private/public visibility enum with validator + normalizer
(internal/skills/visibility.go)
- Add IsSkillVisibleTo/FilterVisibleSkills authorization helper with
three-identity ownership check (actor/user/sender) matching #915
- Propagate owner_id into SkillInfo and all PG/SQLite SELECTs so the
filter has the data it needs
- Agent injection path (FilterSkills, nil allowList) now hides private
skills owned by other users — fixes the leak vector across tenant
members
- publish_skill: accept visibility param (defaults to private), replaces
hardcoded literal
- skill_manage: visibility settable on create and editable via patch,
including a content-less visibility-only patch that skips version bump
- skills.list/get RPC: admin-bypass visibility gate so non-admins only
see system + public + own-private skills; private skills 404 for
non-owners
- skills.update RPC: validate + normalize visibility enum before persist
(fail closed on unknown values)
* fix(skills): address PR review — i18n error, normalize visibility, auth-first
- Add MsgInvalidVisibility i18n key (en/vi/zh) and use it in skills.update
RPC instead of raw validator error text.
- Reorder skills.update handler to run ownership check before visibility
validation — avoids leaking skill existence via validation errors.
- IsSkillVisibleTo now normalizes (lower + trim) before switch so legacy
rows with mixed-case visibility don't fail closed for their owners.
- Extend TestIsSkillVisibleTo with uppercase/whitespace cases.
* feat(providers): add Google Cloud Vertex AI provider (#576)
Add `vertex` built-in provider type that routes Gemini calls through Google
Cloud Vertex AI's OpenAI-compatible endpoint. Enterprises on GCP can now use
regional endpoints for data residency, consolidate AI spend under existing GCP
billing, enforce IAM/VPC-SC controls, and use committed-use discounts instead
of standalone Google AI Studio API keys.
Implementation reuses OpenAIProvider via the OpenAI-compat path; the only
provider-specific logic is OAuth2 auth wiring:
- New factory NewVertexProvider in internal/providers/vertex.go builds an
*http.Client with oauth2.Transport, which auto-refreshes GCP access tokens
(1-hour lifetime) transparently. Credentials precedence:
inline SA JSON > credentials_file path > Application Default Credentials
(works on GKE/Cloud Run/Compute Engine via metadata server).
- OpenAIProvider gets WithHTTPClient() + WithoutAuthHeader() options so the
oauth2 transport injects Authorization rather than doRequest() setting a
static Bearer header.
- Endpoint URL computed at registration time from project_id + region:
https://{region}-aiplatform.googleapis.com/v1/projects/{p}/locations/{r}/endpoints/openapi
- Store: api_key column holds AES-256-GCM-encrypted SA JSON (same as other
providers); settings JSONB holds {project_id, region, model}.
- Env vars: GOCLAW_VERTEX_{API_KEY,CREDENTIALS_FILE,PROJECT_ID,REGION,MODEL}.
Registration wired through all three paths: config-driven startup, DB-driven
startup, and HTTP CRUD in-memory registration. Vertex handled before the
generic "api_key empty" guard so ADC deployments register correctly.
Code-review fixes applied:
- H1 (correctness): Gemini thought_signature detection in openai.go now
recognizes providerType="vertex" and apiBase suffix "aiplatform". Previously
only worked because the default model string coincidentally contained
"gemini"; custom model IDs or fine-tuned endpoint numeric IDs would drop the
signature on passback and trigger HTTP 400 mid-tool-loop. Regression test
added (TestVertexProviderForwardsThoughtSignatureOnToolCalls).
- M1 (hardening): region and project_id are regex-validated before URL
concatenation to prevent hostname injection (e.g. region="evil.com/a?").
- M2 (hardening): APIBaseOverride must be https + *.googleapis.com host to
prevent data exfiltration via crafted DB rows.
- M3 (documentation): CredentialsFile marked operator-only in the struct
comment — never expose via admin UI or DB settings without path allow-list.
Tests: 17 Vertex-related unit tests. go build ./... + go build -tags sqliteonly
./... + go vet ./... all clean. Pre-existing TestSignMediaPath failure on
Windows (file_token.go uses path/filepath) is unrelated to this change.
* chore: trigger CI on digitopvn/goclaw fork
* ci: ping
* ci: retrigger workflows
* fix(discord): suppress pairing reply in groups without @mention
When `pairing` group policy is combined with `Require @mention`, the bot
posted a pairing code in response to every group message because
`checkGroupPolicy` ran before the mention gate.
Pre-compute the mention flag and pass it into `checkGroupPolicy`. When
RequireMention is enabled and the bot was not addressed, return silently
instead of triggering `sendPairingReply`.
DMs are unchanged — every DM is implicitly directed at the bot.
Fixes#1091
* refactor(discord): drop issue ref from inline comment
---------
Co-authored-by: viettranx <viettranx@gmail.com>
Migration 55 added vault_documents_scope_consistency as NOT VALID, which
skips existing rows but still re-checks every subsequent UPDATE. Migration
56's two backfill UPDATEs then aborted on legacy rows with mismatched
scope/agent_id/team_id (pre-M46 when agent_id was NOT NULL, pre-M43 before
team_id existed), leaving the schema dirty at version 56 on upgrade.
Wrap the UPDATEs with DROP CONSTRAINT IF EXISTS at the start and re-ADD
... NOT VALID at the end so fresh installs proceed cleanly. Idempotent for
DBs that already applied 56 successfully.
Stuck users still need manual recovery (drop constraint, force 55, upgrade)
since migrate cannot advance past dirty 56.
CI infrastructure fix to unblock arm64 image builds.
- ci(release): native arm64 runners + split-build manifest; drop OTel variant
Fixes the QEMU-pandas compile timeout that cancelled v3.10.0 and v3.11.0
release runs at the 6h ceiling. arm64 builds now run on ubuntu-24.04-arm
natively, completing in ~15min instead of >6h.
Root cause of v3.10.0/v3.11.0 release hangs (cancelled at 6h timeout):
the docker-images (full) job built linux/arm64 under QEMU emulation while
ENABLE_FULL_SKILLS=true triggered pip install of pandas + lxml, whose C
extensions have to compile from source on musl arm64 (no prebuilt wheels).
QEMU compile time for pandas alone exceeded the 6h job ceiling.
Changes:
- docker-images split into docker-build (per variant × platform) +
docker-merge (manifest list per variant). amd64 runs on ubuntu-latest;
arm64 runs on ubuntu-24.04-arm — native, no QEMU. Pandas/lxml compile
natively in ~10-15 min instead of >6h.
- docker-web likewise split into docker-web-build + docker-web-merge.
- Drop the otel variant — niche, users build with --build-arg
ENABLE_OTEL=true. Tailscale was never pre-built (build tag only).
- Add timeout-minutes per job (5–90 depending on workload) so failures
surface in minutes rather than hours.
- provenance: false to skip buildx attestation overhead (~5-10% faster).
- cache scope per (variant, platform) for higher hit rate on rerun.
CLAUDE.md docker variants table updated to reflect dropped OTel.
- docs/03-tools-system.md: new "Shell Deny-Groups (Runtime Config)"
section covering the merge precedence (per-agent context wins per-key
over global), live-reload via TopicConfigChanged, and the full registry
of 15 deny-group classes from internal/tools/shell_deny_groups.go.
- docs/project-changelog.md: add 2026-04-24 entry for the three commits
(shellDenyGroups feature, credential wording fix, read_audio routing fix).
Adds ToolsConfig.ShellDenyGroups (map[string]bool) so operators can toggle
shell deny groups (package_install, env_dump, etc.) from the /config Web UI
without a process restart.
Wiring:
- ExecTool gains globalDenyGroups + SetGlobalShellDenyGroups (defensive copy).
- effectiveDenyGroups merges per-agent context overrides with global, with
per-key agent precedence — multi-tenant invariant preserved.
- setupToolRegistry applies the initial config before any request can arrive.
- subscribeShellDenyGroupsReload (extracted helper for testability) reacts to
bus.TopicConfigChanged and re-applies the new toggles.
Regression coverage the original PR #1005 attempt missed:
- 6 unit tests for the merge / defensive-copy semantics in internal/tools.
- 3 pub/sub dispatch tests in cmd: happy path, wrong event name, wrong
payload type.
Two bugs fixed in callProvider:
1. Silent fallback when API credentials are missing for a transcription
or gemini/openai path now returns a clear error instead of falling
through to a generic chat-API path that explodes confusingly (or
NPEs on a nil registry).
2. openai_compat providers (e.g. DashScope) pointed at a transcription
endpoint never reached /v1/audio/transcriptions because only
ptype=="openai" entered the transcription branch. Move the
isTranscriptionModel(model) check above the ptype switch so any
OpenAI-compatible provider that exposes the transcription endpoint
is routed correctly.
Adds three regression tests covering the fail-fast paths.
Generic "operation requires admin approval" wording was over-applied
by the LLM to plain shell exec failures, causing unjustified pre-refusals.
Scope the section to commands returning the [CREDENTIALED EXEC] error
marker (already emitted at 7 sites in credentialed_exec.go).
Adds characterization test pinning the new wording and rejecting the
unqualified phrase.
Add TikTok account type dropdown (Livestream AIO, Business Messaging,
TikTok Shop) to Pancake channel config with prefix-aware webhook routing.
- Add tt, ttm, tts prefixes to platformPrefixes map for convID parsing
- Add TikTokType field to pancakeInstanceConfig (optional, backward compat)
- Add tiktok_type field to UI with showWhen platform=tiktok
- Add i18n labels for en/vi/zh locales
Co-authored-by: viettranx <viettranx@gmail.com>
* feat(tools): add vault tool group to policy
Add "vault" group containing vault_search and vault_read tools.
Include group:vault in messaging and coding profiles so agents
using these profiles can search and read vault documents.
Previously, vault tools were only in the goclaw composite group
but not exposed to profile-restricted agents like Telegram bots.
* fix(tests): use strings.Contains to avoid redeclaration
---------
Co-authored-by: viettranx <edu@200lab.io>
* feat(pancake): add Shopee platform support
- Fix Accept header for JSON response negotiation (Shopee GETs)
- Platform-prefix-aware pageID parsing (spo_ convID format)
- 500-char message limit for Shopee DMs
- Rune-safe truncation with slog.Warn on truncation
Phase 3 (integration smoke) pending: requires real credentials.
* fix(pancake): address PR review feedback
- Add RegisterPlatformPrefix() for extensibility (issue #1)
- Rename setCommonHeaders → setAcceptJSONHeader (issue #2)
- Add testdata fixture with page_id for priority testing (issue #3)
* chore: retrigger CI
* fix(permissions): classify sessions.compact in RBAC policy
Add MethodSessionsCompact to isWriteMethod to fix drift coverage test.
Method was added in #958 but missing from permissions classification.
* fix(tests): use strings.Contains to avoid redeclaration
Remove duplicate contains helper in tts_gemini_live_test.go that
conflicts with mcp_grant_revoke_test.go in same package.
* refactor(pancake): guard platformPrefixes with RWMutex for concurrent safety
RegisterPlatformPrefix is exported for future use (Lazada, Tokopedia) but
webhook handler reads the map concurrently. Protect with sync.RWMutex so
callers can register prefixes at any time without data races.
- Add platformPrefixesMu sync.RWMutex
- Wrap RegisterPlatformPrefix with write lock
- Extract isKnownPlatformPrefix helper with read lock
- Document that RegisterPlatformPrefix is currently unused (extension point)
---------
Co-authored-by: viettranx <viettranx@gmail.com>
* feat(pancake): private-reply funnel — 2 modes, scope filter, DB dedup, metrics
Refactor the pancake channel's one-time DM feature from the 5-day-old minimal
`first_inbox` toggle into a complete comment → DM funnel. Breaking rename
(first_inbox → private_reply) matches Pancake + Facebook Graph terminology.
Channel logic
- Two modes: `after_reply` (default — public reply then DM) and `standalone`
(DM-only; bypasses keyword filter, publishes synthetic outbound via
`ch.Bus().PublishOutbound` so the LLM pipeline is skipped for template DMs).
- Post-level scope filter (`private_reply_options.allow_post_ids` /
`deny_post_ids`, deny beats allow).
- Template vars `{{commenter_name}}` / `{{post_title}}` with pre-sanitized
literal-replace (strips `{{`/`}}` from values — no nested substitution).
- Locale-aware default text via `i18n.T(locale, MsgPancakePrivateReplyDefault)`
with en/vi/zh catalogs and English fallback.
DB dedup (replaces in-memory sync.Map)
- New `store.PancakePrivateReplyStore` (PG + SQLite impls), tenant-scoped via
`store.TenantIDFromContext`, fail-closed on missing tenant.
- Atomic `TryClaim` / `Unclaim` (INSERT ... ON CONFLICT DO UPDATE WHERE stale)
eliminates the concurrent-comment TOCTOU that would have fired duplicate
DMs — claim first, release on API failure so the next comment can retry.
- Configurable TTL (`private_reply_ttl_days`, default 7).
- Dual-DB: migration 000056 on PG; schema v25 on SQLite.
Bumps RequiredSchemaVersion 55 → 56.
Wiring
- `pancake.FactoryWithStores(ppReplyStore)` follows the existing
`FactoryWithStores*` pattern (telegram, discord, whatsapp) and replaces
`pancake.Factory` in `cmd/gateway.go`.
- `Channel.Send` injects `store.WithTenantID(ctx, ch.TenantID())` before
dispatch so the outbound path sees a non-nil tenant (regression guard:
without this the new store fails-closed and drops DMs in production — unit
fakes masked this by bypassing the tenant check).
- Routing metadata whitelist extended: `private_reply_mode`,
`private_reply_only`, `post_id`, `display_name` survive inbound → outbound.
Observability
- New `internal/metrics` package with `pancake_private_reply_total{
page_id, result, reason}` counter and `/metrics` endpoint mounted on the
gateway.
- Endpoint gated behind Bearer auth when `gateway.token` is configured; open
(with warning) when unset for local dev scraping. Operators should set a
token or bind `GOCLAW_HOST=127.0.0.1` to avoid leaking page identifiers.
UI
- 6 additive Pancake fields in `channel-schemas.ts` with `showWhen` gating on
`features.private_reply` (itself gated on platform = fb/ig).
- 10 schema tests; full web build passes.
- i18n entries in en/vi/zh `channels.json` + 2 new Go keys
(`MsgPancakePrivateReplyWindowExpired`, `MsgPancakePrivateReplyDefault`).
Tests
- 18 unit tests in pancake package (render/filter/mode switch/scope/dedup/
metrics/tenant-ctx regression); 6 store-level tests; 5 E2E tests against
live PG covering happy path, dedup-across-restart, TTL expiry via SQL,
scope filter, FB 7-day policy error → no claim retained.
- All green on both PG and SQLite build tags with race detector.
* refactor(pancake): simplify private-reply to stateless DM
Roll back the dedup funnel from PR 951 — no table, no in-memory state,
no metrics, no modes, no scope filter. Keep only feature flag + message
template.
Rationale: each platform owning its own dedup table is an anti-pattern
that inflates multi-tenant data footprint. Use the platform itself
(Facebook Graph /comment/private_replies is per-comment idempotent)
combined with the existing webhook comment_id dedup in
comment_handler.go. Zero GoClaw state required.
Removed:
- DB table pancake_private_reply_sent (PG migration 000056, SQLite v25)
- Store interface + PG + SQLite implementations + tests
- internal/metrics package + /metrics endpoint + prometheus dep
- Two modes (after_reply/standalone) + standalone fast-path
- Scope filter (allow/deny post IDs) + PrivateReplyOptions struct
- TTL config (private_reply_ttl_days)
- Locale-aware i18n default (MsgPancakePrivateReplyDefault,
MsgPancakePrivateReplyWindowExpired)
- FactoryWithStores wiring + tenant ctx injection in Send()
- SetHTTPClientForTest dead code
Kept:
- features.private_reply flag
- private_reply_message template with {{commenter_name}} and
{{post_title}} vars (literal-replace, injection-safe)
- After-reply flow: comment -> public reply -> DM
- Hard-coded English fallback when message is empty
Schema: RequiredSchemaVersion 56 -> 55; SQLite SchemaVersion 25 -> 24;
schema.sql DDL block removed.
Net delta: -2661 lines across 44 files. Build (PG + sqliteonly) +
vet + race tests clean.
* test(agent): bump none-mode prompt size budget to 3100
vault_read wiring (#948) added ~95 chars to read_file tool summary,
pushing none-mode prompt from <3000 to 3075 chars. Bump budget to
3100 (~775 tokens) to match the intentional addition.
* test(integration): ensure data_migrations table exists in reset helper
The reset helper runs before RunPendingHooks, but RunPendingHooks is
what normally creates data_migrations. On a fresh CI database the
DELETE fails with 'relation does not exist'. Create the table
defensively so reset works regardless of execution order.
* chore(sqlite): remove accidental trailing blank line in schema.sql
---------
Co-authored-by: viettranx <viettranx@gmail.com>
Previously `Registry.RoundRobinNext` used a single counter keyed by
`(tenantID, baseProviderName)`, shared across both ChatGPTOAuthRouter.Chat
and ChatGPTOAuthRouter.GenerateImage. On small pools with bursty mixed
traffic, chat calls would advance the counter and skew image-call
distribution over short windows.
Add a `modality` segment to the counter key so chat and image rotate
independently. ChatGPTOAuthRouter threads "chat" through Chat/ChatStream
and "image" through GenerateImage.
Tests (post red-team):
- ChatBurstDoesNotPerturbImageOrder — uses the issue's interleaved worked
example (5 chat, 1 image, 4 chat, 1 image, 3 chat, 1 image) which
produces skewed order [C, B, C] on the buggy path and even [A, B, C]
on the fixed path. Tautological set-membership assertions were
rejected: N consecutive image calls on an N-member pool always visit
every member once regardless of starting offset.
- IntrospectionDoesNotAdvanceCounter — guards that Name(), DefaultModel(),
HasAvailableProviders() call orderedProviders with advance=false.
- ChatAndImageConcurrent_CountersIndependent — 90 chat + 90 image
goroutines concurrently; assert even per-modality distribution under
-race.
Closes#1018
* refactor(codex-pool): remove redundant primary_first strategy
* test(tools): update tool schema fixtures to pointer form
* feat(create_image): route Codex pools through chain with member failover
Codex pool chain entries now iterate pool members per the pool's own
strategy (round_robin or priority_order) and fail over internally before
the outer chain advances to the next entry.
- ChatGPTOAuthRouter.GenerateImage implements NativeImageProvider:
iterates orderedProviders, tries each member, advances round-robin state
only on success, aggregates errors on pool exhaustion.
- media_provider_chain.wrapPoolProvider wraps a *CodexProvider in a router
when RoutingDefaults has extras or a non-primary-first strategy. Solo
Codex (no extras) stays unwrapped.
- Router exposes ProviderType() so chain telemetry records "chatgpt_oauth".
Closes#1008
* feat(ui/builtin-tools): pool badge on create_image chain entries
Chain entry card shows a read-only "Pool · <strategy>" badge when the
entry's provider is a Codex pool base. Strategy label translates in
en/vi/zh. No new toggle or selector — pool config lives on the provider
itself.
#1008
* refactor(providers): move Codex test helpers to providertest subpackage
Addresses code-review High finding: NewTestCodexProviderFast and its
staticTestTokenSource were exported from a non-_test.go file, compiling
into the production binary. Relocating to internal/providers/providertest/
keeps the helper importable from other packages' tests without leaking
test-only symbols into production.
Also tightens wrapPoolProvider — pools with zero extra members no longer
get wrapped in a router (KISS: nothing to rotate between).
- Added CodexProvider.WithRetryConfig as a legitimate fluent option (the
test helper now uses the public API).
- Dropped the internal-package test helper file.
#1008
* docs(pr-1006): add pool badge UI evidence
Force-UI style capture of the read-only "Pool · Round-robin" badge on
the create_image chain entry card when the selected provider carries
settings.codex_pool with extras. Captured against staging gateway.
* refactor(ui/builtin-tools): unify pool UX with Create Agent pattern
The create_image chain Provider dropdown was listing pool members alongside
pool owners, letting users accidentally bypass pool semantics by picking a
member directly. Matches the pattern already used by Create Agent: hide
pool members, show an inline "Pool" chip on owners.
- Filter pool members from the chain Provider dropdown via
getChatGPTOAuthPoolOwnership.ownerByMember.
- Inline "Pool" chip on owner options, reusing the existing
providers:list.poolBadge i18n key.
- Drop the separate card-level "Pool · <strategy>" badge — the dropdown chip
alone conveys the information without duplication.
- Remove the now-unused isPoolProvider / poolStrategyOf helpers and the
builtin.mediaChain.poolBadge* i18n keys we briefly introduced.
#1008
* docs(pr-1006): add pool-filtered dropdown screenshot
* fix(ui/builtin-tools): migrate stale pool-member chain entries on load
Red-team blocker: a chain saved before the pool-aware UI landed may
reference a pool member by name (e.g. openai-codex-2). After the
dropdown started hiding members, such an entry produced:
- empty Select trigger (no SelectItem matches the stored value)
- conflicting Row 1 label still showing the member name
- silent runtime misroute (bare solo call, no pool wrap)
parseInitialEntries now consults getChatGPTOAuthPoolOwnership and
rewrites any chain entry whose provider is a pool member to the owner's
name + id. The next save persists the migrated value. Safe no-op for
non-pool entries and for entries already pointing at an owner.
Also memoize enabledProviders in the parent form so the card's useMemo
boundaries actually hold (minor perf ding flagged by same review).
* docs(pr-1006): refresh HTML evidence to match filter-based UX
* fix(ui/agent-codex-pool): traffic policy reflects effective strategy under inherit
On the agent's OpenAI Account Pool page, when Agent routing mode is
Use Provider Defaults, the Traffic Policy buttons painted the draft's
placeholder strategy ("priority_order") as selected — contradicting the
top-of-page chip which already correctly shows the provider's effective
strategy. The removal of primary_first in this PR unmasked the latent
bug: the placeholder used to be a deprecated value that didn't match any
live button, so nothing appeared selected.
Derive selectedStrategy from defaultRouting when mode === "inherit":
the button highlight now mirrors what actually runs. Buttons remain
disabled in inherit mode (unchanged), but the displayed selection no
longer misleads the user.
* docs(pr-1006): add screenshot of inherit-mode Traffic Policy fix
* fix(permissions): remove duplicate MethodSessionsCompact entry
The writeExact slice listed MethodSessionsCompact twice. slices.Contains
still returned correct results so runtime behavior is unchanged, but the
duplicate entry was dead code.
Spotted during review of PR #1006.
---------
Co-authored-by: viettranx <edu@200lab.io>
handleMessage ran an extra IsAllowed(senderID) check after
checkDMPolicy/checkGroupPolicy, creating a double-gate. When DM/Group
policy was set to "pairing" or "allowlist" with an allowlist configured,
paired users were wrongly rejected because they were not individually
listed. Telegram has no such post-policy gate — CheckDMPolicy and
CheckGroupPolicy already handle allowlist + pairing correctly.
Removing the redundant check aligns Discord with the shared policy
semantics and fixes issue #985.
- config: align struct tag formatting in TelemetryConfig for readability
- audio tests: modernize test patterns
- cache, http, hooks, providers: small style and cleanup passes
- No functional changes
- Use maps.Copy in hooks dispatcher instead of manual map loop
- Remove implicit loop variable capture in router_abort_test (Go 1.26 semantics)
- Use range without index in http_test where index unused
- Use min() builtin in script_test instead of manual min computation
- edit tool now calls allowedWriteWithTeamWorkspace to prevent cross-chat writes
- shell tool upgraded to use write-allowed prefixes for cwd resolution
- Prevents shell commands from accessing/mutating peer chat workspaces via absolute paths
- Add clarifying comment on shell's write-safety model
- Add WithToolTeamRoot context key to allow agents in same team to read peer-chat files
- Split allowedWithTeamWorkspace into read (includes team root) and write (excludes it)
- Write operations now use allowedWriteWithTeamWorkspace to prevent cross-chat mutations
- Add comprehensive tests for team root merging, deduping, and cross-chat access control
- Refactor path merging into buildAllowedPrefixes helper to reduce duplication
Extend migration 000056 backfill-2 vocabulary from {telegram, ws, delegate}
to the full channel set the gateway supports: discord, zalo, feishu, lark,
whatsapp, slack, line, messenger, wechat, viber (plus api transport).
Verified against staging data: identical chat_id extraction (91 rows,
9 distinct chats) — no regression. Now safe for tenants using any
non-Telegram integration.
- Extend migration 000056 with legacy-docs backfill (telegram/<chat>,
ws/<chat>, delegate/<chat>, and agent/bot/<chat> path shapes) so
scope=shared/personal docs created before team-scope refactor also
get stamped with chat_id and stop leaking across chats
- rescan: inferOwnerFromPath now returns chat_id extracted from team
path third segment; stamp onto VaultDocument so admin /vault/rescan
preserves chat isolation
- HTTP vault search: accept optional chat_id body param, set
TeamIsolated=true when TeamID + ChatID both present (admin UI can
view vault from a specific chat perspective)
Applied to staging DB via MCP: 91 legacy rows backfilled.
- Add vault_documents.chat_id + composite index (migration 000056)
- Filter vault_search by chat_id when team.workspace_scope=isolated
- Stamp chat_id on AfterWrite/AfterWriteMedia for isolated teams
- Deny cross-chat vault_read in isolated teams (M2 fix)
- RunContext.TeamIsolated flag resolved once per run
- Fallback WorkspaceChatID → ChatID in loop_context for entry points
that don't set WorkspaceChatID explicitly (WS direct, HTTP, cron)
Fixes cross-chat doc leak where agent in chat A could see vault docs
from chat B within the same isolated team.
Duy Nguyen