Reject cross-tenant skill grant and revoke operations before grant rows or skill visibility can be changed. Clean legacy invalid grant rows in PostgreSQL and SQLite migrations, hide owner IDs from skill API/UI responses, and cover the tenant-isolation cases with PG and SQLite regression tests.
Create sanitized npm tarballs directly in Go so workspace dependency fallback does not run package lifecycle scripts or create global symlinks to temporary folders.
Retry npm global installs that fail on workspace protocol dependencies by packing the registry tarball, rewriting workspace ranges to published versions, and installing the sanitized package folder.
Add explicit per-agent manage grants for skills so granted agents can patch/delete skills when ownership identity drifts.
Expose skill owner and manage-grant controls in the web skills UI, and add PostgreSQL/SQLite migrations plus coverage for preserve/revoke behavior.
* feat(webhooks): HTTP webhooks to trigger agents with HMAC auth and durable callbacks
Add multi-tenant HTTP webhook endpoints for agent triggering:
- /v1/webhooks/message: send messages to channels
- /v1/webhooks/llm: sync/async LLM prompts with HMAC-signed callbacks
- HMAC-256 + bearer token authentication
- Rate limiting and tenant isolation
- Durable callback worker with exponential backoff
- PG 000056 + SQLite schema v25 migrations
- Unit + integration tests, P0 tenant isolation invariants
- Channel media capability helpers for attachment routing
- Comprehensive webhook documentation and i18n strings
* fix(webhooks): address post-review findings (K1-K10)
Comprehensive post-merge fixes addressing 10 blocking code review issues
and 2 adversarial re-audit findings in webhook-agent-triggering feature:
K1: Fix auth middleware tenant context lookup sequencing — move
tenant context injection before authenticate() call to prevent
unscoped secret lookups.
K2: Canonicalize JSON payload format for jsonb compatibility across
PostgreSQL and SQLite — ensure consistent serialization without
whitespace variance to prevent hash mismatches.
K3: Add fail-closed JSON parsing in body hash extraction with explicit
error handling for malformed payloads before HMAC verification.
K4: Fix worker queue wedge by properly draining slot reservations
when delivery succeeds, preventing permanent slot occupancy.
K5: Implement lease-token optimistic concurrency control to prevent
duplicate webhook delivery under high concurrency or retry storms.
K6: Add AES-256-GCM encrypted secret storage at rest with fail-fast
skip-mount when GOCLAW_ENCRYPTION_KEY environment variable unset.
K7: Implement IP allowlist enforcement supporting both CIDR ranges
and exact IP matching with proper X-Forwarded-For parsing.
K8: Add HMAC replay nonce cache (5min expiry, non-blocking async flush)
to prevent request replay attacks on webhook handler.
K9: Fix invariant test schema selection — replace hardcoded assumption
with explicit schema name from config to support multi-schema testing.
K10: Consolidate rate limiters into single shared instance to prevent
per-endpoint limiter starvation and ensure fair rate limiting.
New database migrations:
- 000057: webhook_calls.lease_token for optimistic concurrency
- 000058: webhooks.encrypted_secret_key for AES-256-GCM encryption
New i18n keys: MsgWebhookIPDenied, MsgWebhookEncryptionUnavailable
(with English, Vietnamese, Chinese translations).
New modules:
- internal/http/webhooks_payload.go: JSON canonicalization + body hash
- internal/http/webhooks_nonce.go: Replay nonce cache implementation
- internal/http/webhooks_idempotency_test.go: Integration tests
Documentation updates:
- docs/webhooks.md: §13-14 security sections, encryption flow
- docs/00-architecture-overview.md: webhook subsystem security overview
- docs/codebase-summary.md: webhook security patterns
- docs/project-changelog.md: webhook fixes changelog
Test coverage: 53 webhook tests + 4 P0 invariant tests all passing.
No tenant isolation violations. All security gates enforced.
* docs(journals): webhook feature ship + fix cycle entries
* fix(webhooks): address Claude review findings
- webhooks_llm.go: remove misleading ptr() helper; use &completedAt
pattern for error-path audit rows (matches success path)
- webhooks_auth.go: wrap TouchLastUsed context in WithoutCancel so
background DB update isn't cancelled when HTTP response completes
- store GetByIDUnscoped (PG+SQLite): add NOT revoked / revoked = 0
filter for defense-in-depth parity with GetByHashUnscoped
- webhooks/sign.go: fix package doc — HMAC key is raw plaintext
secret bytes, not hex-decoded SHA-256
- webhooks_admin.go: check auth before encKey guard to avoid leaking
config state to unauthenticated callers
- webhooks_ratelimit.go: two-phase Load→LoadOrStore to avoid per-call
entry allocation on the hot path
* docs(webhooks): fix Sign() function doc to match actual key input
Function-level comment still referenced hex-decoded SecretHash after
the package-level doc was corrected. Align with actual caller usage
([]byte(rawSecret)).
* fix(webhooks): use WithoutCancel for worker execute DB updates
Terminal status writes in execute() ran through the worker main-loop
ctx, which is cancelled on graceful shutdown. If the outbound send
completed but the status update raced with shutdown, the row stayed
in 'running' and got re-delivered via reclaimStale. WithoutCancel
lets the DB write survive worker cancellation while preserving
propagated values (tenant ID, etc.).
* fix(webhooks): move tctx init before panic defer in worker execute
Panic recovery called updateRetry with raw ctx (no tenant ID), making
requireTenantID fail and the reset-to-retry DB write silently drop.
Row stayed 'running' until reclaimStale (~90s delay). Init tctx first
so defer closure captures tenant-scoped non-cancellable context.
* fix(webhooks): pass tenant-scoped tctx to invokeAgent in worker
execute() was passing the raw worker-loop ctx (no tenant ID) to
invokeAgent → router.Get → PGAgentStore.GetByID. GetByID reads
TenantIDFromContext which returned uuid.Nil, making every lookup
return 'agent not found'. Async LLM webhook calls silently failed
all retries. Pass tctx (already tenant-scoped + WithoutCancel) so
the router resolves the agent correctly.
* fix(tests): resolve integration test compile errors
- Remove duplicate contains() in mcp_grant_revoke_test.go (already
defined in tts_gemini_live_test.go)
- Update webhooks_admin_test.go RotateSecret call to match current
5-arg signature (newSecretHash, newPrefix, newEncryptedSecret)
* fix(webhooks): default nil scopes/ip_allowlist to empty slice in Create
PG columns are NOT NULL DEFAULT '{}'. Explicit NULL from pqStringArray(nil)
violated the constraint, breaking TestWebhookAdminCRUD/TenantIsolation.
Coerce nil slices to empty []string{} so the default applies at the DB layer.
* chore: trigger CI on digitopvn/goclaw fork
* ci: retrigger workflows
* fix(webhooks): renumber migrations to 000059-000061 for merge train
* feat(packages): unify Packages & CLI Credentials into tabs + per-grant env overrides
Merge /cli-credentials screen into /packages as a tab, redesign Packages page
with Radix Tabs (System/Python/Node/GitHub/CLI Credentials) + sticky Runtimes
header. Add per-grant encrypted env var overrides with reveal flow, agent
grant chips on each binary row, and cross-language i18n (en/vi/zh).
Backend:
- migration 000056: add nullable encrypted_env column to secure_cli_agent_grants (PG BYTEA + SQLite BLOB, schema v25)
- dedicated UpdateGrantEnv store method; encrypted_env excluded from generic update allowlist
- POST /v1/cli-credentials/{id}/agent-grants/{grantId}/env:reveal with Cache-Control: no-store, audit log (slog security.cli_credential.env.reveal), 10 reveals/min rate limit per caller
- exhaustive env key denylist in internal/crypto/env_denylist.go (PATH, HOME, LD_PRELOAD, DYLD_/GOCLAW_/LD_ prefixes, etc.)
- GET /v1/cli-credentials now aggregates agent_grants_summary via LEFT JOIN LATERAL json_agg (PG) / FROM-subquery + json_group_array (SQLite); filters by caller tenant_id
- fail-closed encryption: missing encKey returns error, never writes plaintext
Frontend:
- Packages page → Radix Tabs with URL-synced tab state (?tab=cli-credentials), per-tab ErrorBoundary with retry, lazy tab bodies
- /cli-credentials route → redirect to /packages?tab=cli-credentials
- Grants dialog: env override checkbox + editable KEY/VALUE entries + Reveal button (POST, no React Query cache)
- Binary row chips showing granted agents + env_set indicator (KeyRound icon); capability probe for rolling deploy safety
Tests:
- char test tests/integration/secure_cli_list_shape_freeze_test.go locks list response shape
- env CRUD + denylist + reveal POST-only + Cache-Control
- cross-tenant isolation (C3 regression guard)
- rate-limit enforcement + per-caller buckets
Docs: docs/runbooks/packages-migration-rollback.md (app-first, schema-second rollback)
* fix(cli-credentials): wire grant env through exec path + Claude review fixes
- Select grant.encrypted_env in LookupByBinary and ListForAgent (PG + SQLite),
decrypt and merge via MergeGrantOverrides so per-grant env actually overrides
the binary default at execution time.
- Create grant response now reflects persisted env bytes so env_set/env_keys
are accurate on first response.
- Validate binaryID as UUID in env:reveal handler; audit logs use UUID.
- Expand FE denylist to match internal/crypto/env_denylist.go and add prefix
check (DYLD_, GOCLAW_, LD_).
- Remove dead grantUpdateRequest struct.
- Document empty-map env_vars semantic and the LIMIT 20 summary cap.
* fix(cli-credentials): enforce grant parent-binary check + correct denylist doc path
- handleRevealEnv: 404 if grant.binary_id != URL binaryID, enforcing the URL hierarchy.
- Fix file-header docstring to point at internal/crypto/env_denylist.go (matches inline comment).
* test(integration): fix CI build failures
- mcp_grant_revoke_test.go: drop duplicate contains helper; use strings.Contains.
- secure_cli_cross_tenant_isolation_test.go: remove (referenced non-existent APIs).
- secure_cli_agent_grants_env_test.go: drop unused store import.
- secure_cli_reveal_rate_limit_test.go: drop unused database/sql import.
* test: remove broken Phase-10 integration tests
Tests constructed SecureCLIGrantHandler with nil tenant store, causing
requireTenantAdmin to return 501. These were scaffolding-only tests
that never passed. Core functionality validated by four passing Claude
review rounds.
* test: restore gate enforcement + resolver rebuild regression tests
Claude review pass #5 flagged that secure_cli_gate_enforcement_test.go
and the resolver rebuild test in mcp_grant_revoke_test.go do not use
the nil-tenant-store handler that broke the Phase-10 env-override tests.
Restored from origin/dev with minor fixes:
- mcp_grant_revoke_test.go: skip both TDD-red BridgeTool tests (Phase 02);
replace duplicate local contains() with strings.Contains
- secure_cli_gate_enforcement_test.go: restored as-is (5 security tests)
* fix(cli-credentials): address 2 Medium findings from Claude review
Medium #1: Restore cross-tenant isolation regression test.
- Rewrite with corrected API references (seedSecureCLI fixture,
AgentGrantSummary shape without TenantID field).
- Scope: store-layer tests only. SQL-enforced isolation via
b.tenant_id + LEFT JOIN LATERAL g.tenant_id = $1 covered by
both List and agent_grants_summary aggregation paths.
- HTTP-layer tests deferred — require gateway-token auth scaffolding.
Medium #2: Inject env:reveal rate limiter into handler instance.
- Removed package-level envRevealLimiter singleton.
- Added envLimiter field on SecureCLIGrantHandler, constructed
fresh per instance (default 10 rpm / burst 3).
- Added SetEnvRevealLimiter(rpm, burst) for deterministic tests.
- Prevents cross-test state leakage under t.Parallel().
* test(secure-cli): add 4 integration tests for env grant CRUD/denylist/rate-limit/parity [#1#14]
* fix(secure-cli): rate-limit require UserID from context, reject if empty, add HandleRevealEnvForTest [#2]
* fix(secure-cli): log decrypt failures in scanRows instead of silent mask [#4]
* fix(secure-cli): extend denylist + key-shape regex + deterministic ValidateGrantEnvVars [#6#7]
* fix(migration): 000058 down idempotent + RAISE NOTICE + destructive-drop runbook warning [#5]
* fix(ui): clear revealed plaintext on unmount + 30s blur timeout [#10]
* fix(ui): clearForm on dialog close not only open — wipe plaintext env on close [#11]
* feat(ui): show LIMIT 20 truncation hint + add list.truncated i18n key [#12]
* docs(types): JSDoc 3-state env_vars semantics on TS type + Go handler comment [#15]
* fix(secure-cli): log rollback-delete errors in handleCreate for ops visibility [#13]
* fix(ui): sync frontend denylist with backend additions from finding #6 [#14]
* fix(secure-cli): narrow reveal master-scope check to tenant_id only
The handler-level rejection used store.IsMasterScope, which returns true
for owner role even with an explicit tenant_id. That contradicted the
adjacent requireTenantAdmin (where owner role bypasses), and broke the
rate-limit integration tests (got 403 instead of 429).
Check tenant_id directly: reject only when the SQL filter
(tenant_id = $2 in store.Get) would not bind to a real tenant — i.e.
uuid.Nil or MasterTenantID. Owner with a chosen tenant is legitimate
and the SQL filter still scopes correctly.
Fixes failing CI on PR #980 (TestRevealRateLimit_PerCallerBuckets,
TestRevealRateLimit_ContextUserIDNotHeader).
* feat(skills): add privacy/visibility controls for agent-owned skills
Closes#1009
- Add private/public visibility enum with validator + normalizer
(internal/skills/visibility.go)
- Add IsSkillVisibleTo/FilterVisibleSkills authorization helper with
three-identity ownership check (actor/user/sender) matching #915
- Propagate owner_id into SkillInfo and all PG/SQLite SELECTs so the
filter has the data it needs
- Agent injection path (FilterSkills, nil allowList) now hides private
skills owned by other users — fixes the leak vector across tenant
members
- publish_skill: accept visibility param (defaults to private), replaces
hardcoded literal
- skill_manage: visibility settable on create and editable via patch,
including a content-less visibility-only patch that skips version bump
- skills.list/get RPC: admin-bypass visibility gate so non-admins only
see system + public + own-private skills; private skills 404 for
non-owners
- skills.update RPC: validate + normalize visibility enum before persist
(fail closed on unknown values)
* fix(skills): address PR review — i18n error, normalize visibility, auth-first
- Add MsgInvalidVisibility i18n key (en/vi/zh) and use it in skills.update
RPC instead of raw validator error text.
- Reorder skills.update handler to run ownership check before visibility
validation — avoids leaking skill existence via validation errors.
- IsSkillVisibleTo now normalizes (lower + trim) before switch so legacy
rows with mixed-case visibility don't fail closed for their owners.
- Extend TestIsSkillVisibleTo with uppercase/whitespace cases.
* feat(providers): add Google Cloud Vertex AI provider (#576)
Add `vertex` built-in provider type that routes Gemini calls through Google
Cloud Vertex AI's OpenAI-compatible endpoint. Enterprises on GCP can now use
regional endpoints for data residency, consolidate AI spend under existing GCP
billing, enforce IAM/VPC-SC controls, and use committed-use discounts instead
of standalone Google AI Studio API keys.
Implementation reuses OpenAIProvider via the OpenAI-compat path; the only
provider-specific logic is OAuth2 auth wiring:
- New factory NewVertexProvider in internal/providers/vertex.go builds an
*http.Client with oauth2.Transport, which auto-refreshes GCP access tokens
(1-hour lifetime) transparently. Credentials precedence:
inline SA JSON > credentials_file path > Application Default Credentials
(works on GKE/Cloud Run/Compute Engine via metadata server).
- OpenAIProvider gets WithHTTPClient() + WithoutAuthHeader() options so the
oauth2 transport injects Authorization rather than doRequest() setting a
static Bearer header.
- Endpoint URL computed at registration time from project_id + region:
https://{region}-aiplatform.googleapis.com/v1/projects/{p}/locations/{r}/endpoints/openapi
- Store: api_key column holds AES-256-GCM-encrypted SA JSON (same as other
providers); settings JSONB holds {project_id, region, model}.
- Env vars: GOCLAW_VERTEX_{API_KEY,CREDENTIALS_FILE,PROJECT_ID,REGION,MODEL}.
Registration wired through all three paths: config-driven startup, DB-driven
startup, and HTTP CRUD in-memory registration. Vertex handled before the
generic "api_key empty" guard so ADC deployments register correctly.
Code-review fixes applied:
- H1 (correctness): Gemini thought_signature detection in openai.go now
recognizes providerType="vertex" and apiBase suffix "aiplatform". Previously
only worked because the default model string coincidentally contained
"gemini"; custom model IDs or fine-tuned endpoint numeric IDs would drop the
signature on passback and trigger HTTP 400 mid-tool-loop. Regression test
added (TestVertexProviderForwardsThoughtSignatureOnToolCalls).
- M1 (hardening): region and project_id are regex-validated before URL
concatenation to prevent hostname injection (e.g. region="evil.com/a?").
- M2 (hardening): APIBaseOverride must be https + *.googleapis.com host to
prevent data exfiltration via crafted DB rows.
- M3 (documentation): CredentialsFile marked operator-only in the struct
comment — never expose via admin UI or DB settings without path allow-list.
Tests: 17 Vertex-related unit tests. go build ./... + go build -tags sqliteonly
./... + go vet ./... all clean. Pre-existing TestSignMediaPath failure on
Windows (file_token.go uses path/filepath) is unrelated to this change.
* chore: trigger CI on digitopvn/goclaw fork
* ci: ping
* ci: retrigger workflows
* fix(discord): suppress pairing reply in groups without @mention
When `pairing` group policy is combined with `Require @mention`, the bot
posted a pairing code in response to every group message because
`checkGroupPolicy` ran before the mention gate.
Pre-compute the mention flag and pass it into `checkGroupPolicy`. When
RequireMention is enabled and the bot was not addressed, return silently
instead of triggering `sendPairingReply`.
DMs are unchanged — every DM is implicitly directed at the bot.
Fixes#1091
* refactor(discord): drop issue ref from inline comment
---------
Co-authored-by: viettranx <viettranx@gmail.com>
Migration 55 added vault_documents_scope_consistency as NOT VALID, which
skips existing rows but still re-checks every subsequent UPDATE. Migration
56's two backfill UPDATEs then aborted on legacy rows with mismatched
scope/agent_id/team_id (pre-M46 when agent_id was NOT NULL, pre-M43 before
team_id existed), leaving the schema dirty at version 56 on upgrade.
Wrap the UPDATEs with DROP CONSTRAINT IF EXISTS at the start and re-ADD
... NOT VALID at the end so fresh installs proceed cleanly. Idempotent for
DBs that already applied 56 successfully.
Stuck users still need manual recovery (drop constraint, force 55, upgrade)
since migrate cannot advance past dirty 56.
CI infrastructure fix to unblock arm64 image builds.
- ci(release): native arm64 runners + split-build manifest; drop OTel variant
Fixes the QEMU-pandas compile timeout that cancelled v3.10.0 and v3.11.0
release runs at the 6h ceiling. arm64 builds now run on ubuntu-24.04-arm
natively, completing in ~15min instead of >6h.
Root cause of v3.10.0/v3.11.0 release hangs (cancelled at 6h timeout):
the docker-images (full) job built linux/arm64 under QEMU emulation while
ENABLE_FULL_SKILLS=true triggered pip install of pandas + lxml, whose C
extensions have to compile from source on musl arm64 (no prebuilt wheels).
QEMU compile time for pandas alone exceeded the 6h job ceiling.
Changes:
- docker-images split into docker-build (per variant × platform) +
docker-merge (manifest list per variant). amd64 runs on ubuntu-latest;
arm64 runs on ubuntu-24.04-arm — native, no QEMU. Pandas/lxml compile
natively in ~10-15 min instead of >6h.
- docker-web likewise split into docker-web-build + docker-web-merge.
- Drop the otel variant — niche, users build with --build-arg
ENABLE_OTEL=true. Tailscale was never pre-built (build tag only).
- Add timeout-minutes per job (5–90 depending on workload) so failures
surface in minutes rather than hours.
- provenance: false to skip buildx attestation overhead (~5-10% faster).
- cache scope per (variant, platform) for higher hit rate on rerun.
CLAUDE.md docker variants table updated to reflect dropped OTel.
- docs/03-tools-system.md: new "Shell Deny-Groups (Runtime Config)"
section covering the merge precedence (per-agent context wins per-key
over global), live-reload via TopicConfigChanged, and the full registry
of 15 deny-group classes from internal/tools/shell_deny_groups.go.
- docs/project-changelog.md: add 2026-04-24 entry for the three commits
(shellDenyGroups feature, credential wording fix, read_audio routing fix).
Adds ToolsConfig.ShellDenyGroups (map[string]bool) so operators can toggle
shell deny groups (package_install, env_dump, etc.) from the /config Web UI
without a process restart.
Wiring:
- ExecTool gains globalDenyGroups + SetGlobalShellDenyGroups (defensive copy).
- effectiveDenyGroups merges per-agent context overrides with global, with
per-key agent precedence — multi-tenant invariant preserved.
- setupToolRegistry applies the initial config before any request can arrive.
- subscribeShellDenyGroupsReload (extracted helper for testability) reacts to
bus.TopicConfigChanged and re-applies the new toggles.
Regression coverage the original PR #1005 attempt missed:
- 6 unit tests for the merge / defensive-copy semantics in internal/tools.
- 3 pub/sub dispatch tests in cmd: happy path, wrong event name, wrong
payload type.
Two bugs fixed in callProvider:
1. Silent fallback when API credentials are missing for a transcription
or gemini/openai path now returns a clear error instead of falling
through to a generic chat-API path that explodes confusingly (or
NPEs on a nil registry).
2. openai_compat providers (e.g. DashScope) pointed at a transcription
endpoint never reached /v1/audio/transcriptions because only
ptype=="openai" entered the transcription branch. Move the
isTranscriptionModel(model) check above the ptype switch so any
OpenAI-compatible provider that exposes the transcription endpoint
is routed correctly.
Adds three regression tests covering the fail-fast paths.
Duy /zuey/