When rescan finds no new/updated files but some docs still lack summaries
(e.g. previous enrichment failed due to provider timeout), automatically
re-enqueue them for enrichment retry.
- Add VaultStore.ListUnenrichedDocs() to fetch docs with empty summary
- Add EnrichWorker.EnqueueUnenriched() to emit events for retry
- Add RescanResult.Reenqueued field to track re-enqueued count
- Update UI to show "X re-queued for enrichment" toast
SQLite implementations were missing tenant_id WHERE clauses that the
PostgreSQL equivalents have. While Lite edition is single-tenant in
practice, this maintains dual-DB parity and prevents potential issues
if the codebase evolves.
Follows the same pattern as other methods in the file (GetSkillFilePath,
CreateSkillManaged, etc.) that use store.TenantIDFromContext with
MasterTenantID fallback.
Only count BlockReplies when tool calls are present (matching when
EmitBlockReply actually fires in think_stage). Final answers without
tool calls must not increment the counter, otherwise gateway dedup
falsely suppresses delivery on non-streaming channels.
- Update ObserveStage to check len(resp.ToolCalls) > 0
- Rename test to reflect new behavior (tool calls required)
- Add regression test for #838 scenario
Closes#838
- Fix TestPrivateReply_ReturnsError: use HTTP 400 instead of 200 since
doRequest only returns error for status >= 400
- Remove unused MaxThreadDepth config field from CommentReplyOptions
- Log loaded system_configs (background.provider, agent.default_provider)
- Log provider lookup attempts with source and error details
- Warn when falling back to first registered provider
- Surface config load errors instead of silently swallowing them
Helps diagnose provider resolution failures in episodic/vault workers.
Backend:
- Add error tracking to EnrichProgress (error_count, last_error)
- Broadcast error events when LLM calls fail
- Add POST /v1/vault/enrichment/stop endpoint
- Wire enrichWorker to VaultHandler for stop functionality
Web UI:
- Add stop button (appears when enriching)
- Show error toast when enrichment errors occur
- Display error count in progress bar
- Add useStopEnrichment hook
i18n: en/vi/zh translations for new strings
The previous commit only listened to TopicSystemConfigChanged (periodic
DB refresh), but config.patch from UI fires TopicConfigChanged. Add
subscriber in gateway_lifecycle so vault enrichment picks up provider/
model changes immediately when user saves config.
- Increase classify max_tokens 1024→2048, summarize 1536→4096 to
prevent truncated JSON from models like gemini-2.5-pro
- Add debug logging: raw LLM output on parse failures, finish_reason
truncation warnings with model name
- Hot-swap vault enrichment provider/model on config change without
restart (wired into TopicSystemConfigChanged handler)
- Use RWMutex-guarded llm() accessor for thread-safe provider reads
Frontend sent null for empty emoji (emoji.trim() || null), which violated
the NOT NULL constraint on the promoted emoji column (migration 000037).
This caused all agent saves to fail with 500 when emoji was unset.
- Frontend: send empty string instead of null for emoji field
- Backend (PG + SQLite): add null-coercion for all promoted NOT NULL
columns — TEXT (emoji, agent_description, thinking_level) coerce to "",
INT (skill_nudge_interval, max_tokens) coerce to 0
9 fixes for vault enrichment pipeline:
1. Queue key = tenant-only (was per-agent, caused multiple batches
blocking EventBus workers and progress bar flashing)
2. Classify chunks 5 candidates per LLM call (prevents response
truncation that caused parse_still_failed errors)
3. Classify prompt improved: explicit "EXACTLY one entry per
candidate", 5-entry example, ctx capped at 30 words
4. max_tokens kept at 1024 (sufficient for 5 candidates)
5. Progress AddDone removes !running guard (safe before Start)
6. Rescan defers event publishing via PendingEvents — Start()
called before workers receive events, eliminating race
7. Upload handler same deferred publish pattern
8. Frontend enrichment timer cancels stale "complete" timeout
when new enrichment starts (prevents bar disappearing)
9. Sidebar tree reloads after rescan completes
Classify now searches across entire tenant (empty agentID) to
build cross-agent links for future vault sharing. Access control
enforced at query time — agents only see their own docs.
With 10 candidates, classify response can exceed 1024 tokens when
ctx descriptions are verbose, causing json unmarshal failures.
Bump to 2048 to accommodate worst-case 10-candidate responses.
After rescan agent_key fix, docs split into per-agent batch queues.
Each batch was calling Start()/Finish() independently, causing the
progress bar to flash 0/1 per agent instead of showing global progress.
Now: handler Start(total) sets global count, worker batches call
TrackBatch()/MarkBatchDone() for lifecycle, AddDone(n) auto-completes
when done >= total. Progress bar shows smooth 0→N across all agents.
Start() is called each drain loop in processBatch to update total.
Only reset done=0 when transitioning from idle→running (new batch),
not when already running (same batch, total growing).
- Add AgentID to VaultTreeOptions, pass agent_id filter through
backend tree endpoint and frontend hook
- Fix rescan inferOwnerFromPath: match root-level {agent_key}/...
folders against agentMap (workspace uses agent_key directly,
not agents/ prefix). Keep legacy agents/ prefix for compat
- Preserve full relPath for DB storage in all patterns
- Show filename with extension in tree (use path basename, not title)
- Start tree folders collapsed (prevent stuck loading state)
- Remove orphaned useVaultGraphData call from vault-page
- Tenant isolation verified: agentMap built per-tenant via scopeClause
- Add 10 missing tools to the goclaw group in policy.go: skill_manage,
publish_skill, use_skill, delegate, memory_expand, knowledge_graph_search,
vault_search, create_audio, datetime, heartbeat. Fixes skill creation
permission denied when agents use group:goclaw allow list.
- Relax agent update ownership check: tenant admins can now update any
agent in their tenant (adminMiddleware + tenant-scoped GetByID already
ensures proper authorization). Previously only agent owner or system
owner could update.
- Improve agent update error logging: include user_id and tenant_id in
structured log, return actual error message instead of generic
"internal error" for better debugging.
Replace flat paginated document list with collapsible tree view.
Backend: two-query-per-level approach (files + DISTINCT deeper paths
for virtual folder derivation), path parsing in Go for PG+SQLite
compatibility, text_pattern_ops index for prefix range scans.
Frontend: VaultTree component with doc_type colored icons, compact
single-line nodes, truncate-middle filenames, scope dots, hover
tooltip for date. Sidebar expanded to 384px on large screens.
Also fixes pre-existing SQLite scanVaultDocRow missing path_basename.
API keys for Exa, Tavily, and Brave can now be set directly in the
web search provider chain settings dialog. Keys are saved to
config_secrets (AES-256-GCM encrypted, tenant-scoped) and stripped
from the persisted settings blob. Raw key values are never returned
in API responses — only boolean set/unset status.
Backend:
- Add ConfigSecretsStore to BuiltinToolsHandler
- extractAndSaveSecrets: parse api_key from settings, save to
config_secrets, strip before persisting to builtin_tool_tenant_configs
- getSecretsStatus: returns boolean map per tool (never raw values)
- Enriched handleList response with secrets_set per tool
Frontend:
- Per-provider API key input in web search chain cards
- "Key set" status badge when key exists, "Change" button to replace
- Staged keys sent with settings on save, backend extracts
hasEditableSettings() only checked if tool.settings was non-empty,
hiding the Settings button for web_search, tts, etc. when no settings
existed yet. Now tools with dedicated form components always show the
button so admins can create initial settings.
Add dedicated UI forms for web_search and TTS builtin tool settings,
replacing the raw JSON editor for these tools.
- web-search-chain-form: drag-drop provider order (Exa/Tavily/Brave),
DuckDuckGo locked as always-on fallback, color-coded rails,
per-provider max_results config
- tts-provider-form: primary provider selector + auto mode dropdown
with inline descriptions (off/inbound/tagged/always)
- Wire both into builtin-tool-settings-dialog dispatch
- i18n strings for en/vi/zh
TTS tool now reads per-tenant primary provider override from
BuiltinToolSettingsFromCtx before falling back to the manager's
default. Same per-call resolution pattern as web_search/web_fetch.
Simpler path A approach (no pool) — tenant can select which
registered provider their agents use. Per-tenant API keys deferred
to a future pool implementation if needed.
- Add resolvePrimary(ctx, mgr) with ttsOverride struct
- Refactor Execute to use resolved primary with fallback chain
- 5 new unit tests for tenant provider override scenarios
web_fetch now reads per-tenant policy overrides from
BuiltinToolSettingsFromCtx before falling back to the tool's
default policy. Same per-call resolution pattern as web_search.
- Add resolvePolicy(ctx) with webFetchPolicyOverride struct
- Refactor Execute + doFetch + fetchRawContent to use webFetchPolicy
- InProcessExtractor also resolves policy from ctx
- Remove isDomainAllowed/isDomainBlocked (replaced by matchDomainList)
- 6 new unit tests for tenant policy override scenarios
Role checks are not tenant checks — a non-master tenant admin holds
RoleAdmin in their own tenant and passes role-only middleware by
design. CLAUDE.md gains a one-line directive pointing at CONTRIBUTING
for the full decision table + anti-patterns. CONTRIBUTING gains:
- Target-table decision table (global vs tenant-scoped) with the
matching guard for each (requireMasterScope vs requireTenantAdmin).
- Shared predicate reference: store.IsMasterScope(ctx).
- Anti-pattern list for reviewers: writes to no-tenant_id tables
without master-scope check, SQL tenant_id IS NULL arms on write
paths, role-only admin gates, revoke/delete handlers that skip
pre-fetch ownership verification.
Phase 5 of tenant tool config refactor. Wires the existing builtin tools
page + settings dialog to the tenant-config HTTP endpoint shipped in
Phase 4, so tenant admins can edit per-tenant tool overrides from the
same "Settings" button master admins already use.
- BuiltinToolData DTO gains tenant_settings (already returned by the
GET /v1/tools/builtin enrich path when request is tenant-scoped).
- useBuiltinTools hook gains setTenantSettings + clearTenantSettings —
both hit PUT /v1/tools/builtin/{name}/tenant-config, with null clearing
the settings column while preserving tenant_enabled (column-list
upsert on the store side, shipped in Phase 2).
- BuiltinToolSettingsDialog accepts a tenantScope flag. In tenant mode:
- initial values come from tenant_settings ?? settings (pre-fill
with global defaults when creating a fresh override)
- a small badge + hint line communicates the mode
- optional "Reset to global default" appears when an override exists
- BuiltinToolsPage branches the save callback: master scope → updateTool
(global settings), non-master → setTenantSettings. The backend enforces
the same rule defensively per Phase 0b, so this branch is a UX guard
(avoid 403) not a security boundary.
- New en/vi/zh keys: tenantOverrideBadge, tenantOverrideHint,
tenantOverrideNewHint, resetToGlobalDefault.
Verified: pnpm build clean (tsc -b + vite).
Phase 0b of tenant tool config refactor. Closes 3 privilege-escalation
vulnerabilities in the same bug class as commit b419f352 (Phase 1
config.* hotfix):
- CRITICAL: PUT /v1/tools/builtin/{name} — non-master tenant admin
could overwrite global builtin_tools.settings, corrupting tool
defaults for every tenant.
- CRITICAL: POST /v1/packages/install|uninstall — non-master tenant
admin could run pip/npm/apk server-wide. Supply-chain vector.
- HIGH: POST /v1/api-keys/{id}/revoke (HTTP + WS) — tenant admin
could revoke NULL-tenant system keys because store SQL matches
(tenant_id = \$N OR tenant_id IS NULL).
Implementation:
- Export store.IsMasterScope as the single predicate; rewire Phase 1
config.* middleware to delegate (no behaviour change).
- Add http.requireMasterScope helper symmetric to requireTenantAdmin.
- Guard handleUpdate (builtin_tools) and handleInstall/handleUninstall
(packages) with master-scope check before any mutation or shell exec.
- Fix api_keys.Revoke at the handler layer: fetch key via new
APIKeyStore.Get, verify key.TenantID matches caller tenant for
non-owner callers. Applies to both HTTP and WS paths.
- Harden WS router to inject role into ctx so store.IsOwnerRole works
from WS handlers (closes a latent drift between the HTTP and WS
layers that broke the initial WS api_keys.revoke fix).
- Drop unused APIKeyStore.Delete (YAGNI + removes dormant vuln with
the same tenant_id IS NULL arm).
- Emit security.tenant_scope_violation and security.api_key_revoke_
forbidden slog events on every rejection for future SIEM alerting.
- New MsgMasterScopeRequired i18n key + en/vi/zh catalogs.
Tests cover the guard predicate, all 3 HTTP endpoints, and the full
WS api_keys.revoke matrix (cross-tenant deny, system-key deny for
tenant admins, system owner bypass, own-tenant happy path). 14 other
admin-gated write endpoints verified safe by static audit.
Refactor web_search Execute to resolve its provider chain per-request
via BuiltinToolSettingsFromCtx instead of iterating the singleton's
hardcoded provider list. Tenant admins can now reorder or disable
providers via builtin_tool_tenant_configs.settings using the shape
{"provider_order":[...], "brave":{"enabled":false}, ...}. Unknown
provider names in provider_order are logged + skipped — no injection
path via this settings blob. Malformed JSON falls back to defaults
(fail-open so a bad admin paste never crashes the tool). Secrets stay
in config_secrets — tenant cannot inject API keys through this path.
11 unit tests cover reorder, filter, disable, unknown-skip, malformed,
global-layer, and tenant-vs-global precedence.
Note: This is the MVP tenant adoption half of the plan's Phase 7. The
Exa + Tavily feature port from PR 825 and coordination with its author
is deferred to a separate follow-up (the port is net-new feature work
orthogonal to the overlay adoption).
Phase 6 of the tenant tool config refactor adopts zero production code
changes for media tools — media_provider_chain.go:77 already reads via
BuiltinToolSettingsFromCtx, and the Phase 3 merge automatically feeds
tenant overrides into that call. This adds 5 verification tests proving
the end-to-end flow: tenant layer beats global, per-agent arg still wins
over tenant, fallback to global when no tenant row, tenant-only layer
works, and cross-tool isolation (create_image override does not leak
into create_audio). Covers the 4-tier overlay on a real tool without
touching any media tool's Execute signature.
Plumb per-tenant tool settings into the agent Loop without touching any
tool's Execute signature. Adds WithTenantToolSettings ctx helper and
rewrites BuiltinToolSettingsFromCtx with fast-path merge semantics —
tenant layer wins over global defaults at tool-name level (no field-level
deep merge). Resolver preloads ListAllSettings for the agent's tenant at
Loop construction; store errors log + fall back to global. Zero allocs
on single-tier reads. Tier 1 (future per-agent override) is reserved and
documented in context_keys.go. 8 unit tests cover empty / single-tier /
both-merged / RunContext fallback / fast-path semantics.
Wake up the dead builtin_tool_tenant_configs.settings column (exists in
migrations 000027/SQLite 1180 since v3 tenant foundation, never read/written).
Add GetSettings/SetSettings/ListAllSettings interface methods with a
json.RawMessage Settings field on BuiltinToolTenantConfig. Both DBs use
explicit column-list DO UPDATE SET so Set(enabled) and SetSettings(raw)
never clobber each other. Add ErrInvalidTenant sentinel so nil-tenant
callers fail fast (no silent master fallback). ListAll now filters
enabled IS NOT NULL — rows created via SetSettings stay in their own lane.
9 SQLite unit tests + 4 new PG integration tests (round-trip,
column coexist, cross-tenant isolation, nil-tenant guard).
Non-master tenant admin calling config.patch/apply/get/schema would
corrupt master *config.Config + config.json on disk via m.cfg.ReplaceFrom
and m.Save, leaking master config across tenants. Add requireMasterScope
middleware that rejects non-master tenant ctx (system-owner role still
bypasses for consistency with requireTenantAdmin). Chain before
requireOwner on all 4 config.* methods. Adds MsgConfigMasterScopeOnly
i18n key (en/vi/zh) and 10 unit tests covering helper + middleware +
chained fail-closed behavior.
Integration tests were running locally only — CI never exercised
the 29 v3_*_test.go files under tests/integration/. This left the
session tenant-isolation + vault scan-arity bugs undetected until
a manual run this session.
Changes:
- tests/integration/v3_test_helper.go: testDB() now calls
pg.InitSqlx(db) inside sharedDBOnce. Previously the test suite
relied on whichever test ran first happening to call InitSqlx,
so running with `-run <filter>` could segfault with a nil
pkgSqlxDB. Removes the ordering-dependency land mine.
- .github/workflows/ci.yaml: add services.pg block running
pgvector/pgvector:pg18 with pg_isready healthcheck, set
TEST_DATABASE_URL at job level, and add a new "Integration
tests" step after unit tests. Uses -timeout=180s (vs 90s unit)
because the first test runs migrations from scratch.
Local integration suite: 2.9s. CI cold start expected ~30-60s
with image pull + migrations.
Two unrelated integration-test failures, both real source bugs:
1. Session ListPaged/ListPagedRich leaked cross-tenant data.
buildSessionFilter only honored opts.TenantID and never read
tenant from ctx — so callers relying on ctx scoping (the common
case) got no tenant filter at all. List() already had the
correct pattern using store.TenantIDFromContext(ctx) gated by
!store.IsCrossTenant(ctx). Replicate inside buildSessionFilter;
thread ctx into the helper; opts.TenantID still wins as admin
override. Fixes TestStoreSession_ListPaged_Pagination/tenant_isolation
and TestStoreSession_ListPagedRich_TenantIsolation.
2. Vault GetDocument / GetDocumentByBasename panicked with
"sql: expected 15 destination arguments in Scan, not 14".
Migration 000047 added `path_basename` as a generated column;
the SELECT lists were updated but the Scan arg lists were not.
GetDocumentByID had the correct 15-target pattern — copy it.
vaultDocRow.PathBasename field already exists. Audit sweep of
vault_links.go + vault_documents_enrichment.go confirms no
other drift. Fixes TestStoreVault_UpsertAndGetDocument.
viettranx