mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-11 03:13:24 +00:00
dev
35
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 isaacgao4396andisaacgao4396
|
169e0bafaf |
fix(tools): use_skill inlines content when read_file isn't granted (#1547)
* feat(store): add context helper for per-iteration tool allowlist * feat(pipeline): surface resolved tool allowlist to tool dispatch via context * fix(tools): use_skill inlines content when read_file isn't granted Fixes #1477 --------- Co-authored-by: isaacgao4396 <gaoyuan4396@gmail.com> |
||
|
bilogic
|
6a316fc23d | rescan filesystem for new skills when setup (#1493) | ||
|
Clark Cant
|
4b865508da |
Merge pull request #1028 from krichprollsch/lightpanda
Add Lightpanda as alternative to Chrome for the browser automation tool |
||
|
ntduc
|
876104950b | fix: recover missing embeddings safely | ||
|
Duc Nguyenandntduc
|
27f4b7743b |
fix(skills): preserve custom skills during bundled seeding (#1435)
Co-authored-by: ntduc <ntduc@cpp.ai.vn> |
||
|
|
db69ccf882 |
tracing: nest tool calls under LLM spans, emit hook spans, fix usage_events FK (#1415)
- Reparent tool_call spans under their producing llm_call span via RunState.CurrentLLMSpanID so a model turn's trace visibly contains the tool calls it triggered. - Wire internal/hooks EmitHookSpan into the dispatcher writeExec so every hook execution (pre/post tool use, etc.) produces a trace span with input, console output, decision, error, and duration for agent troubleshooting. - Fix usage_events_span_id_fkey violations: usage events were inserted synchronously referencing a span flushed ~5s later, silently dropping token/cost data. Route them through the collector so they flush after spans in the same cycle. No schema migration; FK preserved. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
nguyenha935
|
f826738ee6 |
feat: add team work classification routing (#1379)
Approved by github-maintain bot. Clean team work classification feature with comprehensive tests and i18n. |
||
|
|
c1b89e96e1 |
chore: merge main into dev (#1370)
* fix: unify NO_REPLY detection (#1233) Co-authored-by: GoClaw Operator <operator@goclaw> * feat(acp): surface session/update tool_call notifications at Info level (#1141) session/update notifications carrying a ToolCall (or inline tool_call / tool_call_update kind) were only dumped at Debug level inside the params blob. Operators could see `security.tool_granted` (permission granted) but had no way to tell whether the tool actually executed successfully — both "permission granted then failed silently" and "permission granted and succeeded" looked identical in journalctl. Add a structured Info log emitting toolCallId, name/title, status, and a content preview (truncated at 400 chars) whenever the notification contains tool-call state. This is what made it possible to diagnose the recent .goclaw/-path-deny regression — `status=failed` immediately after `security.tool_granted` revealed the gap that the granted-only log hid. No behavior change beyond logging volume; preview is bounded. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(exec): exempt venv python interpreter from .goclaw/ path deny (#1140) The ExecTool path-deny rule blocks any token containing `.goclaw/` unless it matches one of the AllowPathExemptions prefixes (skills-store, tenants). This silently rejected legitimate commands invoking the goclaw-managed Python interpreter via its absolute path: /home/user/.goclaw/venv/bin/python3 .../script.py The first token `/home/user/.goclaw/venv/bin/python3` matched the deny pattern but no exemption, so the entire command was denied. Naive exemption (".goclaw/venv/bin/") does not work: matchesAnyPathExemption resolves both tokens and exemption candidates via EvalSymlinks, and the venv's python3 is a symlink into the host's python cellar (e.g. linuxbrew). The token canonicalizes to /home/linuxbrew/.../python3.14 while a literal ".goclaw/venv/bin/" prefix never gets touched. Fix: resolve venv/bin/python3 once at startup and exempt the dirname of the resolved target. Failure to resolve (no venv present) silently falls through. Without this, ACP-driven agents either fail outright or work only via fragile heuristics (cwd-local symlinks generated on the fly by the LLM). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cron): invert stateless gate to match UI label (#1139) The cron job handler reset the session when stateless=false and skipped reset when stateless=true — the opposite of what every UI locale labels the field (en/ko/zh/vi all describe stateless as "each run starts fresh without loading previous messages"). The buggy gate caused stateless=true crons to silently accumulate session history across every execution, leading to context bloat and increasing the chance of LLMs short-circuiting tool calls in favor of replaying prior assistant turns. One affected daily ETL cron grew to 38 messages over 18 days before the regression was noticed. Fix: gate the Reset on `if job.Stateless` so the runtime matches the UI contract. No DB migration is required — existing values were set by users based on the UI label, so they already encode the intended behavior. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(providers): retry transient Codex response failures (#1332) * fix(providers): retry transient Codex response failures * test(cron): tolerate nil session store in handler tests * fix(background): clear stale provider alerts (#1362) Co-authored-by: Collective Developer <man@collective.dev> --------- Co-authored-by: Duy /zuey/ <duy@wearetopgroup.com> Co-authored-by: nguyenha935 <nguyenthanhha935@gmail.com> Co-authored-by: GoClaw Operator <operator@goclaw> Co-authored-by: codebit0 <34156842+codebit0@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Zezae Oh <zezaeoh@gmail.com> Co-authored-by: Collective Developer <man@collective.dev> |
||
|
|
90e7035100 |
fix(mcp): tenant isolation, tool policy engine, and prompt preview fixes (#1333)
* fix(prompt): remove duplicate Team Members section from system prompt The TEAM.md context file already provides the Members section with better formatting. The system prompt section was redundant and inconsistently formatted. - Remove buildTeamMembersSection() call from system prompt building - Remove unused buildTeamMembersSection() function * fix(mcp): wire store to manager for prompt preview tool visibility MCP manager needs database access to query configured servers for tool visibility in prompt preview. - Add SetStore() method to MCP manager - Wire pgStores.MCP to manager after initialization - Add debug logging for MCP initialization flow * fix(systemprompt): hide Tooling section when agent has no tools The Tooling section header and boilerplate were displayed even when the agent had no tools available. Add early return in buildToolingSection() to skip the section entirely when toolNames is empty. This reduces prompt noise for agents with no tool access. * feat(mcp): cache tool descriptions for prompt preview visibility MCP tools now show descriptions in prompt preview without requiring a live server connection. - Add CacheToolDescriptions() method to MCPServerStore (PG + SQLite) - Cache tool descriptions in settings['tool_cache'] when server connects - Use cached descriptions in ListToolsForAgent (fallback: hints → cache → global) - Descriptions are auto-populated from live server manifest on connection - Admins can still override via tool_hints in server settings * test(mcp): add CacheToolDescriptions to MCPServerStore test fakes Commit 5ce410b6 added CacheToolDescriptions() to the MCPServerStore interface but missed updating test mock implementations, breaking go vet across internal/mcp, internal/agent, internal/http, and internal/channels/bitrix24. Add no-op implementations matching each fake's existing style. * fix(providers): enforce per-agent tool policy for Claude CLI provider The Claude CLI provider (stdio+MCP bridge) was not enforcing per-agent tool policy, unlike other providers where the policy-filtered tool list already drives the system prompt's Tooling section. Two gaps closed: 1. --disallowedTools was previously skipped entirely when no MCP config path was resolved, letting the CLI subprocess run with its full native toolset (Bash, Edit, Read, Write, Glob, Grep, WebFetch, WebSearch) regardless of agent policy. It's now unconditional and derived from the agent's actual allowed-tools list (state.Tool.AllowedTools), mapped to Claude CLI's native tool names. 2. The MCP bridge server executed any tool call without checking the calling agent's policy. It now resolves the agent's policy from context (via the existing HMAC-verified agent lookup) and denies calls to tools outside that agent's allowed set, logging security.mcp_bridge_denied on denial. Both gaps were closed using existing plumbing (PolicyEngine.WouldAllow, AgentData.ParseToolsConfig, the bridge context middleware) — no new cross-cutting mechanism was introduced. * feat(web): show MCP/tool schemas in system prompt preview dialog The prompt-preview API response includes a separate `tools` field (the actual JSON schemas sent to the LLM as the tools API parameter) alongside `prompt` (the system prompt text), but the web UI only rendered `prompt`, silently dropping the tools list. Add a collapsible Tools section to both the full-screen System Prompt dialog and the inline agent-detail preview, showing tool count, name, description, and expandable parameter schema per tool. i18n keys added to en/vi/zh locales. * fix(prompt): render pinned skills on bootstrap turns Pinned skills are documented (web UI copy) as "always inlined in the system prompt", but the entire Skills section was gated behind !cfg.IsBootstrap, so pinned skill XML never appeared on bootstrap turns (first message of a session) despite the promise. Separate pinned-skill rendering from bootstrap-suppressed guidance: - Bootstrap + pinned skills present: render pinned XML only, no search/manage guidance (which stays suppressed as before) - Non-bootstrap: unchanged behavior - Minimal/none modes: pinned skills always render regardless of bootstrap state Add regression tests covering all four prompt modes on bootstrap turns, plus a non-bootstrap guard confirming existing behavior is preserved. * fix(skills): resolve managed skills directory per-tenant, not master-only skills.Loader was wired at startup to scan a single fixed directory (the master tenant's managed-skills dir), making any skill belonging to a non-master tenant invisible to both pinned-skills prompt resolution and skill_search/use_skill, regardless of DB visibility settings. - Loader now resolves the calling tenant's managed-skills directory per-call via context (store.TenantIDFromContext), never enumerating other tenants' directories - Skill cache is now tenant-keyed to prevent slug collisions and cross-tenant cache leaks across tenants using the same skill slug - gateway_setup.go passes the root data dir instead of a pre-resolved master-tenant path Write-side tooling (skill_manage, publish_skill) was already correctly tenant-scoped per-operation — no changes needed there. Added TestLoader_ManagedSkills_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's content, including after cache population from a different tenant's lookup. Known follow-up (not in this commit): skill_search's BM25 index is still a single process-global index shared across tenants, which is a related but separate cross-tenant search-result leak requiring its own scoped fix (per-tenant index maps + threading tenant context through ensureIndex/rebuildIndex). * fix(tools): scope skill_search BM25 index per-tenant SkillSearchTool held a single process-global BM25 index built once from whichever tenant's context first triggered ensureIndex, then reused for all subsequent Execute() calls regardless of caller — leaking one tenant's skill search results into another's, the search-path counterpart to the managed-directory bug fixed in 7b4668ad. - index/lastVersion are now keyed per-tenant (map[uuid.UUID]*tenantIndexState) - ensureIndex resolves the calling tenant from context and only builds/reads that tenant's index entry, never touching another tenant's cached state - Builtin/bundled skills remain visible in every tenant's index (Loader already merges those tiers correctly per 7b4668ad) Loader.Version() remains a single global counter — a version bump in one tenant causes unnecessary rebuilds in others but does not cause cross-tenant leakage, an acceptable tradeoff to avoid scope creep. Added TestSkillSearchTool_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's search results, including after cache population from a different tenant. * fix(tools): fix group-spec expansion in tool policy engine PolicyEngine.registry was only ever set via SetRegistry(), which was never called in production (only in one test) — so pe.registry was permanently nil in production. Every group-expansion helper (applyProfile, intersectWithSpec, unionWithSpec, subtractSpec, expandSpec, matchDenySpec, filterByCapability) silently dropped any "group:*" spec entry instead of expanding it when registry was nil. Concretely: "group:mcp" (auto-injected into agentToolPolicy.AlsoAllow for any agent with MCP tools) never resolved to real tool names, so MCP tools connected successfully and appeared in prompt text (which reads the registry directly, bypassing PolicyEngine) but were never included in the actual ChatRequest.Tools payload sent to the LLM — confirmed live via mcp.agent.tools_loaded tools=6 immediately followed by mcp.filtered_tools mcp_defs_count=0 in the same request. This affects any agent relying on group-based grants, not just MCP. PolicyEngine is a shared/global singleton used concurrently across all agents (constructed once at gateway startup), so mutating a registry field per-call would be a data race. Fix instead threads the registry as an explicit parameter from FilterTools down through all internal group-expansion helpers, and adds IsDenied/WouldAllow registry parameters, removing the dead SetRegistry() mechanism entirely. Also fixes group expansion for the per-user-MCP-tools path: FilterTools is sometimes called with a userToolOverlay wrapping a *Registry rather than a *Registry directly; added Unwrap() to userToolOverlay so the new registry-resolution logic works for both cases. Added 4 tests proving group expansion works via the threaded parameter alone (no SetRegistry): plain registry allow, userToolOverlay allow, deny-side group expansion, and the WouldAllow bridge-server path. Blast radius note: this restores intended access for every agent configured with group:* specs (group:mcp, group:vault, group:goclaw, group:coding, etc.) that were silently inert before. Existing agent configs relying on group grants will gain the tool access they were nominally already configured for. * fix(mcp): cache tool descriptions from pool-connected servers too 5ce410b6 added tool-description caching (for prompt-preview visibility) only inside connectServer. connectViaPool — the separate connect path used when MCP connections go through the shared pool — never got the same caching hook, even though it shares the same underlying connectAndDiscover wire handshake. Confirmed live: cloudflare/docker connected via connectViaPool and received real descriptions over the wire, but prompt preview still showed blank descriptions because this path never wrote to the cache. Also removes the temporary mcp.connect.raw_tool debug log added earlier this session for diagnosing the same issue — no longer needed now that the root cause is fixed. * chore(skills): remove temporary pinned-skills diagnostic logging Confirmed live: pinned skills (caveman, infra-ansible-knowledge) now resolve correctly end-to-end for tenant-scoped agents. Debug logging added to trace the resolution chain is no longer needed. * fix(tools): deny always wins over AlsoAllow group grants AlsoAllow's unionWithSpec could reintroduce a tool explicitly listed in Deny, since it added tools back from allTools without re-checking deny specs. Previously masked because AlsoAllow's group-expansion was also broken (fixed in f7af95de this session) — group specs silently expanded to nothing, so this ordering bug never manifested. Now that group expansion works, an admin-denied tool that's also reachable via a group:* AlsoAllow entry (e.g. group:mcp) would silently reappear. Re-apply deny-spec subtraction as a final step after AlsoAllow union, for both global and per-agent policy, so deny always wins regardless of which allow mechanism tries to add a tool back. Added tests proving global and per-agent Deny correctly override an overlapping AlsoAllow group grant, while sibling non-denied tools in the same group remain allowed. * fix(mcp): enumerate cached tools instead of wildcard placeholder in prompt preview ListToolsForAgent (the prompt-preview path) collapsed any server with an empty ToolAllow (unrestricted grant — the common case) into a single "server__*" placeholder entry, even when tool_cache already had every real tool name and description from connect time (5ce410b6, 8ffd67b9). This meant agents with unrestricted MCP server access never saw individual tool names or descriptions in prompt preview, forcing trial-and-error tool usage. When ToolAllow is empty and tool_cache is populated, enumerate every cached tool (skipping any explicitly denied) and emit one MCPToolPreviewInfo per tool, matching the construction logic already used for the ToolAllow-non-empty case. Falls back to the single placeholder only when tool_cache is also empty (server never connected). The live (non-preview) conversation path, buildMCPToolDescs, does not have this bug — it resolves tool identity from the live connected registry, never from ToolAllow, so no placeholder shortcut exists there. Added tests covering both the cache-populated enumeration case and the no-cache placeholder fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(prompt): filter alias re-injection and add MCP schemas to preview ToolDefs Prompt-preview's tools: schema array (PreviewResult.ToolDefs) had two bugs, both preview-only — confirmed live conversations already use correctly-filtered tool payloads via buildToolsPayload/PolicyEngine.FilterTools, unaffected by either bug: 1. Alias re-injection iterated ALL registry aliases globally with no check against the deny-filtered toolNames list, letting denied tools reappear via their alias name (e.g. a denied canonical tool still showing up as its Claude-Code-compat alias like Bash/Edit/Write/Read). Now skips any alias whose canonical tool isn't in the filtered set. 2. MCP tool descriptions (from the store-based, connection-free ListToolsForAgent path) only ever fed the prompt TEXT section, never got converted into ToolDefinition schema objects — so the tools: array never showed MCP tools at all, even when the MCP section of the prompt text correctly listed them. Now appends a ToolDefinition per MCP tool with name+description populated and a placeholder {"type":"object"} Parameters schema, documented as preview-only (real parameter schemas require a live MCP connection, only available during an actual conversation turn). Added tests proving denied-tool-alias exclusion and MCP tool inclusion in preview ToolDefs. * feat(skills): inline full content for pinned skills instead of pointer-only Web UI documented "Pinned skills are always inlined in the system prompt", but BuildPinnedSummary was just BuildSummary with an allowlist filter — same as the general searchable skill list: name/description(truncated)/location pointer only, requiring use_skill+read_file round trips to get actual content. No code path inlined real SKILL.md content for pinned skills specifically. BuildPinnedSummary now reads and inlines full SKILL.md content (frontmatter stripped) per pinned skill inside <skill_instructions> tags. Per-skill (10000 bytes) and total (30000 bytes) size caps fall back to the original pointer-only format with a note when a skill is too large to inline, so oversized skills degrade gracefully instead of blowing the prompt budget. Added tests: full-content inlining, size-cap fallback, and tenant isolation for the new inline path (mirroring the existing managed- skills tenant isolation test). * fix(mcp): real parameter schemas and full policy enforcement in prompt preview Three interconnected fixes to prompt preview, none affecting the live conversation path (which was already correct): 1. Real MCP parameter schemas instead of a useless empty placeholder. tool_cache previously stored only name+description; extended to also capture the real JSON Schema from the MCP server's tools/list response (CachedToolInfo{Description, Parameters}) at connect time, for both direct-connect and pool-connect paths. Preview now shows complete, real input schemas instead of {"type":"object"} with no properties — verified against actual wire data from a live cloudflare MCP server showing genuinely rich schemas (zone_id, type, name, content, ttl, proxied, all typed with descriptions and correct required arrays) that were previously being discarded. Backward-compatible: old-shape cache entries degrade gracefully to description-only rather than crashing, self-healing on next connect. 2. Global tool deny now enforced in preview. BuildPreviewPrompt previously hand-rolled a partial policy reimplementation (per-agent deny only, explicitly skipping the full PolicyEngine "because runtime state isn't available in preview") — but PolicyEngine.WouldAllow already handles this per-tool-name without needing channel context. A tool denied via the global config (not per-agent) would appear in preview despite being correctly denied in every real conversation. Preview now calls WouldAllow per candidate tool, with a graceful per-agent-deny-only fallback when no PolicyEngine is wired (e.g. in tests). 3. MCP tools now also subject to the same policy check — previously the MCP tool supplement (store-based, connection-free tool listing) added MCP tool names unconditionally, bypassing WouldAllow entirely, so a denied MCP tool could still appear in preview. Added tests for all three: real-schema presence, global-deny exclusion for both core and MCP tools, and backward-compat cache handling. * fix(prompt): remove redundant per-tool MCP enumeration from prompt text Now that MCP tool schemas in the tools: API parameter are real and complete (1290d4f1), the ## MCP Tools prompt-text section's per-tool "- mcp_x__y: description" enumeration is pure duplication with zero added value — the model already gets each tool's real schema (including description) via tools:. buildMCPToolsInlineSection now keeps only the behavioral instructions that aren't expressible via JSON schema and thus aren't duplicated: prefer-MCP-over-core-tools guidance, and the optional-parameter guidance (don't guess/fill optional fields). The per-tool name+ description enumeration loop is removed. Section still only appears when the agent has MCP tools (len(cfg.MCPToolDescs) > 0, unchanged gate). Updated tests to assert the enumeration is gone while the behavioral instructions remain; ToolDefs assertions are now the authoritative check for MCP tool allow/deny filtering behavior (prompt text no longer enumerates names at all). * test(agent): update TeamContextInjection test for removed Team Members section TestBuildSystemPrompt_TeamContextInjection asserted the presence of a 'Team Members' prompt-text section that was intentionally removed in 71d33180 (duplicate of the canonical TEAM.md-context-file Members section, which has better formatting). The test was never updated to match, causing it to fail on every run since. Moved the assertion from wantIn to wantNotIn for the 3 affected subtests -- BuildSystemPrompt correctly no longer renders team-member roster info directly; that info now comes exclusively from the TEAM.md context-file mechanism, outside this test's isolated scope. * fix(prompt): resolve real registry for WouldAllow calls in preview BuildPreviewPrompt's two WouldAllow calls hardcoded reg=nil, silently breaking group:* expansion (e.g. group:mcp) needed to resolve the AlsoAllow grant production actually uses to grant MCP tool access (resolver_helpers.go's agentToolPolicyWithMCP injects AlsoAllow: ["group:mcp"]). With reg=nil, WouldAllow could match literal tool names fine (the 18 core/static tools) but could never resolve group-based grants, so every MCP tool silently failed WouldAllow and was excluded from preview -- confirmed live via curl: 18 tools returned, zero mcp_* ones, for an agent with genuinely working MCP access in real conversations. Live conversations were never affected -- internal/mcp/bridge_server.go's WouldAllow call already correctly passes a real registry. Fix resolves a real *tools.Registry from deps.ToolLister via tools.ResolveConcreteRegistry (the same helper used at the live call site), passing it to both WouldAllow calls instead of nil. Falls back to nil gracefully for test mocks that don't implement the full ToolExecutor interface, preserving existing test behavior. Added a test proving an MCP tool granted via the exact production AlsoAllow: ["group:mcp"] pattern is now correctly included in preview ToolDefs, where the old reg=nil bug would have silently excluded it. * fix(prompt): use literal deny check for MCP tools in preview, not group expansion The MCP-tools policy gate added in 1290d4f1 called WouldAllow with a real registry (per 9b5fd2eb), which requires group:mcp expansion against that registry to grant access via the production AlsoAllow: ["group:mcp"] pattern. But MCP tools are only ever registered into ephemeral per-agent registry clones at live connection time (manager_connect.go) -- never into the shared/global registry preview uses. group:mcp always resolved empty in preview's connection-free context, so WouldAllow denied every MCP tool -- confirmed live via tool-name diff: live conversations correctly included all 6 MCP tools, preview included zero. MCP access-granting is already correctly handled by ListToolsForAgent's own per-server tool_allow/tool_deny grant logic (confirmed working correctly earlier this session). The preview gate only needs to catch the narrower case of a literally-denied tool name via global/per-agent policy config -- it never needed group expansion. Replaced WouldAllow with IsDenied(nil, name, agentPolicy), which forces a pure literal-name match with zero registry dependency, matching the existing usage pattern already established elsewhere in policy.go. This class of bug cannot recur: there's no registry-passing code path left in this check to silently reintroduce group-expansion dependence. Added a test proving MCP tool inclusion in preview is independent of group-expansion outcome (no AlsoAllow: group:mcp needed for a non-denied tool to appear). Also reverts the temporary loop.filtered_tool_names/ preview_prompt.filtered_tool_names diagnostic logging used to capture the live-vs-preview tool-name comparison that diagnosed this bug. * fix(http): preserve real MCP parameter schemas through HTTP preview adapter mcpPreviewAdapter.ListToolsForAgent (the HTTP-layer glue converting mcp.MCPToolPreviewInfo to agent.MCPToolPreviewInfo for BuildPreviewPrompt) only copied RegisteredName and Description, silently dropping Parameters -- a bug present since this adapter was introduced (2499d0be/7e250244), unrelated to today's other MCP preview fixes. This was masked until d5fc6344 fixed MCP tools being excluded from preview entirely (a separate bug) -- once MCP tools started appearing again, this pre-existing adapter gap became visible: tools showed up correctly, but always with the bare {"type":"object"} placeholder instead of their real cached schema (confirmed live: update_dns_record missing its 7 real properties). One-line fix: copy Parameters through in the adapter's struct literal. Added a regression test constructing a real *mcp.Manager with populated tool_cache, asserting the adapter's output preserves specific real schema properties (not just non-nil Parameters) -- verified this test fails without the fix and passes with it. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Bruno ClermontandBruno Clermont
|
2499d0be78 |
fix(mcp): show configured MCP tools in prompt preview (#1321)
* fix(mcp): show configured MCP tools in prompt preview MCP tools were only visible in prompts if currently loaded in registry (from active sessions). Prompt preview showed empty MCP tools section. Solution: Query MCP store for configured servers and tool lists. Show tools that would be available, not just currently loaded. - internal/mcp/manager.go: add ListToolsForAgent() for store-based tool discovery - internal/agent/preview_prompt.go: supplement live registry with store tools - internal/http/agents.go: add mcpPreviewMgr field and setter - internal/http/agents_prompt_preview.go: create adapter bridging mcp.Manager to preview - cmd/gateway.go: wire up MCPPreviewAdapter during setup * fix(mcp): load MCP servers from database at startup, not config file Single source of truth: MCP servers are now loaded from mcp_servers table at gateway startup, instead of from the config file. This ensures that MCP servers configured via web UI are automatically available without requiring config file updates. - cmd/gateway_setup.go: remove config fallback, add initMCPFromDB() helper - cmd/gateway.go: call initMCPFromDB() after store initialization - internal/mcp/manager.go: add SetConfigs() for runtime configuration Now mcpMgr will be non-nil when MCP servers are in the database, enabling MCP tools in prompt preview. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Pierre Tachoire
|
74c1d3a7be |
feat(gateway): wire browser Backend option through config
Pass cfg.Tools.Browser.Backend to browser.WithBackend when set, so the "chrome" or "lightpanda" selector from config / GOCLAW_BROWSER_BACKEND reaches the manager. Empty value still triggers /json/version auto-detection in browser.Start(). |
||
|
Goon
|
4002bb4c13 | feat(usage): add event analytics dashboard | ||
|
Duy /zuey/
|
61fdcdcb81 |
feat(tools): add command keyword allowlist
Closes #42 |
||
|
viettranx
|
df5d48c78d |
feat(tools/shell): add runtime-reloadable global shell deny-groups via config.tools
Adds ToolsConfig.ShellDenyGroups (map[string]bool) so operators can toggle shell deny groups (package_install, env_dump, etc.) from the /config Web UI without a process restart. Wiring: - ExecTool gains globalDenyGroups + SetGlobalShellDenyGroups (defensive copy). - effectiveDenyGroups merges per-agent context overrides with global, with per-key agent precedence — multi-tenant invariant preserved. - setupToolRegistry applies the initial config before any request can arrive. - subscribeShellDenyGroupsReload (extracted helper for testability) reacts to bus.TopicConfigChanged and re-applies the new toggles. Regression coverage the original PR #1005 attempt missed: - 6 unit tests for the merge / defensive-copy semantics in internal/tools. - 3 pub/sub dispatch tests in cmd: happy path, wrong event name, wrong payload type. |
||
|
viettranx
|
3a85d625a6 |
feat(tools): add send_file for delivering existing workspace files
- new send_file(path, caption?) tool with DenyPaths guard and duplicate-delivery check - patch message(MEDIA:) to Mark DeliveredMedia on send success (closes cross-tool dup gap) - register in gateway wiring + builtin seed - add to systemprompt coreToolSummaries; clarify write_file deliver=true description - 16 tests green (PG + SQLite builds clean, invariants green) |
||
|
viettranx
|
ff1ab33fbc |
refactor(gateway): wire web_search with tenant-scoped stores
Move web_search tool registration from gateway_setup (where it received hardcoded config) into wireExtraTools with access to ConfigSecretsStore and MessageBus for runtime per-tenant resolution. Update gateway_setup.go to remove web_search-specific config loading. |
||
|
viettranx
|
3b4004946e |
chore: apply go fix syntax upgrades (min/max, switch patterns)
Apply Go 1.21+ min/max builtins and improved switch pattern matching across test files and utilities. No functional changes. |
||
|
viettranx
|
b7002a67d1 |
feat(audio): wire Scribe STT + legacy bridge in gateway setup
Register ElevenLabs Scribe and proxy STT providers in gateway initialization. Auto-wire legacy STT bridge from existing channel STTProxyURL configs. Implement setupAudioExtras() with RegisterSTT and SetSTTChain calls. Maintains backward compat with per-channel configs. |
||
|
viettranx
|
aeef3396e4 | refactor(tools): create_audio delegates to audio.Manager | ||
|
viettranx
|
b0b87da7cd |
feat(config): add optional Audio config for STT/Music
Add optional Audio *AudioConfig pointer field on Config with STT and Music sub-structs. Nil-safe — absent in JSON5 decodes as nil, no breaking change. cfg.Tts retained unchanged for backward compat. setupAudioExtras stub wired for Phase 3/4 STT/Music provider registration. |
||
|
viettranx
|
1ac08155b0 |
feat(trace): reliable stop/abort with ctx-aware streams and 2-phase router
Makes the Stop button on the traces page actually stop running traces. Seven-phase implementation across provider HTTP, agent router, trace persistence, WS events, tool exec, i18n, and integration tests. - Provider HTTP+SSE ctx-aware: close socket on cancel via CtxBody wrapper - Router 2-phase abort: CAS state machine, 3s grace, force-mark fallback - Trace retry: 3 inline retries + 10-max retry queue, stale recovery 10min - trace.status WS event: real-time UI updates (invalidates query on receive) - Tool exec: process-group kill (SIGTERM→3s→SIGKILL), Rod page ctx watch - i18n: 6 abort toast variants in en/vi/zh - Integration: 9 scenarios, -race clean Fixes tenant-ctx loss in forceMarkTraceAborted and retry worker broadcast (caught by code-reviewer: C1/C2). Stale threshold intentionally 10min because start_time-based; last_span_at migration is a follow-up. |
||
|
viettranx
|
30a40bbe59 |
feat(tools): add Exa + Tavily web search providers with ranked ordering
Port Exa and Tavily provider clients from PR #825 into the tenant-aware overlay architecture (builtin_tool_tenant_configs.settings). - Add ExaConfig + TavilyConfig to WebToolsConfig with provider_order - Add Exa HTTP client (api.exa.ai/search, x-api-key auth) - Add Tavily HTTP client (api.tavily.com/search, Bearer auth) - Extend Brave + DDG constructors with per-provider maxResults - Add config_secrets plumbing for exa/tavily API keys (apply/collect/mask/strip) - Refactor gateway_setup.go to use WebSearchConfigFromConfig - Add NormalizeWebSearchProviderOrder (DDG always last, dedup, unknown skip) - Extract web_search_config.go (builder, normalizer, shared helpers) - 11 new unit tests for provider order, builder, clamp, normalize Credit: @kaitranntt (PR #825) for the original Exa + Tavily implementation. |
||
|
|
8f56ddaa64 |
feat(v3): core architecture redesign — pipeline, memory, vault, evolution, providers, orchestration (#790)
* feat(v3): add core interface contracts and migration for v3 redesign
Foundation interfaces: TokenCounter, WorkspaceContext, DomainEventBus,
ProviderAdapter/Capabilities. Pipeline: Stage, RunState, MessageBuffer,
substates, Pipeline orchestrator. Memory: EpisodicStore, AutoInjector,
KG temporal extensions, consolidation workers. System integration:
PromptConfig, ToolCapability, Retriever. Orchestration: OrchestrationMode,
EvolutionMetrics/SuggestionStore. Migration 000037: episodic_summaries,
evolution tables, KG temporal columns. Schema version 36→37.
* refactor(plans): mark all v3 design phases complete with file references
* fix(v3): address code review findings on design contracts
- C1: add missing l0_abstract column to episodic_summaries migration
- C2: align EpisodicSummary ID/TenantID/AgentID to uuid.UUID
- H1: document tenant_id scoping requirement on EpisodicStore
- H2: add UNIQUE constraint on (agent_id, user_id, source_id) for dedup
- H4: clarify ProviderAdapter vs Provider relationship in doc
- M3: set state.ExitCode on BreakLoop/AbortRun in pipeline
- M6: store full PipelineConfig in Pipeline struct
- Edge: add WHERE embedding IS NOT NULL on HNSW index
* fix(v3): second-pass review fixes
- H1: use context.WithoutCancel for finalize + set ExitCode on ctx cancel
- H2: use utf8.RuneCountInString consistently in FallbackCounter
- H3: longest-prefix-match in ModelContextWindow (prevents wrong tokenizer)
- H4: return unsubscribe cleanup func from consolidation.Register
* feat(v3): implement DomainEventBus with worker pool, dedup, and retry
Worker pool processes events from buffered channel. SourceID-based dedup
prevents duplicate processing. Exponential backoff retry on handler error.
Panic recovery per handler. Graceful shutdown via Drain(). 8/8 tests pass
with race detector.
* feat(v3): implement ProviderAdapter for Anthropic, OpenAI, DashScope, Codex
Add CapabilitiesAware to all 6 providers. Create ProviderAdapter
implementations that delegate to existing buildRequestBody/parseResponse
for DRY. ClaudeCLI and ACP get capabilities only (subprocess transport).
DashScope wraps OpenAI adapter with StreamWithTools=false override.
* feat(v3): implement WorkspaceContext Resolver for 6 scenarios
Stateless resolver produces immutable WorkspaceContext at run start.
Handles personal/group/predefined/team-shared/team-isolated/delegation.
Wired into loop_context.go behind v3PipelineEnabled flag (additive,
v2 path unchanged). Includes delegation path boundary check,
master tenant bypass, and tenant slug path composition.
* feat(v3): implement tiktoken TokenCounter with BPE encoding + cache
Adds tiktoken-go for accurate cl100k_base/o200k_base token counting.
Per-message FNV-1a hash cache avoids re-encoding unchanged history.
Falls back to rune/3 heuristic for unknown models. NewTokenCounter
factory selects implementation at build time.
* feat(v3): promote 12 other_config JSONB fields to dedicated agent columns
Extract emoji, agent_description, thinking_level, max_tokens,
self_evolve, skill_evolve, skill_nudge_interval, reasoning_config,
workspace_sharing, chatgpt_oauth_routing, shell_deny_groups, and
kg_dedup_config from the catch-all other_config JSONB into proper
columns with DB-level types and defaults.
- Migration: PG (000037) + SQLite (schema v6→7) with backfill
- Go: AgentData struct + simplified Parse* methods
- Store: SELECT/INSERT/scan updated for both PG and SQLite
- Gateway: create/update handlers accept promoted fields
- HTTP: export/import with legacy backward compat
- Web UI: all 15 frontend files read/write from top level
* feat(v3): implement Knowledge Vault with unified search, wikilinks, and FS sync
Migration 000038 adds vault_documents (FTS+pgvector), vault_links, vault_versions
tables. VaultStore interface with PG implementation for document CRUD, hybrid
FTS+vector search, and bidirectional link management. All queries enforce
tenant_id isolation including JOIN-based scoping on link operations.
FS sync layer: SHA-256 content hashing, VaultInterceptor hooks into write_file/
read_file for auto-registration and lazy sync, fsnotify watcher with 500ms
debounce. Wikilink engine parses [[target]] syntax, resolves targets via
3-step strategy, and maintains vault_links on write.
VaultSearchService fans out queries across vault, episodic, and KG stores in
parallel with per-source score normalization and weighted merge. AutoInjector
and Retriever implementations for pipeline integration.
Three agent tools: vault_search (unified discovery), vault_link (explicit
linking), vault_backlinks (dependency tracing). Feature-flagged via
v3_vault_enabled agent setting.
* feat(v3): wire vault into gateway startup + add unit tests
Wire VaultStore embedding provider, VaultSearchService, VaultInterceptor
on read/write tools, and register vault_search/vault_link/vault_backlinks
tools in gateway_vault_wiring.go. All wiring gated by stores.Vault != nil.
Add 28 unit tests for ContentHash, ContentHashFile, and ExtractWikilinks
covering edge cases, unicode, display text, context windows, and offsets.
* feat(v3): implement stage-based pipeline loop with 8 pluggable stages
Decompose monolithic agent loop into internal/pipeline/ package:
- 6 stages: Context, Think, Prune+MemoryFlush, Tool, Observe+Checkpoint, Finalize
- Foundation types: Stage interface, RunState with 7 typed substates, MessageBuffer
- Pipeline orchestrator with setup/iteration/finalize 3-phase execution
- Callback-based PipelineDeps avoids circular import with agent package
- Feature-flagged via v3PipelineEnabled in Loop.Run()
- All 7 exit conditions preserved (no tools, max iter, truncation, loop kill,
read-only streak, tool budget, ctx cancel)
* feat(v3): wire pipeline callbacks to Loop methods + add 71 unit tests
Wire 15 of 17 PipelineDeps callbacks from Loop methods via closures:
- Context: LoadContextFiles, BuildMessages, EnrichMedia, InjectReminders
- Think: BuildFilteredTools, CallLLM (stream/sync)
- Prune: PruneMessages, CompactMessages
- Memory: RunMemoryFlush
- Finalize: SanitizeContent, FlushMessages, UpdateMetadata, BootstrapCleanup, MaybeSummarize
- Remaining: ExecuteToolCall, CheckReadOnly (deep loop.go integration)
Add comprehensive test suite (71 tests, all passing with -race):
- MessageBuffer: 10 tests (append, flush, replace, counts)
- Pipeline.Run: 14 tests (3-phase flow, exit conditions, ctx cancel)
- Stage tests: 47 tests (ThinkStage nudges/truncation, PruneStage budget,
ToolStage parallel/exit, ObserveStage content, CheckpointStage interval,
FinalizeStage cleanup)
* feat(v3): wire remaining 2 callbacks (ExecuteToolCall, CheckReadOnly)
Complete callback wiring — 17/17 PipelineDeps callbacks now active:
- ExecuteToolCall: resolves tool name, executes via registry, processes
result via existing processToolResult with loop detection bridge
- CheckReadOnly: delegates to checkReadOnlyStreak via bridge runState
- Bridge runState shares loop detection state between pipeline and agent
* fix(v3): eliminate data race in tool execution + capture injected messages
- Remove parallel tool execution path — serialize all tool calls to avoid
data races on shared bridgeRS (loop detector, media results, deliverables)
- Loop kill checked after each tool (mid-batch early exit)
- BuildFilteredTools: capture and append injected tool-awareness messages
- Rename test to reflect sequential execution
* feat(v3): wire ResolveWorkspace, safe parallel tools, ContextStage tests
- Wire ResolveWorkspace callback via workspace.NewResolver() with
ResolveParams from Loop fields (no longer a nil stub)
- Re-add safe parallel tool execution: split into ExecuteToolRaw
(parallel I/O) + ProcessToolResult (sequential state mutation)
with opaque rawData pass-through (no double execution)
- Add 12 unit tests for ContextStage (8) + MemoryFlushStage (3)
- Split tool callbacks to loop_pipeline_tool_callbacks.go (under 200 lines)
- Capture buildFilteredTools injected messages
* feat(v3): add episodic memory store + temporal KG columns
Phase 1 — Episodic Store:
- Migration 000039: episodic_summaries table with pgvector, FTS, L0 abstracts
- EpisodicStore PG impl: CRUD, hybrid FTS+vector search, ExistsBySourceID,
PruneExpired. Idempotent via source_id UNIQUE constraint.
Phase 2 — Temporal KG:
- Migration 000040: valid_from/valid_until on kg_entities + kg_relations,
partial indexes for current-facts queries, epoch→timestamptz backfill
- ListEntitiesTemporal: current-only, point-in-time, or include-expired modes
- SupersedeEntity: atomic expire-old + insert-new in single transaction
Schema version bumped to 40.
* fix(v3): review fixes for episodic store + temporal KG
- C1: Fix column name mismatch turn_count vs message_count in Go SQL
- C2: Remove redundant migration 000040 (000037 already adds temporal KG columns)
- H1: Use time.Time not int64 for TIMESTAMPTZ columns in SupersedeEntity
- H2: Add tenant_id scoping to Get/Delete for tenant isolation
- M2: Fix scanEntityTemporal to convert TIMESTAMPTZ→UnixMilli correctly
- L1: Remove unused uuid import from episodic_search.go
- Schema version corrected to 39 (only 000039 is new)
* feat(v3): implement consolidation pipeline with 3 event-driven workers
Event chain: session.completed → EpisodicWorker → episodic.created →
SemanticWorker → entity.upserted → DedupWorker
- EpisodicWorker: reuses compaction summary or calls LLM, generates L0
abstract (extractive), idempotent via source_id check
- SemanticWorker: extracts KG facts from episodic summary via existing
Extractor, sets temporal valid_from, publishes entity.upserted
- DedupWorker: runs DedupAfterExtraction on new entity IDs (terminal)
- L0 abstract: sentence-based extraction (~50 tokens), no LLM needed
- All workers registered via DomainEventBus.Subscribe()
* feat(v3): implement progressive loading with L0 auto-inject + unified search
- AutoInjector: searches episodic store, builds L0 prompt section (~200 tokens),
skips trivial messages via stopword filter
- L1Cache: in-memory LRU (500 entries, 1h TTL) for structured overviews
- UnifiedSearch: cross-tier search merging episodic + document results by score
- ContextStage integration: AutoInject callback appends memory section to system prompt
- MemorySection field added to ContextState for observability
* feat(v3): add memory_expand tool for L2 episodic retrieval
New tool: memory_expand(id) returns full episodic summary with metadata.
Complements memory_search L0/L1 results with deep L2 access.
Nil-safe: returns error message when episodic store not available.
Gateway wiring + memory_search depth param + kg_search temporal param
deferred to runtime integration phase.
* feat(v3): complete Phase 5 — tool extensions + gateway wiring
- memory_search: add depth param + episodic tier search merged with docs
- kg_search: add as_of temporal param, use ListEntitiesTemporal
- memory_expand: registered in gateway startup
- Gateway: Episodic field in Stores, PGEpisodicStore in factory,
embedding provider wired, tools connected to episodic store
* fix(v3): Phase 3 review fixes — tenant isolation + AutoInject args
- C1: Add tenant_id filter to ftsSearch, vectorSearch, List queries
(prevents cross-tenant episodic memory leaks)
- C2: Fix AutoInject callback signature — agent/tenant captured by
closure, only userMessage + userID passed explicitly
- H1: Add tenant_id to List query
* feat(v3): wire per-agent v3 flags from DB into dual-mode gate
Parse v3_pipeline_enabled, v3_memory_enabled, v3_retrieval_enabled from
agent other_config JSONB via ParseV3Flags(). Resolver now sets all flags
on LoopConfig so the existing gate in loop_run.go reads from DB.
- V3Flags struct + ParseV3Flags() + ValidateV3Flags() in store layer
- v3MemoryEnabled/v3RetrievalEnabled added to Loop, LoopConfig, PipelineConfig
- Auto-inject gated on V3RetrievalEnabled (was unconditional)
- Structured perf logging for v3 pipeline runs
- v3 flag validation on both WS agent.update and HTTP PUT endpoints
* feat(v3): wire AutoInjector into pipeline for L0 memory auto-inject
Create AutoInjector at gateway startup from episodic store, pass through
ResolverDeps → LoopConfig → Loop. Pipeline adapter builds AutoInject
callback capturing agent/tenant context via closure.
ContextStage already gates on V3RetrievalEnabled + AutoInject != nil.
* feat(v3): add tool metadata map + capability-based deny rules
Registry gains per-tool ToolMetadata map with RegisterWithMetadata()
and GetMetadata() (infers defaults from tool name when not explicit).
PolicyEngine gains DenyCapability() for RBAC integration — tools with
denied capabilities filtered at step 8 after existing 7-step pipeline.
* fix(v3): add RWMutex to PolicyEngine capability deny fields
DenyCapability() and SetRegistry() now guarded by sync.RWMutex.
FilterTools reads snapshot under RLock. Prevents data race when
capability rules are modified concurrently with tool filtering.
* feat(v3): implement delegate tool for inter-agent task delegation
New `delegate` tool wraps existing agent_links infrastructure
(CanDelegate, DelegateTargets). Supports async (fire-and-forget)
and sync (block with timeout) modes. Permission checked via
AgentLinkStore. Events emitted: delegate.sent/completed/failed.
DelegateRunFunc injected by gateway to avoid circular dependency.
* feat(v3): complete 3 deferred implementations
1. OrchestrationMode resolution: ResolveOrchestrationMode() checks
team membership → delegate links → spawn (priority order).
2. PG EvolutionMetricsStore: RecordMetric, QueryMetrics, aggregate
tool/retrieval metrics, TTL cleanup. All queries tenant-scoped.
3. BridgePromptBuilder: implements PromptBuilder interface by
delegating to existing BuildSystemPrompt(). Appends v3 memory
L0 section when enabled. Ready for template engine swap later.
* fix(v3): address code review findings on commits 5-6
- C1: CanDelegate now tenant-scoped (fail-closed on missing tenant)
- H1: Sync delegate timeout capped at 600s
- H2: Async goroutine gets 10min deadline (prevents leaks)
- H3: JSONB casts use COALESCE/NULLIF guards (handles missing fields)
- M1/M2: Remove dead code (formatVaultSection, memoryL0ToStrings)
* fix(teams): stop auto-creating agent_links for team members
Teams use agent_team_members table directly — agent_links caused
context confusion between team dispatch and delegation systems.
- Remove autoCreateTeamLinks() calls from team create + member add
- Remove link cleanup from member remove
- Remove dead autoCreateTeamLinks() function
- Append DELETE to migration 000039: clear team-created agent_links
* fix(v3): tenant isolation for all agent_links queries + PromptBuilder Instructions
- DelegateTargets, GetLinkBetween, SearchDelegateTargets,
SearchDelegateTargetsByEmbedding, DeleteTeamLinksForAgent all now
scoped by tenant_id (fail-closed on missing tenant)
- BridgePromptBuilder now maps Instructions/InstructionContent to
AGENTS.md context file (was silently dropped)
* feat(v3): wire orchestration mode + evolution metrics into agent loop
- Orchestration mode: resolver resolves mode from team/links, tool filter
hides delegate/team_tasks based on mode, prompt builder injects delegation
targets section
- Evolution metrics: non-blocking goroutine records tool execution metrics
(name, success, duration) via EvolutionMetricsStore in both v2 loop and
v3 pipeline paths (sequential + parallel)
- Fix review findings: tenant ID propagated via store.WithTenantID in
background goroutine, 5s timeout prevents goroutine leak
* feat(v3): implement suggestion engine with pluggable analysis rules
- PG EvolutionSuggestionStore: CRUD for agent_evolution_suggestions table
- SuggestionEngine: aggregates 7-day metrics, runs rules, deduplicates
pending suggestions per type before creating new ones
- 3 initial rules: LowRetrievalUsage (usage_rate<0.2), ToolFailure
(success_rate<0.1), RepeatedTool (>100 calls/week → suggest skill)
- EventSuggestionCreated event type added to eventbus
- Cron wiring deferred to gateway startup integration pass
* feat(v3): implement auto-adapt guardrails with apply/rollback
- AdaptationGuardrails: max delta per cycle, min data points, locked
params, rollback-on-drop percentage
- ApplySuggestion: applies threshold suggestions to agent other_config
JSONB, stores baseline for rollback
- RollbackSuggestion: restores baseline values from suggestion params
- EvaluateApplied: compares post-apply metrics to baseline, auto-rolls
back when quality drops beyond threshold
- Scope limited to retrieval params only (never security settings)
* feat(v3): wire evolution stores + daily/weekly cron for suggestions
- Add EvolutionMetrics + EvolutionSuggestions to Stores struct + PG factory
- Wire EvolutionMetricsStore into ResolverDeps (cmd/gateway_managed.go)
- Add gateway_evolution_cron.go: daily suggestion analysis + weekly
evaluation/rollback for applied suggestions
- Cron runs as background goroutine with 5-min timeout per cycle
* fix(v3): address code review findings on evolution engine
- C1: persist baseline parameters before marking suggestion as applied
(was building map but never saving — rollback would always fail)
- H1: add tenant_id isolation to UpdateSuggestionStatus, GetSuggestion,
and new UpdateSuggestionParameters method
* test(v3): add unit tests for orchestration, suggestions, guardrails, prompt
- orchestration_mode_test: orchModeDenyTools (4 modes) + ResolveOrchestrationMode
(4 scenarios with mock stores)
- suggestion_rules_test: LowRetrievalUsage, ToolFailure, RepeatedTool with
threshold boundary tests (at/below/above min data points)
- evolution_guardrails_test: DefaultGuardrails values + CheckGuardrails
(insufficient data, locked params, zero-min fallback)
- prompt_builder_orchestration_test: BridgePromptBuilder orchestration section
presence/absence across 4 scenarios + target content verification
* test(v3): add integration tests for evolution metrics + suggestions
- Test helper: shared PG connection with sync.Once migration, per-test
tenant+agent seed with cleanup
- Evolution metrics: RecordMetric, AggregateToolMetrics (success rate),
Cleanup (TTL deletion)
- Evolution suggestions: full CRUD, UpdateSuggestionParameters (baseline
persist), tenant isolation (cross-tenant read blocked)
- Pipeline E2E: seed 25 failed tools + 55 low-usage retrievals, verify
SuggestionEngine creates suggestions, verify dedup on second run
- Fix: migration 039 de-duped (episodic_summaries already in 037)
- Fix: NULL reviewed_by scan via sql.NullString
* feat(v3): add HTTP API handlers for evolution, vault, episodic, orchestration, v3-flags
5 new handler files exposing v3 backend stores as REST endpoints:
- evolution_handlers.go: metrics query/aggregate + suggestions CRUD
- vault_handlers.go: cross-agent document listing + search + links
- episodic_handlers.go: episodic summaries list + hybrid search
- orchestration_handlers.go: computed mode + delegate targets (read-only)
- v3_flags_handlers.go: per-agent v3 feature flag get/toggle
Store fixes from code review:
- episodic FTS: use inline to_tsvector (no stored tsv column)
- episodic: conditional user_id filter in List + Search (admin view)
- episodic: add tenant_id to ExistsBySourceID + PruneExpired
- evolution: require tenant_id in context (no struct fallback)
- evolution: check RowsAffected on suggestion updates
- vault: optional agent_id filter in ListDocuments (cross-agent)
* feat(v3): add web UI for evolution tab, v3 settings, vault page, episodic memory
Agent Detail enhancements:
- V3 Settings section: pipeline/memory/retrieval flag toggles
- Orchestration section: mode badge + delegate targets display
- Evolution section: added metrics + suggestions v3 flag toggles
- Evolution tab: Recharts metrics charts + suggestion review table
with approve/reject/rollback actions + guardrails card
New pages:
- /vault: Knowledge Vault document registry with cross-agent listing,
hybrid search dialog, document detail with wikilinks
- Memory page: added Episodic Memory tab with summary cards,
expandable details, key topic badges, and hybrid search
Infrastructure:
- HttpClient: added patch() method
- Query keys: v3Flags, orchestration, evolution namespaces
- 4 new hooks: use-v3-flags, use-orchestration, use-evolution-metrics,
use-evolution-suggestions, use-vault, use-episodic
- i18n: vault namespace (en/vi/zh), agents + memory keys updated
- Reused formatRelativeTime from lib/format.ts (eliminated 3 duplicates)
* refactor(http): add bindJSON helper and migrate all decode call sites
Replace 36 json.NewDecoder(r.Body).Decode + error blocks with bindJSON
across 20 HTTP handler files. Standardizes decode error responses to
structured writeError format. Fixes unchecked decode in handleIndexAll.
* refactor(store): adopt sqlx for PG scan operations (Phase 1+2)
Add jmoiron/sqlx v1.4.0 with camelToSnake json tag mapper.
Migrate scan-heavy PG store methods to sqlx Get/Select:
- tracing.go: GetTrace, ListTraces, ListChildTraces, GetTraceSpans, GetCostSummary
- heartbeat.go: Get, ListDue, ListLogs
- providers.go: GetProvider, GetProviderByName, ListProviders, ListAllProviders
- mcp_servers.go: GetServer, GetServerByName, ListServers
- pairing.go: ListPending, ListPaired
- agents_export_queries.go: 5 export functions
- agents_export_team_queries.go: exportTeamMembers, ExportAgentLinks
All writes (INSERT/UPDATE/DELETE), execMapUpdate, and dynamic WHERE
builders remain raw SQL. Zero behavior change.
* refactor(store): adopt sqlx for SQLite scan operations (Phase 3)
Migrate SQLite store scan methods to sqlx Get/Select:
- providers.go: GetProvider, GetProviderByName, ListProviders, ListAllProviders
- tenants.go: GetTenant, GetTenantBySlug, ListTenants, GetTenantUser, ListUsers, ListUserTenants
- mcp_servers.go: GetServer, GetServerByName, ListServers
Create sqlx_scan_structs.go with sqliteTime-aware scan structs
(providerRow, tenantRow, tenantUserRow, mcpServerRow) to handle
SQLite TEXT timestamp parsing via StructScan.
* refactor(store): migrate PG bulk scan operations to sqlx (Phase 4)
Migrate scan-heavy methods across 6 PG store files:
- tenant_store.go: GetTenant, GetTenantBySlug, ListTenants, GetTenantUser,
ListUsers, ListUserTenants — removed 3 scan helpers
- teams.go: ListTeams, GetTeam, ListMembers, ListMembersByTenant
- teams_tasks_activity.go: ListComments, ListEvents, ListFollowUps
- pending_message_store.go: ListPending, ListByHistoryKey
- skills_grants.go: ListAgentGrants
- config_permissions.go: CheckPermission
~20 scan ops converted. Files with encryption post-processing,
pq.Array, pgvector, or dynamic SQL kept raw.
* refactor(store): extract shared CamelToSnake mapper, add UUIDArray usage note
- Move camelToSnake to internal/store/column_mapper.go (DRY)
- Both pg and sqlitestore packages now import shared CamelToSnake
- Add planned-use comment on UUIDArray type
* refactor(cli): migrate commands from config.json to HTTP API, add providers/setup/TUI
- Add unified HTTP client (gateway_http_client.go) with auth, error parsing, typed generics
- Rewrite agent list/add/delete to use gateway HTTP API instead of config.json
- Rewrite channels list to HTTP API, add channels add/delete subcommands
- Replace models command with full providers CRUD (list/add/update/delete/verify)
- Add setup wizard command (provider → agent → channel post-onboard flow)
- Add Bubble Tea TUI behind build tag (tui/!tui with noop fallback)
- Update onboard next-steps to mention goclaw setup
- Add build-tui Makefile target
- Fix URL path injection (url.PathEscape on all user-supplied path segments)
- Fix UTF-8 truncation in skills description display
* refactor(store): add explicit db struct tags, fix sqlx mapper for heartbeat scan error
Switch sqlx mapper from NewMapperFunc (which only applies CamelToSnake to
field names, not tag values) to NewMapperFunc("db", CamelToSnake) with
explicit db:"column_name" tags on all store structs.
Root cause: NewMapperFunc("json", fn) sets mapFunc but not tagMapFunc,
so camelCase json tags like "agentId" were used as-is instead of being
converted to "agent_id", causing "missing destination name" scan errors.
Fix: use db struct tags as the source of truth for column mapping.
Every DB entity field gets db:"column_name", nested JSON configs and
runtime-only structs get db:"-".
* test(store): add integration tests for 13 store interfaces (70 tests)
Cover Tier 1 (critical) + Tier 2 (security) stores with integration tests
running against pgvector pg18. Coverage from 2.4% to ~54%.
Stores tested: Session, Agent, Team/Task, Memory, KnowledgeGraph, Vault,
MCP Server, API Key, ConfigPermission, Contact.
Infrastructure: fixture builders (seedTeam, seedMCPServer, etc.),
mock EmbeddingProvider, multi-tenant helpers, expanded cleanup.
* fix(store): resolve NULL scan bugs in MCP server and task metadata
- mcp_servers: COALESCE nullable TEXT columns (display_name, command,
url, api_key, tool_prefix) to prevent sqlx scan failures
- mcp_servers_access: COALESCE nullable JSONB columns in ListAgentGrants
(tool_allow, tool_deny, config_overrides) to prevent silent row drops
- teams_tasks: default task metadata to '{}' instead of nil to satisfy
NOT NULL constraint on CreateTask
- sqlx_helpers: export InitSqlx for integration test setup
* feat(pipeline): fix v3 pipeline context injection, tracing, KG temporal filters
- Pipeline context: add InjectContext + LoadSessionHistory callbacks to
ContextStage, propagate enriched ctx via state.Ctx for iteration stages
- Pipeline tracing: wrap makeCallLLM with emitLLMSpanStart/End, wrap
makeExecuteToolCall/Raw with emitToolSpanStart/End
- Token counter: switch pipeline from FallbackCounter to TiktokenCounter
- KG temporal: add valid_until IS NULL filter to all entity/relation
queries (list, search, vector, FTS, traversal CTE, stats)
- Skills: add SkillEmbedder interface for future hybrid BM25+vector search
- Cache: remove unused tenantResolve dead code from PermissionCache
- Store: fix NULL scan bugs in tracing metadata and agent skill_nudge
- Test: add TestStoreKG_TemporalFilter integration test
- UI: add v3 version badge, evolution section, memory/traces improvements
* refactor(store): migrate KG store from raw sql.Rows to sqlx StructScan
Migrate 6 knowledge graph store files from manual rows.Scan() to
pkgSqlxDB.GetContext/SelectContext with intermediate scan row structs.
- Add entityRow, relationRow, traversalRow, dedupCandidateRow structs
with json.RawMessage for jsonb and time.Time for timestamptz columns
- Add toEntity()/toRelation() converters (UnixMilli + json.Unmarshal)
- Add sqlxTx() helper for wrapping *sql.Tx with sqlx mapper
- Fix ScanDuplicates passing time.Now().Unix() to TIMESTAMPTZ column
- Fix ListEntitiesTemporal missing tenant scope (scopeClause)
- Fix SupersedeEntity missing tenant scope and tenant_id on INSERT
- Fix DedupCandidate.CreatedAt using Unix() instead of UnixMilli()
- Update agents_export_queries.go to reuse new scan row structs
- Net -160 lines of manual scan boilerplate removed
* refactor(store): migrate memory, skills, agents, sessions, mcp, cron, vault stores to sqlx
Batch migration of 19 store files from raw rows.Scan() to
pkgSqlxDB.GetContext/SelectContext with intermediate scan row structs.
Groups migrated:
- Memory: memory_docs, memory_admin, memory_search, memory_embedding_cache
- Episodic: episodic_search, episodic_summaries
- Skills: skills, skills_admin, skills_embedding, skills_export_queries
- Agents: agents (backfill+shares), agents_context, agents_export_team_standalone
- Sessions: sessions_list (List, ListPaged, ListPagedRich)
- MCP: mcp_servers_access, mcp_export_queries
- Cron: cron_exec (GetRunLog)
- Vault: vault_documents (ListDocuments, ftsSearch, vectorSearch)
- Tenant: tenant_configs (ListDisabled, ListAll)
7 new scan row files created. Net -510 lines of manual scan boilerplate.
INSERT/UPDATE/DELETE and scalar COUNT queries kept as raw SQL.
* fix(store): fix 3 sqlx scan struct db tag issues found by audit
- Fix vault FTS alias mismatch: `AS rank` → `AS score` (critical: runtime scan error)
- Fix episodic key_topics type: json.RawMessage → pq.StringArray (TEXT[] column)
- Fix agentShareRow.CreatedAt: string → time.Time, wire to output struct
* feat(providers): implement Wave 2 provider resilience and intelligence
9-phase implementation covering:
- Request middleware chain with composable body transformers
- OpenAI prompt caching, service tier, and fast mode middlewares
- Error classification (9 categories) with two-tier failover
- Model registry with forward-compat resolvers (Anthropic + OpenAI)
- Embedding providers (OpenAI + Voyage) with 1536-dim validation
- Cooldown/probe system with per-provider:model state tracking
- Markdown-aware chunking shared across 5 channels
- Session recall via FTS + pgvector on episodic summaries
- Dreaming/promotion pipeline for long-term memory consolidation
Migrations: 000040 (episodic search index), 000041 (promoted_at column)
Schema version: 39 → 41
* feat(providers): wire model registry into gateway provider construction
Create InMemoryRegistry with Anthropic + OpenAI forward-compat resolvers
at gateway startup. Pass to all Anthropic and OpenAI providers created
from both config and DB sources.
* feat(consolidation): wire DomainEventBus and consolidation pipeline
Create DomainEventBus at gateway startup, thread through resolver →
LoopConfig → Loop → PipelineDeps. Emit session.completed event after
each run finalization. Register consolidation pipeline (episodic →
semantic → KG dedup → dreaming) with event bus subscriptions.
* fix(store): fix episodic key_topics pq.Array, ON CONFLICT, and migration 040 immutability
- episodic_summaries.go Create: json.Marshal(KeyTopics) → pq.Array (text[] column)
- episodic_search.go scanEpisodic/scanEpisodicRow: json.RawMessage → pq.StringArray
- episodic_summaries.go Create: ON CONFLICT add WHERE source_id IS NOT NULL for partial index
- migration 040: add immutable_array_to_string wrapper (array_to_string is STABLE in PG)
* test(store): add 17 integration tests for skills, cron, episodic, tenant configs
- Skills store: 6 tests (CRUD, grants, tenant isolation)
- Cron store: 4 tests (job CRUD, run log sqlx scan, pagination, tenant isolation)
- Episodic store: 4 tests (summary CRUD, list, FTS search, tenant isolation)
- Tenant configs: 3 tests (tool/skill disable, list, tenant isolation)
- Test helper: add cleanup for skills, cron, episodic tables
* fix(permissions): use cron-specific permission check for cron tool (#725)
* fix(security): harden exec path exemption matching (#721)
- Add absolute path exemption for dataDir/skills-store/ (fixes skill
scripts using absolute paths like /app/data/skills-store/ being denied)
- Strip surrounding quotes before prefix matching (LLMs often quote paths)
- Reject path traversal ("..") in exempt fields to prevent escape
- Switch from "any field exempt → skip" to per-field matching: only exempt
if ALL fields that match the deny pattern are individually exempt
- Closes pipe/comment bypass vectors where an exempt path in one argument
would exempt the entire command including non-exempt paths
Includes 27 test cases covering: legitimate access, quoted paths,
path traversal, unicode bypass, pipe/comment bypass, mixed args.
* fix(permissions): use cron-specific permission check for cron tool
Cron tool was hardcoded to check `file_writer` configType via
CheckFileWriterPermission(), ignoring the `cron` configType that
the UI actually saves when granting cron permissions. This caused
agents in group chats to be denied cron access even with correct
permission configured.
Add ConfigTypeCron constant and CheckCronPermission() that checks
`cron` configType first, falling back to `file_writer`.
---------
Co-authored-by: Viet Tran <viettranx@gmail.com>
* fix(chat): load message history on first conversation click (#730)
* fix(chat): load message history when selecting existing conversation from clean state
The skipNextHistoryRef was unconditionally set when sessionKey transitioned
from empty to non-empty. This prevented loadHistory() from running when
clicking an existing conversation from the initial /chat page. The skip
was only intended for the new-chat send flow where the optimistic message
is already displayed.
Guard the skip with expectingRunRef so it only activates when a message
send is in flight.
Closes #729
* docs: add UI diff evidence for PR #730
Before/after screenshots and HTML comparison report showing
first conversation click behavior fix.
* feat(whatsapp): port native WhatsApp channel with whatsmeow from dev
Cherry-pick
|
||
|
viettranx
|
20c4478fe1 |
fix(security): harden exec path exemption matching
- Add absolute path exemption for dataDir/skills-store/ (fixes skill
scripts using absolute paths like /app/data/skills-store/ being denied)
- Strip surrounding quotes before prefix matching (LLMs often quote paths)
- Reject path traversal ("..") in exempt fields to prevent escape
- Switch from "any field exempt → skip" to per-field matching: only exempt
if ALL fields that match the deny pattern are individually exempt
- Closes pipe/comment bypass vectors where an exempt path in one argument
would exempt the entire command including non-exempt paths
Includes 27 test cases covering: legitimate access, quoted paths,
path traversal, unicode bypass, pipe/comment bypass, mixed args.
|
||
|
Viet Tran
|
e183b459c9 |
feat: SQLite desktop edition — full desktop app with team tasks (#505)
* feat(store): add SQLite backend foundation with build-tag injection
Add sqlitestore package (//go:build sqlite) as alternative to PostgreSQL:
- pool.go: WAL mode, busy_timeout, 4 concurrent read connections
- helpers.go: ? param helpers, JSON array storage, nullable/update utils
- scope.go: tenant scope with ? placeholders (vs PG's $N)
- schema.sql: 1296-line flattened DDL from 29 PG migrations (51 tables)
- schema.go: embedded schema with transactional apply + version tracking
- factory.go: NewSQLiteStores() stub (stores wired in Phase 2)
Build-tag split for store initialization:
- cmd/gateway_stores_pg.go (//go:build !sqlite) — default PG-only
- cmd/gateway_stores_sqlite.go (//go:build sqlite) — runtime PG/SQLite switch
- cmd/gateway_setup.go: extracted wireTracingAndCron() shared helper
Config: GOCLAW_STORAGE_BACKEND + GOCLAW_SQLITE_PATH env vars.
Security: goclaw.db added to DenyPaths (exec, read_file, filesystem tools).
* feat(edition): add centralized edition package for feature tier limits
New internal/edition/ package — single source of truth for all edition limits:
- Edition struct with MaxAgents, MaxTeams, MaxChannels, KGEnabled, TeamFullMode, etc.
- Standard (default, all features) and Lite (desktop, 5 agents, 1 team) presets
- atomic.Pointer for thread-safe Current()/SetCurrent()
Wiring:
- cmd/gateway.go: GOCLAW_EDITION env override (lite/standard) at startup
- cmd/gateway_stores_sqlite.go: auto-set Lite when backend=sqlite
- /v1/edition HTTP endpoint for UI comparison modal (public, no auth)
* feat(sqlitestore): implement Phase 2A core stores + sqliteonly build tag
Implement 12 SQLite store backends (4200+ LOC) mirroring existing PG stores:
- SessionStore, AgentStore, ProviderStore, TracingStore, SnapshotStore
- ConfigSecretsStore, SystemConfigStore, TenantStore, HeartbeatStore
- BuiltinToolStore, BuiltinToolTenantConfigStore, SkillTenantConfigStore
All stores wired in factory.go. Remaining stores (Memory, Cron, Skills,
Teams, etc.) left nil — gateway handles gracefully.
Add sqliteonly build tag for PG-free desktop builds:
- go build . → PG only
- go build -tags sqlite . → PG + SQLite (runtime switch)
- go build -tags sqliteonly . → SQLite only (no pgx dependency)
Key SQLite adaptations:
- ? placeholders (not $N), json_extract/json_each/json_array_length
- DISTINCT ON → GROUP BY + Go dedup, ANY($1) → IN (?,?...)
- rows.Err() checks on all scan loops, execMapUpdateWhereTenant in helpers
- GetOrCreateUserProfile uses RowsAffected() instead of PG's xmax trick
* feat(sqlitestore): implement Phase 2B feature stores
Port 8 additional store backends to SQLite (19 new files, ~6000 LOC):
Cron: cron.go, cron_crud.go, cron_exec.go, cron_scheduler.go
- Job scheduling with cache, ListDue, MarkRunning, MarkComplete
Skills: skills.go, skills_crud.go, skills_content.go, skills_grants.go
- CRUD, grants, content management. LIKE search (no FTS/vector in Lite)
MCP: mcp_servers.go, mcp_servers_access.go, mcp_user_credentials.go
- Server CRUD, agent/user grants, encrypted credentials
Channels: channel_instances.go, pairing.go, pending_messages.go, contacts.go
- Channel management, device pairing, message queue, contact store
Teams: teams.go, teams_tasks.go, teams_tasks_lifecycle.go, teams_tasks_activity.go
- Team/task CRUD, lifecycle transitions, activity log, progress tracking
- JSON array for members/blocked_by (replaces PG text[])
Remaining nil stores: Memory, AgentLinks, KnowledgeGraph, Activity,
SecureCLI, APIKeys, ConfigPermissions — gateway handles gracefully.
* fix(sqlitestore): fix critical arg ordering + data races in Phase 2B stores
C1: ListTasks arg mismatch — limit+1 in userID slot, wrong results
C2: SearchTasks missing duplicate userID arg for (? = '' OR t.user_id = ?)
H3: cron_crud.go cacheLoaded written without mutex → use InvalidateCache()
H4: cron_exec.go discarded ExecContext error → log warning
H1: Add rows.Err() checks in cron_crud, cron_scheduler, cron_exec, pairing
* feat(sqlitestore): implement Phase 2C — Memory, Activity, APIKeys, ConfigPermissions
Complete remaining essential stores (6 new files, ~900 LOC):
Memory: memory.go, memory_docs.go, memory_search.go
- Document/chunk CRUD, LIKE-based search (no vector in Lite edition)
- Embedding methods return empty results gracefully
Activity: activity.go — simple activity logging
APIKeys: api_keys.go — API key CRUD with SHA-256 hash lookup
ConfigPermissions: config_permissions.go — permission rules with TTL cache
24/27 stores now wired. Only AgentLinks, KnowledgeGraph, SecureCLI
remain nil (disabled in Lite edition by design).
Total: 49 files, 11,435 LOC in internal/store/sqlitestore/
* fix(sqlitestore): fix variable shadow in GetDocument + handle chunk delete error
C1: GetDocument scopeClause used := inside if/else blocks, shadowing
outer err variable — query errors silently swallowed. Fixed by renaming
to tcErr matching PG pattern.
H1: IndexDocument chunk deletion ExecContext error was discarded, could
cause duplicate chunks. Now returns error on failure.
* feat(desktop): Phase 3 — Wails v2 desktop app shell with chat UI
Desktop app (ui/desktop/) using Wails v2 + React 19 + Tailwind CSS 4:
Go backend:
- main.go: Wails entry point with embedded frontend assets
- app.go: gateway embedding via goroutine, health check, Wails bindings
- keyring.go: OS keyring secrets with file fallback
- cmd/gateway_export.go: exports RunGateway() for desktop embedding
React frontend:
- WS v3 client: protocol handshake, exponential backoff, call queue
- Chat system: Zustand stores, RAF-batched streaming, 10 event handlers
- Components: MessageBubble, MarkdownRenderer (rehype-sanitize),
CodeBlock, ToolCallBlock, ThinkingBlock, ActivityIndicator, InputBar
- Layout: AppShell (2-column), Sidebar with agent/session list, TopBar
- Onboarding wizard (5 steps): welcome, gateway, provider, agent, ready
- Magic Blue theme (dark/light), Inter + JetBrains Mono typography
Build: all 3 variants pass (PG, sqlite, sqliteonly)
* chore: ignore Wails build artifacts (wailsjs, build, package.json.md5)
* fix(desktop): correct onboarding provider list and agent creation API contract
- ProviderStep: expand from 3 to 16 providers in 4 groups (Popular, Cloud, Local, Regional)
with correct provider_type values and api_base defaults
- AgentStep: fix API payload — use agent_key (slug), provider (name string),
agent_type=predefined with description in other_config
- use-agents: fix field mapping — agent_key, display_name from backend response
- Clean up failed provider on verify error
* fix(desktop): add missing providers — Bailian Coding, Z.ai Coding, Ollama Cloud
* fix(desktop): match web dashboard brand colors + persist onboarding in store
- Replace Magic Blue theme with web dashboard's warm blue OKLCH palette
- Move onboarding state from localStorage to Zustand persist store
- Add "Run Setup Wizard" option in TopBar settings menu to re-trigger onboarding
- Fix light mode theme activation (explicit :root:not(.dark) overrides)
* fix(desktop): apply dark theme before first paint + better error messages
- Add class="dark" default on <html> + inline script to read persisted theme
before React hydrate (prevents light flash on dark mode)
- Improve provider test error message for network failures
* feat(desktop): add GoClaw logo from web UI to onboarding + topbar
* fix(desktop): add Vite proxy for dev mode to avoid CORS gateway errors
- Proxy /v1, /ws, /health to localhost:18790 in Vite dev server
- Use relative URLs in dev mode (import.meta.env.DEV) so proxy handles CORS
- Production build uses direct gateway URL from Wails binding
* fix(desktop): GoClaw dock icon + fix duplicate provider slug on re-test
- Convert goclaw-icon.svg to 1024x1024 PNG for macOS dock icon
- Fix provider creation: handle existing slug by finding and updating
- Track build/appicon.png in git (exclude only build/bin/)
* fix(sqlitestore): UUID text/blob mismatch in scopeClause breaks all queries
scopeClause passed uuid.UUID (16-byte array) to SQLite ? placeholder,
but tenant_id column stores TEXT strings. SQLite compared BLOB vs TEXT
→ no match → all scoped queries returned empty results.
Fix: pass scope.TenantID.String() to ensure TEXT comparison.
Also includes:
- macOS dock icon (.icns from GoClaw logo)
- Onboarding auto-detect existing providers/agents
- Debug logging for token + API URL
* fix(desktop): CORS for dev mode + direct gateway URL
- Wails dev server (port 34115) doesn't proxy API calls, causing 405
- Frontend now connects directly to gateway URL from Wails binding
- Added GOCLAW_DESKTOP=1 env → enables CORS middleware on gateway
- desktopCORS wraps mux with Access-Control-Allow-* headers + OPTIONS
- Simplified provider test flow: list-then-create/update
* fix(desktop): split onboarding into 6 steps matching web UI flow
Web UI flow: create provider → select model + verify → create agent.
Desktop was incorrectly trying to verify before creating provider.
Changes:
- ProviderStep: now only creates/saves provider (no verify)
- NEW ModelVerifyStep: loads models from provider, test connection
- AgentStep: receives pre-selected model, shows read-only model field
- OnboardingWizard: 6 steps with auto-detect skip logic
- Auto-detect: has agents→Ready, has providers→ModelVerify, nothing→Provider
* feat(desktop): add Combobox component, use searchable model selector
- New Combobox: searchable dropdown with custom value support
- ModelVerifyStep: replace plain input/select with Combobox
- Models loaded from GET /v1/providers/{id}/models API
- Allows typing custom model name if API returns empty list
* fix(desktop): rename icon to iconfile.icns matching Wails convention
* refactor(desktop): overhaul UI/UX, fix onboarding, fix event handling
Desktop app major refactor:
UI/UX:
- Chat-focused layout with floating panels on dotted canvas
- Sidebar: agent list + sessions grouped by date (no resource counts)
- Modern input bar: rounded pill with attach/send buttons
- User bubble: card style matching web UI (not solid color)
- Thinking block: collapsible, max-height, proper label
- ErrorBoundary wrapping app
- Dock icon: regenerated with transparent bg + dark rounded frame
Onboarding:
- 3-step flow matching web UI (Provider → Model → Agent)
- Proper create vs update (check DB before POST)
- SetupStepper with step circles + connectors
- Agent presets from web UI (Fox Spirit, Artisan, Astrologer)
- Auto-detect existing setup via use-bootstrap-status hook
Event handling (verified from Go source):
- chunk: payload.content (not payload.chunk)
- thinking: payload.content (not payload.thinking)
- tool.call: payload.id/name (not toolId/toolName)
- tool.result: payload.is_error/content (not error field)
- run.completed: usage.prompt_tokens/completion_tokens
- New: block.reply, run.retrying handlers
Backend fixes:
- SQLite scanTime helper for modernc.org/sqlite text timestamps
- X-GoClaw-User-Id header in desktop API client (not X-User-ID)
- user_id: system (owner role in desktop single-user mode)
- CORS: allow X-GoClaw-User-Id header
- SQLite busy timeout: 5s → 10s
- Snapshot SQL: cross-DB compatible (FILTER→CASE, ::BIGINT→CAST)
- Promise.allSettled for bootstrap status (one fail doesn't block other)
* feat(desktop): chat polish, SQLite fixes, summoning modal, bootstrap guard
Chat Polish (Phase 1):
- ToolCallBlock: Wrench/Zap icons, phase badges, arg summary, grouped rendering
- ThinkingBlock: auto-expand on stream, Brain icon, cursor pulse
- MessageBubble: isStreaming prop, streaming cursor, grouped tool calls
- ImageLightbox: fullscreen overlay, gallery nav, keyboard shortcuts, download
- MediaBlock: grid layout, click-to-open lightbox, hover overlay
- ActivityIndicator: phase-specific icons (Brain/Wrench/RefreshCw)
- ChatCanvas: track lastAssistantId for streaming, EmptyState with prompts
- Filter [System] nudge messages and tool-role messages from chat history
SQLite Fixes:
- sessions_list: fix time.Time scan failure (3 sites) using sqliteTime scanner
- snapshots: fix ON CONFLICT expression mismatch with unique index
- snapshots: fix GetLatestBucket using nullSqliteTime
- pool: explicit PRAGMAs (busy_timeout=15s, WAL, synchronous=NORMAL)
- schema: seed master tenant (was missing, causing FK violations)
- schema: incremental migration framework (version-gated patches)
Onboarding:
- SummoningModal: port from web UI with framer-motion animations
- AgentStep: show summoning modal after create, continue button
- App: auto-detect empty DB and reset onboarded flag
- ChatCanvas: loading spinner while agent loads after onboarding
Bootstrap Guard:
- After auto-cleanup of BOOTSTRAP.md, check if USER.md is still empty
- Inject reminder if agent cleared BOOTSTRAP but didn't fill USER.md
Session Management:
- Load chat history on session click (was missing useEffect)
- Fix race condition: don't clear messages on session switch (atomic replace)
- SidebarFooter: center New Chat button text
* fix(desktop): session management, delete confirm, event listener race fix
- New Chat: only clears state, no empty session creation (sendMessage auto-creates)
- Delete session: hover X button with ConfirmDialog confirmation
- Event listener: use sessionKeyRef instead of closure to prevent stale events
- Remove "skip to dashboard" link from onboarding
- Add reusable ConfirmDialog + ConfirmDeleteDialog common components
* feat(desktop): settings view with tabbed layout (Phase 4)
- SettingsView: tab container with header, close button, canvas-dots bg
- SettingsTabBar: 9 tabs (Appearance, Providers, Agents, MCP, Skills, Tools, Cron, Traces, About)
- AppearanceTab: dark/light theme toggle, language + timezone placeholders
- AboutTab: version, edition limits, runtime info
- ui-store: activeView, settingsTab, openSettings(), closeSettings()
- AppShell: switch between chat and settings views
- SidebarFooter: gear icon opens settings (was "Run Setup Wizard")
- Keyboard: Cmd+, opens settings, Escape closes
- Branding: "GoClaw Lite" in sidebar header
- Agent status: online (green) instead of idle for desktop
- Tab content wrapped in solid bg card with border
* feat(desktop): provider management CRUD in settings (Phase 5)
- use-providers hook: list, create, update, delete, verify via HTTP API
- ProviderList: list view with Add button, empty state
- ProviderRow: status dot, type badge, edit/delete actions
- ProviderFormDialog: create/edit with type selector, masked API key, test connection
- Wire ProviderList into Settings Providers tab
* fix(desktop): remove Test Connection from provider form (requires model + provider ID)
* feat(desktop): agent management CRUD in settings (Phase 6)
- AgentData/AgentInput types matching web UI contracts
- use-agent-crud hook: list, create, update, delete, resummon (5 agent limit)
- AgentCard: emoji, status dot, provider/model/type badges, edit/delete/resummon
- AgentFormDialog: provider Combobox + model Combobox (from /v1/providers/{id}/models)
- Create: agent_type selector, "Check & Create" verifies model before create
- Edit: no type change, no re-verify needed
- Personality textarea for predefined agents
- AgentList: grid, edition limit warning, create triggers SummoningModal
- Delete uses ConfirmDeleteDialog (type name to confirm)
- Sidebar agent list refreshes after CRUD
* feat(desktop): agent detail panel with full config (Phase 6 polish)
Agent detail panel (fullscreen overlay covering sidebar):
- PersonalitySection: emoji editor, display name, description, status select, default toggle, agent key display
- ModelBudgetSection: provider/model Combobox with verify-before-save, context window, max tool iterations
- EvolutionSection: self_evolve toggle with info callout (predefined agents only)
- Sticky save bar: backdrop blur, cancel/save buttons, spinner on save
- Save blocked if provider/model changed but not verified
AgentList: card click opens detail panel, create dialog separate
AgentFormDialog: create-only, Check & Create with verify
* fix(desktop): resummon confirm, summoning z-index, save bar UX
- Resummon requires confirm dialog before triggering
- SummoningModal z-index z-50 → z-[70] (above detail panel z-[60])
- Save bar: show error inline, "Verify model first" when blocked
* feat(desktop): agent detail quality polish — files tab, memory config, rich cards
Types (synced with web UI):
- AgentData: added owner_id, workspace, restrict_to_workspace, frontmatter, context_window, max_tool_iterations as required fields
- MemoryConfig, CompactionConfig interfaces
- BootstrapFile type for WS file operations
AgentCard (matching web UI agent-card.tsx):
- Star icon for default agent
- Animated pulse badge for summoning status
- Self-evolve sparkle indicator (orange when active)
- Frontmatter/expertise with line-clamp-3
- Context window display (e.g. "200K ctx")
- Safe emoji extraction, UUID name detection
AgentDetailPanel:
- Tab navigation: Overview + Files tabs
- Overview: Personality + ModelBudget + Memory + Evolution sections
- Files tab: WS-based file editor (agents.files.list/get/set)
- File sidebar with selection
- Textarea editor with dirty tracking
- Save button with spinner
- Sticky save bar with backdrop blur (overview only)
- Resummon with confirm dialog
MemorySection (matching web UI memory-section.tsx):
- Enable/disable toggle
- 6 config fields: max_results, min_score, max_chunk_len, chunk_overlap, vector_weight, text_weight
- "Using global defaults" when disabled
* feat(desktop): agent files — hide USER/HEARTBEAT, add Edit with AI
- Hide USER.md, USER_PREDEFINED.md, HEARTBEAT.md from files tab (managed by bootstrap/cron)
- Add "Edit with AI" button → RegenerateDialog → POST /v1/agents/{id}/regenerate
- Auto-select first file on load
- Show file size in sidebar
- Pass agentId to files tab for regenerate API call
* fix(desktop): remove memory config section (no embedding in SQLite)
* fix(desktop): remove misleading bytes display from file sidebar
* fix(desktop): replace native checkboxes/selects with custom Switch + Combobox
- New Switch component matching Radix UI toggle style
- Replace all native <input type="checkbox"> with Switch in agent/provider forms
- Replace native <select> for status with Combobox
- All interactive elements have cursor-pointer
* fix(desktop): evolution callout colors — use opacity-based for both themes
* fix(desktop): global cursor-pointer for all interactive elements
* fix(desktop): improve dark mode contrast for text and status colors
- text-secondary: 0.62 → 0.68 lightness (better readability on dark bg)
- text-muted: 0.52 → 0.58 lightness (was below WCAG AA 4.5:1 minimum)
- success: 0.45 → 0.55 (green was too dim on dark bg)
- warning: 0.65 → 0.70 (slightly brighter)
- idle: 0.52 → 0.58 (match text-muted)
* fix(desktop): revert color values to exact web UI match (0.62/0.52/0.45)
* fix(desktop): agent card badge colors — match web UI badge variants exactly
* feat(desktop): add MCP servers + Builtin Tools settings tabs
Phase 7 implementation:
MCP Tab:
- Full CRUD with 5-server edition limit
- Form dialog with transport-conditional fields (stdio/SSE/streamable-http)
- Test Connection with inline success/error feedback
- Agent grants dialog (grant/revoke per agent)
- Tools discovery dialog (view server tools)
- KeyValueEditor with sensitive field masking (auth/token/secret)
Tools Tab:
- Category-grouped list of 41 seeded builtin tools
- Toggle enable/disable with optimistic update
- Specialized settings forms: web_fetch extractor chain, media provider chain
- Generic JSON editor fallback for other tools
- Provider/model Combobox selection for media tools
Common:
- RefreshButton component with 500ms min spin animation
- KeyValueEditor with password masking for sensitive keys
Fixes:
- SQLite builtin_tools scan: use scanTimePair() for timestamps
- Session click while in settings: now closes settings view
- Dark mode text contrast: bumped text-secondary/text-muted lightness
- Light mode text contrast: darkened text-secondary/text-muted
- Focus ring thickness: ring-2 → ring-1 globally
- Misleading "memory layering (Postgres)" log label
* feat(desktop): add Skills tab, agent skill grants, emoji avatar, agent form redesign
Skills:
- Skills settings tab with upload ZIP, toggle, delete, runtime check
- Agent skill grants section in agent detail panel (toggle per agent)
- SQLite SkillManageStore interface compliance fixes
Agent form:
- 2-column layout, wider modal (max-w-3xl)
- 6 personality presets (Fox Spirit, Artisan, Astrologer, Researcher, Writer, Coder)
- Separate Verify Model + Summon buttons
- Always predefined type (removed open option)
Fixes:
- SQLite time.Time scan: mcp_servers, mcp_grants, activity_logs
- Combobox portal with scroll/resize tracking
- AgentAvatar shows emoji from other_config
- uploadFile sends X-GoClaw-User-Id header + generic type
- isApiClientReady guard prevents ErrorBoundary crash
- Verify model field: valid (not success)
* fix(sqlitestore): comprehensive timestamp scan sweep + UI fixes
SQLite timestamp sweep (13 files, 25+ sites):
All time.Time direct scans replaced with sqliteTime/scanTimePair/nullSqliteTime.
Files: activity, teams, teams_tasks, teams_tasks_activity, config_permissions,
agents_access, tracing_spans, tracing_scan, channel_instances, tenants,
pending_messages, api_keys, heartbeat.
UI fixes:
- Agent switching: clear active session + chat when agent changes
- MCP table: vertical align middle on row cells
* fix(sqlitestore): fix json.RawMessage scan + sqliteVal for dynamic updates
- mcp_servers_access: scan json columns via string intermediates (SQLite
TEXT → json.RawMessage incompatible, use string then convert)
- helpers: add sqliteVal() to auto-marshal map/slice/struct to JSON
string in execMapUpdate/execMapUpdateWhereTenant — fixes agent save
500 error when updating other_config, tools_config, etc.
- Add AgentMcpSection: toggle MCP server grants per agent in detail panel
- Clean up debug logging from McpGrantsDialog
* feat(desktop): add i18n (react-i18next) + toast system + language/timezone pickers
i18n:
- Install react-i18next + i18next
- Copy 12 web locale namespaces (en/vi/zh) + desktop.json namespace
- Create i18n/index.ts with browser language detection + localStorage persist
- Replace ~300 hardcoded strings across 40+ components with t() calls
- Language picker in ChatTopBar (top-right) + Settings > Appearance
- Timezone picker in ChatTopBar with search + Intl.supportedValuesOf fallback
- All 6 agent presets fully translated (vi/zh) with prompts from locale files
- Agent Key never translated (uses stable English slugs)
Toast:
- Zustand toast store (success/error/warning/default, 4s auto-dismiss)
- Toaster component (bottom-right, z-100, slide-in animation)
- Toast calls in all CRUD hooks (agents, providers, MCP, skills, tools)
* feat(desktop): add Cron Jobs + Traces settings tabs with syntax highlighting
Cron Jobs (Phase 8):
- WS RPC hook (cron.list/create/delete/toggle/run/runs)
- CronList table with schedule formatting, status badges, run/toggle/delete
- CronFormDialog with slug name, agent selector, 3-way schedule (every/cron/once)
- CronRunsDialog for execution history
Traces (Phase 9):
- HTTP REST hook (GET /v1/traces with pagination + agent filter)
- TraceList table with duration, tokens, spans, relative time
- TraceDetailDialog with metadata, collapsible input/output, flat span list
- Syntax-highlighted JSON/code previews via react-syntax-highlighter
* fix(sqlitestore): fix tracing scan — endTime *time.Time → nullSqliteTime
Both trace and span scan functions used *time.Time for nullable end_time
column, which fails on SQLite TEXT timestamps. Changed to nullSqliteTime
with Valid check before assigning pointer.
* fix(desktop): fix cron schedule type label — map 'at' kind to 'once' i18n key
* fix(desktop): prevent flash on cron/traces refresh — keep data while refetching
* feat(desktop): add file attachment rendering — FileButton + FilePreviewDialog
- FileButton: compact attachment button with emoji icon, filename, size, download
- FilePreviewDialog: modal with type-detected preview (image/video/audio/markdown/code/text)
- MarkdownRenderer: override a/img for /v1/files/ links → FileButton + resolved URLs
- MediaBlock: non-image files render as FileButton instead of plain links
- api.ts: add getBaseUrl() for file URL resolution
* fix(desktop): resolve file URLs in chat — toFileUrl for media_refs + relative paths
- use-chat: add toFileUrl() to convert raw paths to /v1/files/{basename} URLs
for both run.completed media and history media_refs
- MarkdownRenderer: detect relative file paths (./path/file.ext) via isFileLink
in addition to /v1/files/ links, resolve all to gateway URL
* fix(desktop): authenticated file serving — media cache + blob URLs
Security fix: all /v1/files/ requests now use Bearer auth via fetchFile().
No more raw <a href> or <img src> with unauthenticated file URLs.
- Add media-cache.ts (blob cache with 5-min TTL, dedup inflight fetches)
- Add use-media-url.ts hook (returns cached blob URL for authenticated media)
- Add AuthImage component (loads images via auth blob) + downloadFile helper
- Update MarkdownRenderer: file images use AuthImage, downloads use downloadFile
- Update FileButton: authenticated download via blob
- Update FilePreviewDialog: authenticated fetch for text/preview content
- Clean filename display (strip timestamps + query params)
* fix(desktop): fix file attachments — use media_refs.id for URL, auth all media
Root cause: media_refs from backend has {id, mime_type, kind} but no path/url.
The id IS the filename basename. Fixed toFileUrl to use ref.id as fallback.
Also: AuthImage for lightbox, authenticated audio/video in MediaBlock,
clean filename display (strip timestamps).
* fix(desktop): file serving, traces, and media rendering improvements
Backend:
- Store MediaRef.Path in loop_finalize for direct file serving
- Use full path (not basename) in gateway_managed event signing
- Add fuzzyMatchInDir for LLM-hallucinated filenames
- Add findInWorkspace support for agent dirs and ws/ directory
- Add POST /v1/files/sign endpoint for client-side URL signing
- Always save LLM span input_preview (not just verbose mode)
- Truncate previews from tail (keep recent context), limit 2000 chars
- Add exact filename hint to create_image/video/audio tool results
Desktop frontend:
- File attachments: FileButton, FilePreviewDialog, AuthImage with blob cache
- Media cache: sign URLs via API for non-ft URLs, Bearer auth fallback
- Download via Wails SaveFile binding (native Save As dialog)
- OpenFile + DownloadURL Wails Go bindings
- Traces: rewrite with span tree hierarchy, formatTokens (90.8K),
formatDuration with start/end fallback, expandable spans
- Traces: export via DownloadURL, copy with checkmark state
- Code preview: JSON = oneDark syntax highlight, text = light pre block
- MarkdownRenderer: baseUrl prop for relative image resolution
- ImageLightbox: preventDefault on keyboard nav (no macOS beep)
- ErrorBoundary: reload instead of re-render on retry
- Combobox: compact sizing (py-1.5, text-sm)
- FileButton: fix nested button HTML violation
* feat(desktop): team tasks kanban board + edition policy + file preview fixes
Phase 11 implementation:
Backend:
- TeamActionPolicy interface — lite/full edition gating for team_tasks tool
- Filter blocked actions from schema enum + early guard in Execute()
- System prompt: edition-specific team member guidance
- Skip skill_manage/publish_skill registration + seeding in lite
- teams.create: require at least 1 member
- files.go: 2-layer path isolation (workspace boundary + tenant scope)
Desktop UI:
- Kanban board with 6 status columns + framer-motion layout animation
- Task detail modal with collapsible description/result sections
- Team create dialog with styled member checkboxes
- Sidebar: teams section with create button + Lite edition badge
- Chat TaskPanel: compact active tasks with real-time WS updates
- Real-time: debounced get-light fetch (300ms) + progress patch (1s)
- Edition comparison modal (Lite vs Standard feature table)
- Custom dropdown filter replacing native select
- i18n: teams namespace (en/vi/zh)
Fixes:
- WS params: camelCase (teamId, taskId, sessionKey) matching backend
- FileButton: span wrapper (HTML nesting) + createPortal for dialog
- FilePreviewDialog: defensive filename normalization for extension check
- use-chat: prevSessionRef init null — fix blank chat on view switch
- Scrollbar: 3px auto-hide
- Agent click in sidebar returns to chat view
* feat: Update frontend build assets, change the default agent thinking level to 'low', and update Go module dependencies.
|
||
|
viettranx
|
731a98221a |
fix(teams): gracefully handle task status races in progress/complete
- executeProgress: early-exit on terminal tasks using pre-fetched status, check TaskActionFlags before re-querying DB, auto re-assign on stale recovery - executeComplete: handle already-completed/failed/cancelled gracefully, auto re-assign pending tasks reset by stale recovery ticker - Increase taskLockDuration 30min→60min, heartbeat 5min (was 10min) for 12x safety margin against stale recovery race conditions - Add diagnostic logging in UpdateTaskProgress when 0 rows affected |
||
|
viettranx
|
ee4d34ae28 |
feat(browser): add timeout, idle auto-close, and max-pages safety mechanisms
Prevent resource leaks and hanging actions in the browser tool: - Per-action context timeout (default 30s, configurable via timeoutMs param or config) - Idle page reaper goroutine closes pages unused for 10min (configurable) - Max pages per tenant (default 5) with LRU eviction - RefStore cleanup on page close/evict/reap to prevent memory leaks |
||
|
viettranx
|
7aa47d1baf |
refactor(security): remove cross-tenant bypass, enforce strict tenant isolation
- Add RoleOwner ("owner") to permissions hierarchy (above admin)
- Replace all IsCrossTenant/client.IsCrossTenant() permission guards with
IsOwnerRole/client.IsOwner() role-based checks (20+ locations)
- Remove cross-tenant bypass from tenantClauseN SQL helper — always adds
AND tenant_id = $N (fail-closed)
- Add explicit unscoped store methods for startup: ListAllProviders,
ListAllInstances, ListAllEnabled, GetByIDUnscoped, GetTeamUnscoped
- Remove CrossTenant from auth layer: owner users get role "owner" with
concrete tenant ID (MasterTenantID fallback), no bypass flag
- Remove crossTenant field from WS Client; connect response sends
is_owner instead of cross_tenant
- Simplify event filter from 3-mode to 2-mode (tenant-scoped + owner sees system events)
- Fix channel instance loader to use inst.TenantID for agent lookup
- Fix agent embedding goroutine to capture tenant from caller context
- Fix provider cache invalidation to use event.TenantID
- Separate skills store: ListAllSystemSkills (system only) vs
ListAllSkills (system + tenant-scoped), StoreMissingDeps restricted
to system skills only
- Frontend: isCrossTenant → isOwner, add "owner" to UserRole type,
RequireAdmin uses hasMinRole for owner > admin hierarchy
|
||
|
viettranx
|
651072a9ea |
feat(config): add system_configs DB table with per-tenant isolation and System Settings modal
- Add system_configs table (migration 029) with per-tenant key-value config - Add SystemConfigStore interface with strict tenant isolation (no cross-tenant fallback) - Add ApplySystemConfigs overlay: DB values → in-memory cfg at startup + after save - Add System Settings modal (navbar gear icon) with 3 sections: - Embedding: verified 1536d models (OpenAI, OpenRouter, Mistral), custom verify endpoint - UX Behavior: tool_status, block_reply, intent_classify toggles - Pending Compaction: provider/model + threshold/keepRecent/maxTokens - Fix block_reply label inversion: ON = delivers intermediate text (not suppresses) - Fix embedding status endpoint to read from system_configs DB (not provider JSONB) - Fix config page save to only sync current tenant (not all tenants) - Fix bus event payload to use fresh context (prevent canceled request context) - Remove config.json fallback for embedding provider/model resolution - Add "More Config" link in modal footer to full config page - Add i18n for system-settings namespace (en/vi/zh) |
||
|
viettranx
|
6e80a1af61 |
fix(skills): unblock agent read_file on skill paths and fix web UI file browser
- Add builtinSkillsDir + tenant-scoped dirs to read_file AllowPaths - Use DB file_path in HTTP skill file handlers instead of hardcoded baseDir - Add bundled dir fallback (system skills only) when managed copy is missing - Tenant-scope publish_skill tool destination directory - Extract walkSkillFiles/readSkillFile/skillSlugDir helpers (DRY) - Return is_system from GetSkillFilePath for security-guarded fallback |
||
|
Viet Tran
|
cd022699f6 |
feat: multi-tenant isolation — complete implementation (#359)
* feat(security): multi-tenant user data isolation (Plan 1)
Comprehensive user data isolation for non-owner system users:
- API key identity binding: owner_id column forces user_id on auth,
prevents spoofing via X-GoClaw-User-Id header
- Sessions: ownership checks on list/preview/patch/delete/reset,
non-admin users see only their own sessions
- Cron: user_id filtering on list, ownership checks on mutations
- Server-side WS event filtering: agent/chat/session/cron/team events
scoped per-user instead of broadcast to all clients
- Web UI role guards: RequireAdmin on 15 admin-only pages, role
propagated from WS connect response to auth store
- Tracing/activity: user_id enforcement for non-admin HTTP callers
- Teams: HasTeamAccess membership checks on get/delete/list
- Skills: fail-closed ownership check (deny non-admin if store
doesn't support owner lookup)
- HTTP auth: requireAuthBearer now enforces owner_id + user context
for file/media downloads (was missing)
- Dead code: removed delegation_history, handoff_routes tables and
all related handlers/store code
- New: team_user_grants table for user-to-team access control
Migration 000026: api_keys.owner_id + team_user_grants + DROP legacy tables
* feat(security): multi-tenant foundation — tenants table, tenant_id propagation, permission cache (Plan 2)
Add tenant isolation infrastructure across the entire gateway:
Schema (migration 000027):
- Create tenants + tenant_users tables with master tenant seed
- Add tenant_id column to 30 user-scoped tables (NOT NULL DEFAULT master)
- api_keys.tenant_id nullable (NULL = system-level cross-tenant key)
- Create builtin_tool_tenant_configs + skill_tenant_configs for per-tenant overrides
- Drop custom_tools table (agent loop integration never wired)
Store layer:
- TenantStore interface + PGTenantStore (CRUD tenants + tenant_users)
- TenantID field on AgentData + APIKeyData
- tenant_id in agents/api_keys/skills SQL (Create, Get, List)
Context propagation:
- WithTenantID/TenantIDFromContext (uuid.Nil = fail-closed)
- WithCrossTenant/IsCrossTenant (owner/system admin flag)
Auth tenant resolution:
- HTTP: resolveAuthBearer sets TenantID/CrossTenant on all 5 auth paths
- WS: handleConnect sets tenantID/crossTenant on Client
- API key 2-tier: NULL = cross-tenant (system), set = tenant-scoped
Runtime isolation:
- Event bus: TenantID field on Event, fail-closed filter in event_filter.go
- Cron: tenant context injected in RunJob handler
- Subagent: tenant validation prevents cross-tenant spawn
- Security logging: tenant_id in auth resolution logs
Tenant management:
- WS RPC: 7 methods (tenants.list/get/create/update, tenants.users.*)
- HTTP: 7 endpoints (/v1/tenants/*)
- Slug validation + path traversal prevention
- Role validation (owner/admin/operator/member/viewer)
Infrastructure:
- PermissionCache: 4 sub-caches (tenant resolve, role, agent access, team access)
- tenant_paths.go: filesystem path helpers with master-tenant backward compat
- i18n: MsgInvalidRole key + translations (en/vi/zh)
Dead code removed: custom_tools store, HTTP handler, DynamicToolLoader (-828 lines)
* feat(security): tenant query filtering + workspace isolation (Plan 3)
Add WHERE tenant_id filtering to all 30+ tenant-scoped store queries,
wire workspace filesystem isolation, and harden restrict_to_workspace.
Store query filtering:
- Add tenantClauseN/tenantIDForInsert/requireTenantID helpers
- Filter all SELECT/INSERT/UPDATE/DELETE by tenant_id for non-cross-tenant
- Refactor SessionStore.GetOrCreate and CronStore.AddJob/ListJobs to
accept context.Context for tenant propagation
- System skills (is_system=true) bypass tenant filter for all tenants
- Special cases: GetByKey (channels), GetByHash (auth) skip filter
Workspace isolation:
- Resolver computes tenant-scoped workspace + dataDir for non-master tenants
- Add WithTenantSlug/TenantSlugFromContext to context propagation
- Add TenantStore + Workspace to ResolverDeps
- Force effectiveRestrict() to always return true (multi-tenant security)
- Remove restrict_to_workspace from agentAllowedFields
UI cleanup:
- Remove custom-tools pages, types, routes, constants (backend removed in Plan 2)
- Clean tool-name-select component of custom tools references
* feat(security): session ctx propagation + execMapUpdate tenant guard (Plan 4)
Session store:
- Add ctx to AddMessage, SetSessionMetadata, SetAgentInfo, List, Save
- List now filters by tenant_id for non-cross-tenant callers
- Save uses ExecContext for cancellation support
- All ~15 callers updated to pass ctx
execMapUpdate tenant guard:
- Remove deleted_at IS NULL from execMapUpdateWhereTenant (only agents has soft-delete)
- Migrate 8 callers to execMapUpdateWhereTenant: agent_links, channel_instances,
mcp_servers, secure_cli, tracing, teams, skills_crud, cron_update
- Add ctx to UpdateSkill, UpdateJob interfaces + all callers
Deferred: cron scheduler global cache (correct by design — system process),
browser per-tenant isolation (separate plan).
* refactor(store): add context.Context to all SessionStore interface methods
Complete ctx propagation across all 24 SessionStore methods for:
- Future tenant-aware DB operations
- Request cancellation/timeout support
- Distributed tracing capability
Updated ~15 files including all callers in agent loop, gateway methods,
heartbeat ticker, tools, and CLI commands.
* fix(security): remove context.Background() shadowing in gateway handlers
Critical fix from code review: gateway agent handlers (create, update,
delete, identity, files, links, teams) were creating ctx := context.Background()
which shadowed the handler's ctx that carries tenant_id. This breaks
tenant-scoped agent queries for non-master tenants.
- Remove ctx shadowing in 7 agent handler files
- Add ctx param to resolveAgentUUID/resolveAgentInfo helpers
- Use store.WithCrossTenant in resolver (system-level operation)
* feat(security): tenant-scoped UNIQUE constraints for multi-tenant isolation
Update UNIQUE indexes to include tenant_id, allowing same names across tenants:
- agents: (agent_key) → (tenant_id, agent_key) WHERE deleted_at IS NULL
- sessions: (session_key) → (tenant_id, session_key)
- skills: (slug) → (tenant_id, slug)
- mcp_servers: (name) → (tenant_id, name)
- channel_contacts: (channel_type, sender_id) → (tenant_id, channel_type, sender_id)
Code changes:
- GetByKey now filters by tenant_id (same pattern as GetByID)
- ON CONFLICT clauses updated for sessions and skills
- Channel consumer uses WithCrossTenant for agent resolution
- Down migration restores original constraints
* fix(security): close remaining tenant isolation gaps from final audit
Critical fixes:
- gateway_setup: WithCrossTenant for default agent lookup at startup (C6)
- channel_contacts: ON CONFLICT updated to (tenant_id, channel_type, sender_id) (Q15)
- agents.Delete: tenant filter on DELETE (Q1)
High priority fixes:
- agents: List, GetDefault, ShareAgent, RevokeShare, ListShares, CanAccess,
ListAccessible, Update unset-default — all now tenant-scoped
- skills_crud: DeleteSkill now takes ctx, verifies tenant ownership
- mcp_servers, channel_instances, secure_cli: Delete methods tenant-scoped
- WithCrossTenant added to: gateway team notifications, team_tool_cache,
pending_messages GetDefault
* fix(migration): add tenant_id to usage_snapshots unique index
Update idx_usage_snapshots_unique to include tenant_id, preventing
cross-tenant upsert collisions when different tenants have agents
with same provider/model/channel combination.
* feat(security): cron tenant guard + browser per-tenant isolation
Phase 3 — Cron API tenant guard:
- Add ctx to 5 CronStore methods (GetJob, RemoveJob, EnableJob, RunJob, GetRunLog)
- All API-facing cron ops now filter by tenant_id (prevents cross-tenant CRUD)
- RemoveJob/EnableJob return "not found" on tenant mismatch (no enumeration)
- GetRunLog JOINs cron_jobs for tenant filtering
- UpdateJob internal reads scoped by tenant (defense-in-depth)
- Scheduler-internal methods (GetDueJobs, refreshJobCache) unchanged (system-level)
Phase 4 — Browser per-tenant isolation:
- Per-tenant incognito browser contexts via rod Incognito() (separate cookie jars)
- All page access (Snapshot, Screenshot, Navigate, Click, Type, etc.) validated
via getPageForTenant — blocks cross-tenant access by targetID
- OpenTab creates pages in tenant's incognito context
- ListTabs scoped to tenant's incognito context
- ConsoleMessages validates page ownership
- Stop/reconnect properly cleans up incognito contexts
* feat(security): isolation gaps + per-tenant config (Plan 5)
Part A — Isolation Gap Fixes:
- Merge migration 028 into 027: add tenant_id to llm_providers +
config_secrets, fix UNIQUE constraints for paired_devices +
channel_instances
- providers.go: tenant filtering on all CRUD queries
- config_secrets.go: ON CONFLICT (key, tenant_id)
- pairing_store: add ctx to all 7 interface methods, remove hardcoded
MasterTenantID, update ~15 channel caller files
- Session cache: prefix keys with tenantID to prevent cross-tenant
collision. DB queries (loadFromDB, Save, Delete, LastUsedChannel)
add tenant filter
- config_permissions cache: prefix keys with tenantID
- Cron ListJobs: fail-closed when tenant context missing
Part B — Per-Tenant Configuration:
- Provider Registry: compound key tenantID/name with fallback to
master tenant. GetForTenant/ListForTenant/RegisterForTenant
- Resolver: uses tenant-aware provider lookup + disabled tools query
- Agent loop: filter disabled tools from LLM tool definitions
- Builtin tool tenant configs: store interface + PG implementation +
PUT/DELETE HTTP endpoints
- Skill tenant configs: store interface + PG + ListAccessible LEFT
JOIN to exclude disabled skills per tenant
- OAuth: DBTokenSource with tenantID field for tenant-scoped token
refresh
- All HTTP provider handlers use RegisterForTenant/UnregisterForTenant
* feat(security): channel tenant propagation + MCP per-user credentials (Plan 6)
- Propagate tenant_id from channel_instances through BaseChannel →
InboundMessage → agent loop context (fixes 5-point break in tenant flow)
- Inject tenant context in WS router dispatch for all gateway methods
- Add MCP per-user credential overrides (api_key, headers, env) with
AES-256-GCM encryption and HTTP API endpoints
- Rewrite MCP pool with tenant-scoped keys, slot semaphore, idle eviction,
and credential rotation support (Evict per tenant+server)
- Bypass pool for users with custom credentials (separate connections)
- Fix MCP APIKey never passed to connections (inject as Authorization header)
* fix(security): close remaining tenant isolation gaps from Plan 1-6 audit
- Add tenant_id to 6 missing tables: agent_context_files,
skill_agent_grants, mcp_agent_grants, team_tasks, spans,
embedding_cache (migration 027)
- Fix tid==uuid.Nil fallback to fail-closed (return error) in 8 update
methods: agent_links, teams, skills, channel_instances, secure_cli,
cron, mcp_servers, tracing
- Add tenant filter to bare DELETEs: DeleteLink, DeleteTeam
- Add tenant filter to queries: ListChildTraces, GetMonthlyAgentCost,
CountAgentGrantsByServer, ListAccessible (MCP), ReviewRequest,
ResolveGroupTitles, buildTraceWhere
- Fix missing tenant_id in INSERTs: CreateSkill, GrantToUser,
ReviewRequest grant INSERTs
- Add tenant filter to api_keys: List, Revoke, Delete
- Fix cron scanJob/RemoveJob/EnableJob fallthrough patterns
* fix(security): inject tenant context into channel handler entry points
Channel handlers used context.Background() which lost tenant context,
causing store operations to either fail-closed or default to master
tenant. Now all 10 handler entry points inject tenant from BaseChannel.
* fix(security): tenant filters for teams, tasks, skills (Plan 6b audit)
- Teams: add tenant filter to GetTeamForAgent, ListMembers,
ListIdleMembers, KnownUserIDs (JOIN agent_teams for tenant check)
- Teams: add tenant_id to GrantTeamAccess INSERT, tenant filter to
RevokeTeamAccess, ListTeamGrants, HasTeamAccess
- Team tasks: add tenant_id to CreateTask INSERT, fail-closed
UpdateTask, tenant filter on all 7 query/delete methods
- Skills: add tenant filter to RevokeFromAgent, ListAgentGrants
- Skills: add ctx param + tenant filter to ToggleSkill
- History: annotate context.Background() locations with TODOs for
future tenant injection (requires PendingHistory struct refactor)
* fix(security): add tenant_id to 4 missing team tables + fix INSERTs
Add tenant_id column to: agent_team_members, team_task_comments,
team_task_events, team_task_attachments (migration 027).
Fix INSERT statements to include tenant_id: AddMember,
AddTaskComment, RecordTaskEvent, AttachFileToTask.
* fix(migration): cast UUID literals in tenant_users seed + usage_snapshots index
PostgreSQL doesn't auto-cast string to UUID in SELECT and expression
index contexts. Add explicit ::uuid casts to prevent migration failure.
* docs: add multi-tenant architecture guide for integrators
Comprehensive solution doc covering auth model, WS protocol, event
system, data isolation, API reference, and integration patterns.
Target audience: developers building custom frontends or SaaS on GoClaw.
* feat(ui): multi-tenant awareness + tenant admin page (Plan 7)
Backend:
- Enrich WS connect response with tenant_name, tenant_slug, cross_tenant
- Add tenants.mine WS method (any user, returns own memberships)
- Parse tenant_hint in connect params for browser pairing multi-tenant
- Wire tenantStore to MethodRouter for connect-time tenant lookup
Frontend:
- Auth store: tenantId, tenantName, tenantSlug, isCrossTenant, availableTenants
- WS client: capture tenant fields from connect, send tenant_hint
- WS provider: auto-fetch tenants.mine on connect
- useTenants() shared hook for all tenant-aware components
- Tenant indicator in sidebar connection status
- Tenant admin page (/admin/tenants) with list + create dialog
- Tenants nav in sidebar (cross-tenant admin only)
- i18n: tenants namespace (en/vi/zh)
- Type updates: tenant_id on AgentData, ApiKeyData
* refactor(ui): move tenant selector into user menu dropdown in topbar
Replace simple logout button with a Radix Popover user menu showing:
- User ID display
- Tenant selector (when multi-tenant: list all tenants with check mark)
- Logout button
Remove tenant indicator from connection-status.tsx (now in topbar).
Tenant switch saves slug to localStorage and reloads for reconnect.
* feat(ui): add logout confirmation dialog
Show destructive confirm dialog before logout via ConfirmDialog
component. Added logoutConfirm i18n key for en/vi/zh.
* fix(ui): security hardening — hide admin nav, fix route guard, fix refresh
- Hide System nav group for non-admin roles in sidebar (was visible to all)
- Replace RequireAdmin with RequireCrossTenant guard on /admin/tenants route
- Add RequireCrossTenant component to require-role.tsx
- Fix refresh button animation: use isFetching instead of isLoading
- Clean up connection-status.tsx (remove tenant indicator, now in topbar)
* feat: cross-tenant admin tenant scope selector
Backend: add tenant_scope connect param. Cross-tenant clients can
narrow their scope to a specific tenant (slug). applyTenantScope()
sets client.tenantID and clears crossTenant flag.
UI: user menu shows "All Tenants" option for cross-tenant admins.
Selecting a tenant saves slug to localStorage as tenant_scope,
reload reconnects with narrowed scope. "All Tenants" clears scope.
* feat: provisioning API key scope + tenant detail page (Plan 8)
Backend:
- Add operator.provision scope for limited tenant management
- Add HasScope() method to gateway Client
- Allow provision-scoped keys to create tenants + add users
- Allow provision-scoped keys to create tenant-bound API keys
Frontend:
- Tenant detail page with user management (list, add, remove)
- Clickable tenant list rows navigate to detail
- i18n: tenant detail keys (en/vi/zh)
- Route /admin/tenants/:id with RequireCrossTenant guard
* fix: tenant scope keeps admin privileges + UI pattern fixes
Backend:
- applyTenantScope keeps crossTenant=true (retains admin features)
- Router: scoped cross-tenant injects WithTenantID (filters data)
while keeping admin role for method access
UI:
- Fix "All Tenants" check mark (compare against nil UUID string)
- Fix tenant label when scope active (show selected tenant name)
- Use ConfirmDialog for user removal (was hand-rolled)
- Add DialogDescription to add-user dialog (Radix a11y)
- Fix table min-w-[600px] consistency
- Fix column header mismatch (was "role", should be "created")
* fix(ui): clean up tenant detail header — remove redundant info panel
Remove duplicate slug/status/created panel. Info now shown in
PageHeader description (slug + date). Status badge removed (redundant
with description). Cleaner, consistent with other admin pages.
* fix(ui): redesign tenant detail with info cards + user cards
* feat(ui): tenant selection gate — require tenant before app access
- Add tenantSelected flag to auth store (persisted via localStorage)
- WS provider auto-selects: single-tenant user auto, cross-tenant
admin defaults to "All Tenants", zero-tenant user blocked
- RequireAuth gate: redirect to /select-tenant when connected but
no tenant selected
- New TenantSelectorPage: centered card layout matching login page,
"All Tenants" amber card for cross-tenant admin, per-tenant cards
with role badges, no-access state with logout button
- i18n: selectTenant, noAccess keys (en/vi/zh)
* fix(security): scope events for cross-tenant admin with tenant_scope
Event filter was checking !crossTenant before filtering — scoped
cross-tenant admins (crossTenant=true + tenantID set) bypassed
tenant event filtering. Now checks tenantID != Nil regardless of
crossTenant flag, ensuring scoped admins only see their chosen
tenant's events.
* fix(security): HTTP API now respects tenant_scope for gateway token
Root cause: UI uses HTTP API (/v1/agents, /v1/mcp/servers, etc.)
for data fetching. HTTP auth middleware with gateway token always
set CrossTenant=true with no tenant filtering. tenant_scope only
worked for WS connection, not HTTP requests.
Fix:
- HTTP client sends X-GoClaw-Tenant-Scope header from localStorage
- HTTP auth resolves header slug → tenant UUID via tenantStore
- requireAuth: CrossTenant + TenantID → WithTenantID (scoped)
- Wire InitTenantStore(pgStores.Tenants) in gateway startup
* feat(security): tenant-aware provider registry, event filter, and membership validation
- Refactor providers.Registry: Get(ctx, name) / List(ctx) extract tenant
from context via injected TenantFromCtx func (avoids circular import)
- Event filter: fail-closed 3-mode tenant filtering
Mode 1: unscoped admin sees all
Mode 2: scoped admin sees tenant events + system events
Mode 3: regular user sees only own tenant (fail-closed)
- WS connect: resolveTenantHint validates membership via GetUserRole
with PermissionCache (30s TTL, bus invalidation)
- BroadcastForTenant helper for tenant-scoped event emission
- Session list: add TenantID to SessionListOpts from context
- Cron handleRun: preserve tenant in background goroutine context
- GOCLAW_LOG_LEVEL env var (debug|info|warn|error) for Docker/K8s
- Cache debug logging: tenant_cache, permission_cache, api_key_cache
- Friendly verify error: timeout → user-readable message
- Verify timeout: 15s → 30s
* feat(ui): setup wizard improvements + agent preset enrichment
- Setup: skip link with confirm dialog, language selector (en/vi/zh)
- Setup: card padding fix (py-0 gap-0 on Card, py-5 on CardContent)
- Setup: remove duplicate skip link from layout
- Step Model: verify countdown timer (30s), stops on result
- Step Agent: default Fox Spirit preset, selected state styling,
hide agent key/name inputs, auto-derive from preset, emoji in config
- Summoning modal: elapsed timer (m:ss format)
- Agent presets: enriched prompts with human-like quirks
Fox Spirit: playful personality, care reminders
Artisan: portrait/banner/ads/logo expertise
Astrologer: reference sites (astro.com, cafeastrology, labyrinthos)
- i18n: "triệu hồi linh hồn" fix, all 3 locales updated
* feat(ui): API Key tenant support + card layout + provider chain fix
- API Key create: tenant selector for cross-tenant admin, provision scope
- API Key create: redesigned dialog with scope cards, Radix Select, icons
- API Key list: card layout with badges (status, tenant, scopes)
- API Key: shortcut in user menu (topbar)
- API Key: keep "API Key" untranslated across all locales
- Provider chain: empty state fix — skip legacy entry when provider
not found in current tenant
- i18n: form.cancel key added to all 3 locales
* fix(ui): add bottom padding to all page layouts + misc improvements
- Add pb-10 to all 24 page containers to prevent content touching
bottom edge of viewport
- Various UI polish from user modifications (summoning colors,
layout icon, agent cards, sidebar adjustments)
* feat(ui): MCP user credentials dialog + builtin tool tenant toggle
- MCP: per-user credentials dialog (api_key, headers, env KV editor)
with status badges, delete all, save
- MCP: "My Credentials" button on each server row
- Builtin Tools: per-tenant enable/disable override toggle
with "Using default" / "Enabled/Disabled for tenant" badges
and reset-to-default button
- Setup: larger logo (h-16) and bolder title (text-4xl font-bold)
- i18n: all keys added to en/vi/zh for both features
* fix(ui): API key card spacing + remove pagination border
- Card padding: px-4 py-3.5 (was px-3 py-2), rows spaced with gap-2
- Scopes on separate row from dates for readability
- Card gap: space-y-2.5 between cards
- Pagination: add className prop, remove border-t on API keys page
- Badge/icon sizes bumped to text-xs / h-3.5 (was text-[10px] / h-3)
* fix(security): comprehensive tenant isolation audit — SQL, events, cache, skills, files
Defense-in-depth hardening across 12 audit phases:
- SQL: add tenant_id WHERE to teams_tasks lifecycle/activity/followup/progress/embedding (~30 functions)
- Events: broadcastTeamEvent + task_ticker + subagent announce now carry TenantID
- Cache: agentKeyCache scoped by tenant (agent keys per-tenant, not globally unique)
- Skills: SkillStore interface accepts ctx, SQL filter (is_system OR tenant_id=$N), per-tenant list cache, GrantToAgent includes tenant_id, tenant-scoped file storage
- Files: StorageHandler/FilesHandler/TeamAttachments/teamWorkspaceDir use config.TenantDataDir/TenantTeamDir
- Security: HMAC signed file tokens (file_token.go) replace gateway token in URLs
- Audit: AuditEventPayload carries TenantID for async subscriber tenant scoping
- InboundMessage: subagent/dispatch/validation/session_send propagate TenantID
- Pending messages: DeleteStale scoped by tenant
* fix(security): skip gateway token in URLs with signed file tokens
toFileUrl() now skips appending ?token=GATEWAY_TOKEN when the URL
already contains ?ft= (HMAC signed file token). Prevents gateway
token exposure via browser history, logs, and referrer headers.
* fix(security): stop persisting auth tokens in session media URLs
mediaToMarkdown() now stores clean paths (/v1/files/path) without
any auth tokens. Previously embedded ?token=GATEWAY_TOKEN (or ?ft=)
into markdown which gets persisted in session messages DB.
Frontend toFileUrl() adds auth at render time — tokens never stored.
* fix(security): migration 027 strips leaked gateway tokens from session URLs
Adds cleanup step to tenant foundation migration: removes ?token=xxx
from persisted media URLs in session messages. Old code embedded the
gateway token; new code stores clean paths only.
* fix(security): sign file URLs at delivery time, not persist time
Add SignFileURLs() utility that finds /v1/files/ and /v1/media/ URLs
in content and appends HMAC signed ?ft= tokens before delivery.
Applied at 4 delivery points:
- WS agent events (OnEvent callback in gateway_managed.go)
- WS chat.history response
- WS sessions.preview response
- HTTP /v1/chat/completions response
Sessions store clean paths only. Tokens are generated per-delivery
with 1h TTL — never persisted in DB. Frontend toFileUrl() skips
appending gateway token when ?ft= is already present.
* fix: file token verify path must match signed path (/v1/files/ prefix)
SignFileURLs() signs the full URL path "/v1/files/{path}" but the
verify in files.go auth() was using "/{path}" (without prefix).
HMAC mismatch caused all signed file tokens to return 401.
* fix(security): scope storage size cache per-tenant
sizeCache was a single global entry — all tenants shared one cached
size. Changed to sync.Map keyed by tenantBaseDir so each tenant gets
its own cached size calculation.
* feat(ui): redesign API keys page — table layout + code snippet dialog
Replace card-based API keys list with table layout matching MCP Servers
pattern. Add "API Key Usage" dialog with tabbed code snippets (cURL,
TypeScript, Go) showing gateway connection examples with syntax
highlighting and copy-to-clipboard.
* fix(builtin-tools): seed media tools disabled, fix tenant toggle, add unconfigured warning
- Seed media tools with Enabled=false and no default provider settings
(user must configure provider chain before enabling)
- Fix provider chain form ghost entries: validate provider exists in
tenant before showing (parseInitialEntries new-format path)
- Fix double toggle: show only tenant override OR global toggle, not both
- Fix list API: merge tenant_enabled from builtin_tool_tenant_configs
into response when tenant-scoped (was always null)
- Add ListAll() to BuiltinToolTenantConfigStore for full override map
- Add amber warning banner for enabled media tools missing provider config
* feat(mcp): require_user_credentials setting + KeyValueEditor for user creds
- Add require_user_credentials setting in mcp_servers.settings JSONB
- Backend: skip MCP server in LoadForAgent when user lacks credentials
- Frontend: toggle in MCP form dialog, persisted in settings field
- Redesign MCP user credentials dialog: replace raw Textarea with
KeyValueEditor (sensitive key masking for auth/token/secret fields)
- Add settings to mcpServerAllowedFields for HTTP update
* fix(security): restrict cross-tenant to owner IDs, config to owners only
- Gateway token + non-owner user ID: admin role but tenant-scoped
(no cross-tenant access). Fallback: only "system" is owner when
GOCLAW_OWNER_IDS not configured (fail-closed).
- Config page (WS config.* methods): wrapped with requireCrossTenant
middleware — non-owner admins get permission denied
- Config sidebar link: hidden for non-cross-tenant users
- Logout: clear tenant_id and tenant_hint from localStorage
(prevents tenant scope leak to next user session)
- Refactor: LOCAL_STORAGE_KEYS.TENANT_ID/TENANT_HINT constants
* fix(ui): chat bubble contrast, login logo, tenant no-access UX
- Chat bubble: use --chat-bubble-user CSS var (darker orange, L=0.50/0.52)
with text-white for WCAG AA contrast (~5.5:1)
- Login page: logo h-20 w-20, title text-3xl font-bold
- Tenant selector no-access: shield icon + hint text explaining
user needs admin to add them to a tenant
- Sidebar: GoClaw text uses text-sidebar-primary (brand color)
* feat(contacts): merge/unmerge contacts to tenant users
Add API and UI for linking channel contacts to tenant_users identity,
enabling cross-channel user identification within a tenant.
Backend:
- POST /v1/contacts/merge — link contacts to existing or new tenant_user
- POST /v1/contacts/unmerge — remove merged_id from contacts
- GET /v1/contacts/merged/{id} — list contacts by tenant_user
- GET /v1/tenant-users — list users for current tenant
- Add display_name + metadata columns to tenant_users (migration 27)
- All endpoints enforce tenant isolation via context tenant_id
Frontend:
- Checkbox multi-select on contacts table
- Selection toolbar with Merge/Unmerge buttons
- Merge dialog: link to existing user or create new
- Link2 icon indicator for merged contacts
- i18n: en/vi/zh translations for merge section
* fix(security): add tenant_id to span and embedding_cache inserts
SpanData struct was missing TenantID field — all span inserts failed
with NOT NULL constraint violation after migration 027 dropped defaults.
Fix captures tenant_id from context at emit time (6 call sites in
loop_tracing.go + subagent_tracing.go), then includes it in both
CreateSpan() and BatchCreateSpans() SQL (25→26 columns).
Also fixes embedding_cache writeEmbeddingCache() which was missing
tenant_id in its batch INSERT — same class of bug.
Both use MasterTenantID fallback for backward compatibility.
* feat: Introduce tenant switcher UI and enhance multi-tenant architecture documentation.
* fix(security): enforce tenant scoping, fix session isolation and UI cleanup
- Force cross-tenant admins to always have a concrete tenant_id (default
MasterTenantID) instead of unscoped WithCrossTenant — prevents mismatch
between session listing (no filter) and writes (MasterTenantID fallback)
- Make agent router tenant-aware: Get(ctx, agentID) resolves agent for
the caller's tenant, preventing cross-tenant agent cache collisions
- Fix context.Background() in title goroutine and summarization — now
uses tenant-aware context (WithoutCancel) so titles and compaction
persist to the correct tenant
- Add read-only SessionStore.Get() method; replace GetOrCreate in auth
checks (preview/patch/delete/reset) to prevent phantom session creation
- Inject tenant from channel instance into inbound message processing
- Remove "All Tenants" option from tenant selector, topbar switcher,
and ws-provider auto-select — admin must always operate within a tenant
- Fix contacts page selection toolbar layout shift (always rendered)
- Widen MCP credentials sensitive header regex to catch API_KEY etc.
* fix(security): propagate tenant_id in consumer handlers and background ops
- InjectTeamDispatch: use context.WithoutCancel instead of context.Background
to preserve tenant_id while avoiding cancel propagation from HTTP/WS handlers
- handleTeammateMessage/handleSubagentAnnounce: inject tenant_id from msg
- Add nil guard for outcome.Result to prevent panic on agent-not-found
- Use BroadcastForTenant for EventTeamTaskFailed/Completed/LeaderProcessing
- Remove unnecessary WithCrossTenant in autoSetFollowup (ctx already scoped)
- resolveAgentByKey: accept ctx param for tenant-scoped agent lookup
- pending_messages: use request ctx instead of cross-tenant for GetDefault
* fix(security): tenant-scope EnsureContact and PendingHistory DB operations
- All channel EnsureContact calls now use tenant-scoped ctx instead of
context.Background (whatsapp, slack, discord, telegram, feishu, zalo)
- PendingHistory: add tenantID field, thread through constructors
- All PendingHistory DB ops (load, flush, compact, delete) use tenantCtx()
- Normalize timeouts: 10s for simple queries, 15s for batch writes
* feat(teams): auto-attach media, retry completed tasks, improve tool messages
- Auto-attach workspace media from any tool (create_image/audio/video) to
team tasks via loop-level hook, not just write_file interceptor
- Store absolute paths in team_task_attachments instead of relative
- Extend retry action to support completed tasks (reopen for follow-up)
- Context-aware comment result messages with next-action guidance for
leader vs member roles and task status
- All tool results include task_id for agent follow-up actions
- Use #N "subject" format instead of raw UUIDs in tool messages
* feat(multi-tenant): tenant isolation for media, events, providers and UI
- Tenant-scoped media store, event filter, provider registry
- Tenant header propagation in WS/HTTP clients
- UI: tenant-aware chat messages, markdown renderer improvements
- Protocol: tenant error codes and event definitions
* docs: add multi-tenant architecture documentation
* fix(teams): store absolute paths in team_task_attachments
- AutoAttachWorkspaceFile: use cleanPath consistently instead of raw absPath
- executeAttach: resolve relative paths to absolute via team workspace
- AfterWrite interceptor already uses filepath.Clean (verified)
* fix(teams): attachment download handles both absolute and relative paths
filepath.Join with an absolute att.Path discards the teamBase prefix,
causing path traversal check to fail and download to serve wrong file.
Now checks IsAbs first — uses path directly for new absolute entries,
falls back to legacy join for old relative entries.
* fix(teams): attachment download validates against workspace root not tenant dir
Absolute paths stored in DB don't match TenantTeamDir structure
(master tenant has no tenants/ prefix). Now validates absolute paths
against dataDir (workspace root) instead. Legacy relative paths still
resolve via TenantTeamDir as before. IDOR check on att.TeamID ensures
cross-team isolation.
* fix(teams): attachment download uses workspace root, not data dir
Files are stored under GOCLAW_WORKSPACE/teams/ but handler was passed
dataDir (GOCLAW_DATA_DIR) — completely different directory. Now passes
workspace. Legacy relative paths resolve via {workspace}/teams/{teamID}/{chatID}/{path}.
* fix(security): use HMAC-signed file tokens for attachment downloads
Replace gateway token exposure (?token=) with HMAC-signed short-lived
file tokens (?ft=) for team task attachment downloads — same mechanism
used by chat file URLs.
Backend:
- team_attachments auth: accept ?ft= signed token (priority 1), Bearer (priority 2)
- teams_tasks RPC: sign download_url with HMAC at delivery time
- Add fileTokenSecret to TeamsMethods, thread through wireChannelRPCMethods
Frontend:
- Use server-signed download_url from attachment data instead of ?token=
- Remove useAuthStore dependency from task-detail-dialog
* fix(security): decouple file token signing from gateway token
- Generate random 256-bit HMAC key at startup (crypto/rand, memory-only)
- All file signing/verification uses FileSigningKey() instead of gateway token
- Remove ?token= query param fallback from /v1/files/, /v1/media/, attachments
- Only ?ft= signed tokens and Bearer header accepted for file access
- Reduce file token TTL from 1h to 5min
- Frontend: remove gateway token from all file URLs and imports
- Note: tokens invalidate on restart (acceptable for 5min TTL + WS reconnect)
* fix(ui): use signed download_url for task attachments
- Add download_url to TeamTaskAttachment type
- Use a.download_url (server-signed ?ft=) instead of bare URL
* fix(security): tenant-scope team workspace paths + show user/tenant in topbar
- WorkspaceDir callers now use config.TenantWorkspace() to resolve
tenant-scoped base dir (non-master tenants get workspace/tenants/{slug}/)
- Fixes: all tenants previously wrote to global /app/workspace/teams/
without filesystem-level isolation
- Affected: loop.go (agent run), team_tasks_mutations.go (task creation)
- teams_workspace.go already correct (uses TenantTeamDir)
- UI topbar: show "userId (tenantName)" in user menu
* feat(ui): redesign task detail dialog with improved UX
- Split monolithic 343-line component into 5 focused files
- New header: subject as title, identifier + status badges above
- Metadata grid with soft bg-muted/30 background, priority icons
- Attachments: card-style with mime-type icons + proper download Button
- Description/Result: markdown rendering via MarkdownRenderer
- Comments: avatar circles + markdown rendering for content
- All sections collapsible with chevron + count badge
- Timeline: vertical dot-line pattern, collapsed by default
- Fix kanban card hover layout shift (opacity instead of display toggle)
* fix(security): tenant-scoped workspace paths and tool cache isolation
- Scope team workspace paths to tenant directory
- Add tenant isolation to tool cache and task reads
- Shell deny pattern improvements
- Agent resolver and context file tenant scoping
- Sidebar tenant/user display fix
- Add tests for workspace, boundary, and context file interceptor
* fix(ui): tenant visibility fallback, merge coming-soon, task detail tweaks
- Tenants page: try tenants.list (owner), fall back to tenants.mine
for regular users; hide create button for non-owners
- Merge contacts dialog: add coming-soon banner (i18n en/vi/zh),
disable form and submit button
- Task detail: collapse attachments/comments by default,
guard download_url before rendering
|
||
|
Viet Tran
|
84650c5c14 |
feat(teams): attachments refactor, semantic search, improved prompting (#310)
* feat(teams): refactor attachments, remove team_message, add task comments UI Major team system refactoring: - Drop team_workspace_files, team_workspace_file_versions, team_workspace_comments, team_messages tables; replace team_task_attachments with path-based schema - Add denormalized comment_count/attachment_count on team_tasks for dashboard perf - Auto-track file writes as task attachments via WorkspaceInterceptor - Remove team_message tool entirely (tool, store, i18n, builtin_tools, MCP bridge) - Members communicate via task comments; approve/reject use comments for audit trail - Add commented/new_task notification types to TeamNotifyConfig - Enrich task completion announce with member comments - User-created tasks stay pending (backlog) — no auto-assign to leader - Configurable member request tasks (member_requests.enabled in team settings) - Structured task description template in TEAM.md for v2 leads - HTTP attachment download endpoint with IDOR + path traversal protection - Web UI: count badges on task list, comments section with input, download button - Team settings UI: completed/commented/new_task toggles, member requests section * feat(teams): priority dispatch, compact prompting, realtime comments - Priority dispatch: DispatchUnblockedTasks dispatches only 1 task per owner per round (highest priority first). Fixes cancel bug where CancelSession killed innocent queued tasks. - Prompt rework: Replace verbose Task Decomposition (25 lines) with compact Task Planning (8 lines). Add explicit UUID warning and sequencing guidance for weak models (Qwen, MiniMax). - Recent comments in dispatch: buildRecentCommentsSummary appends 3 most recent comments to re-dispatched tasks (reject, retry, stale). - Enrich comment event payload with TaskNumber, Subject, CommentText (truncated 500 runes, UTF-8 safe). - UI: Board subscribes to TEAM_TASK_COMMENTED for realtime comment_count badge updates. Task detail dialog auto-refreshes comments on event. - Tool description hint: guide models to write self-contained task descriptions with clear objectives and context. * perf(teams): add ListRecentTaskComments with SQL LIMIT Dispatch only needs 3 most recent comments — avoid fetching all. New ListRecentTaskComments(ctx, taskID, limit) uses ORDER BY DESC LIMIT N then reverses to chronological order. * feat(teams): add subject embedding for semantic task search + improve prompting - Add vector(1536) embedding column to team_tasks with HNSW index - Implement hybrid search: FTS (0.3) + cosine similarity (0.7) with graceful fallback - Auto-generate embeddings on task create/update, backfill existing tasks on startup - Wire embedding provider into PGTeamStore via gateway_setup - Change FTS from OR to AND with prefix matching for precise keyword search - Reduce search page size from 30 to 5 to save tokens - Rename migration 000023 → 000024, bump RequiredSchemaVersion to 24 - Update TEAM.md hints: prefer search over list, batch task creation with blocked_by - Add anti-pattern examples to prevent sequential task creation |
||
|
Duc Nguyenandviettranx
|
2cc9d68cdc |
fix(tts): config save, Edge provider, media dispatch + dark mode chat (#265)
* fix(tts): config save + Edge provider registration + dark mode chat bubbles - Wrap TTS config payload in `raw` field for config.patch RPC (#229) - Always register Edge TTS provider (free, no API key) instead of gating on `enabled` flag - Fix low-contrast user message bubbles in dark mode chat * fix(tts): skip duplicate media dispatch when temp file already delivered When both the agent loop and the message tool dispatch the same TTS temp file, the first dispatch succeeds and cleanup deletes it. Filter out missing temp media files before sending to prevent "file not found" errors and spurious error notifications on Telegram/Slack/Discord. * feat(tts): include edge-tts in Docker image when Python enabled Edge TTS is free (no API key) and serves as a universal TTS fallback. Install it alongside Python in both ENABLE_PYTHON and ENABLE_FULL_SKILLS builds. * chore(docker): expose build args from .env for compose builds Pass ENABLE_OTEL, ENABLE_PYTHON, ENABLE_FULL_SKILLS as env-driven build args so .env can control Docker build features without editing docker-compose.yml directly. * fix(tts): hot-reload TTS config on settings change via pub/sub TTS providers were only registered at startup, so changing provider/API key via the Web UI had no effect until container restart. Add a tts-config-reload bus subscriber that rebuilds the TTS manager on config changes, matching the pattern used by quota, cron, and web_fetch. Always create a TtsTool at startup (even without providers) so the reload subscriber can populate it when settings are first configured. * fix(tts): protect TtsTool.UpdateManager with RWMutex to prevent data race UpdateManager() can be called from the config reload goroutine while Execute() reads t.manager concurrently from agent goroutines. Add sync.RWMutex following the same pattern as WebFetchTool.UpdatePolicy(). Also update setupTTS doc comment which incorrectly stated it could return nil — Edge TTS is now always registered. --------- Co-authored-by: viettranx <viettranx@gmail.com> |
||
|
viettranx
|
b2a74ba487 |
feat(skills): skill_manage tool + skill_evolve learning loop (#218)
Adds skill_manage — a first-class agent tool for creating, updating, and deleting skills from within a conversation — paired with per-agent skill_evolve that nudges predefined agents to capture reusable workflows. Tool (skill_manage): - create: write skill from SKILL.md content string (auto-grant, dep scan) - patch: find/replace producing new immutable version (advisory-locked) - delete: soft-delete (archive in DB, move to .trash/) - Security guard: 25 regex patterns block shell injection, credential exfil, path traversal, SQL injection, privilege escalation - Ownership enforced: only skill owner can patch/delete (admin bypass) - Content size limit: 100KB; companion file copy: 20MB, symlink-safe (WalkDir) - Enabled: false by default — admin opt-in per agent Learning loop (skill_evolve, predefined agents only): - System prompt: SHOULD/SHOULD NOT guidance for skill creation - Budget nudges: [System] prefix at 70%/90% iteration budget (ephemeral, i18n) - Postscript: once-per-run suggestion with explicit user consent - Config: other_config.skill_evolve + skill_nudge_interval (default 15) Security hardening (pre-existing + new): - CreateSkillManaged: RETURNING id + pg_advisory_xact_lock (atomic upsert) - GetNextVersionLocked: advisory lock for race-safe patch versioning - Ownership checks on HTTP update/delete, gateway update, 4 grant/revoke handlers - copyOtherFiles: filepath.WalkDir for real symlink detection UI: Skill Learning toggle + nudge interval in Agent General Tab i18n: backend (en/vi/zh catalogs) + frontend (en/vi/zh locale files) |
||
|
viettranx
|
e138ac7676 |
fix(teams): validate blocked_by terminal state + improve leader orchestration prompt
- Add terminal-state check in executeCreate(): reject blocked_by referencing completed/cancelled/failed tasks with actionable error - Add full validation in executeUpdate(): batch query via GetTasksByIDs, check existence + team membership + terminal state - Add GetTasksByIDs batch query to TeamStore interface + pg implementation - Refactor: modularize gateway, skills store, and team tools into focused files - Update TEAM.md leader prompt: prefer delegation, plan full task graph upfront, create tasks in order with blocked_by UUIDs |