mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-12 20:15:42 +00:00
dev
87
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 yatulandClaude Opus 5
|
e1674b0249 |
fix(teams): dispatch team tasks created by detached child runs (#1528)
Team tasks are not dispatched inline: the turn's PendingTeamDispatch collects them and the post-turn drain assigns and dispatches them once the turn ends. That same tracker also owns the per-(team, chat) create lock taken by team_tasks list/search. Async delegations and async spawns keep the caller's context values (context.WithoutCancel) but run detached — after the caller's turn has already ended and drained. A team lead reached through delegate therefore adds every task it creates to a tracker nobody will drain again, and takes a create lock nobody will release. Two symptoms follow: - tasks stay pending forever and are never dispatched, so the team never starts work. The task ticker does not cover this: it never dispatches pending tasks, it only marks them stale after 2h and asks the lead to retry by hand. - the next team_tasks list/search for the same (team, chat) blocks on the never-released mutex for the remaining lifetime of the process. Give each detached child run its own tracker through the existing InjectTeamDispatch helper, drained when that run ends. The synchronous delegate/spawn paths are deliberately left alone: they execute inside the caller's turn, where the caller's tracker is the correct owner. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Duc Nguyenandntduc
|
bb7712a9ff |
fix(collaboration): harden delegated task isolation (#1486)
* feat(collaboration): isolate delegated artifacts and child runs Isolate delegated inputs and outputs behind secure artifact exchange lifecycles. Scope Agent Link tasks by tenant and root agent, and enforce delegation spawn-tree boundaries. Add process-wide child-run admission and preserve logical media paths across native, MCP, and sandbox execution. * fix(collaboration): harden delegated task isolation Enforce tenant and root-agent task scope across migrations and stores. Add exactly-once async completion delivery, delegated sandbox boundaries, and confined artifact and media recovery across runtime surfaces. * fix(collaboration): recover interrupted async tasks * fix(collaboration): normalize persisted child-run status --------- Co-authored-by: ntduc <ntduc@cpp.ai.vn> |
||
|
Zezae Oh
|
505b7a9d99 |
fix(agent): recover interrupted runs to failed on gateway startup (#1272)
A run records its terminal run.status (completed/failed/cancelled) only when the agent loop returns. If the gateway process is killed mid-run (restart, crash, or SIGKILL after a slow graceful shutdown), the terminal event is never emitted, so the run's last run.status item stays "started" forever — it shows as perpetually running in the timeline and is never counted as failed. Mirror the cron scheduler's startup reset (recomputeStaleJobs flips stale 'running' jobs to 'interrupted'): add RunTimelineStore.RecoverInterruptedRuns, invoked once on gateway startup. It finds runs that have a started run.status but no terminal sibling and appends a terminal failed run.status item, marked interrupted in metadata so it stays distinguishable from a genuine agent failure. Running only at startup means nothing from before is still executing, so there is no false-positive risk — unlike a periodic sweep (see the deliberately-disabled tracing stale-trace loop). - store: RecoverInterruptedRuns on the interface + PG and SQLite impls - cmd/gateway_managed: invoke once after the timeline recorder is built - tests: SQLite store test — orphan recovered, completed run untouched, idempotent on re-run |
||
|
thotam
|
0ae55991bb |
feat(mcp): MCP OAuth 2.1 client for tool servers (#1196)
* feat(mcp): MCP OAuth 2.1 client — full implementation with tests
Implements a complete MCP OAuth 2.1 authorization flow for tool servers that
require user-delegated access, covering all layers from DB to UI.
- discovery.go: RFC 9728 protected-resource → RFC 8414 AS metadata → OIDC
fallback chain with 5-min in-memory cache and InvalidateCache()
- dcr.go: RFC 7591 Dynamic Client Registration with response size guard
- flow.go: PKCE (S256) authorization code flow — StartFlow(), ExchangeCode(),
ClientCredentials(), auto-cleanup of expired flows; carries AS issuer through
PendingFlow for status display
- refresher.go: OAuthTokenProvider with in-memory token cache, automatic refresh
on expiry, per-user vs global slot isolation, InvalidateCache/InvalidateServer
- migrations/000074 + SQLite schema: mcp_oauth_tokens with AES-256-GCM encrypted
access/refresh tokens, partial unique index for global vs per-user rows,
ON DELETE CASCADE from mcp_servers
- store.MCPOAuthTokenStore: Upsert, Get/GetUser, Delete/DeleteUser, and
DeleteServerOAuthTokens (purge all rows for a server)
- PostgreSQL + SQLite implementations
- POST /v1/mcp/oauth/start — discovery + optional DCR + PKCE redirect URL;
client_credentials completes server-side (no redirect) and returns completed=true
- GET /v1/mcp/oauth/callback — exchange code, persist token, publish WS event;
payload built via json.Marshal (no reflected XSS via error_description)
- GET /v1/mcp/oauth/status/{id}, DELETE /v1/mcp/oauth/token/{id} — admin-gated
- POST /v1/mcp/oauth/discover/{id} — on-demand discovery probe
- All outbound calls go through the SSRF-safe client with pinned IPs
- pkg/protocol/mcp_events.go: EventMCPOAuthComplete routed only to the initiating
user (admins in-tenant included); fail-closed across tenants
- getUserMCPTools() injects Authorization: Bearer from OAuthTokenProvider; on a
401 for OAuth servers it purges the cached token so the next turn re-resolves
- handleUpdateServer purges all OAuth tokens (global + per-user), drops the
refresher cache, and evicts the pool when a server's URL or OAuth config
(client_id / endpoints / grant_type / scope / auth_type) changes — so the
status UI and agent never use a token minted for the old resource/AS
- MCPOAuthDialog (WS-driven), unified user-credentials dialog, OAuth settings
fields; handles the no-redirect client_credentials completion
- internal/mcp/oauth/*_test.go: discovery cache, PKCE, DCR, refresher
- internal/http/mcp_oauth_test.go + mcp_update_oauth_purge_test.go: routes, auth
gating, WS event, purge-on-URL/OAuth-config-change
- tests/integration: store + encryption + tenant isolation, E2E start→callback,
DeleteServerOAuthTokens
- internal/gateway/event_filter_test.go, internal/agent/loop_mcp_user_test.go
* fix(mcp): return 400 on OAuth callback with code but missing state
The callback handler rendered a 200 HTML page whenever code or state was
absent. An auth code WITH a missing state is a malformed / CSRF-risk
callback (state is the CSRF token), so reject that case with HTTP 400.
A bare hit with neither code nor state (user opening the URL directly),
provider errors, and exchange failures keep their 200 HTML popup page.
Adds a status code parameter to writeCallbackHTML. Fixes the
TestOAuthCallbackMissingState integration regression while keeping
TestHandleCallbackMissingCodeAndState (no params -> 200) green.
* fix(mcp): scope-based OAuth auth + honor manual OAuth endpoints
Addresses the two MCP/OAuth security-review findings.
Finding 1 — authorization. mcp_oauth_tokens is tenant-scoped, but
start/status/revoke were gated only by requireAuth(RoleAdmin), an RBAC
role check, not tenant membership, so a RoleAdmin caller could act on a
tenant they don't administer. A blanket requireTenantAdmin would have
broken per-user self-service, which the UI exposes (the per-user
MCPUserCredentialsDialog shows an "Authorize" button to regular users for
their own credentials). Instead mirror the existing per-user MCP
credentials model (resolveTargetUserID in mcp_user_credentials.go):
- start/status/revoke accept any authenticated user; each handler calls
authorizeOAuthScope.
- a caller may manage their OWN per-user token (self-service); the
global/server token (user_id="") and other users' tokens require
tenant-admin (owner bypass), so a RoleAdmin that is not a tenant admin
is rejected.
- discover stays admin-only (it only previews AS metadata for a server).
Add a TenantStore dependency. Tests cover self-service, on-behalf-of-
another (403), and global-by-non-tenant-admin (403).
Finding 2 — honor manual OAuth config end-to-end. The UI sent use_dcr /
auth_endpoint / token_endpoint and the update path fingerprinted them for
purge, but handleStart always discovered + DCR'd and ignored them. Now:
- use_dcr=false (a *bool, so legacy/absent stays discover+DCR) skips
discovery/registration and uses the operator endpoints, SSRF-validated.
- token_endpoint is always required; auth_endpoint only for auth-code
grants — client_credentials needs no authorization URL, matching the UI
which hides that field for that grant.
- the refresher already refreshes against the stored token_endpoint and
the callback persists it, so manual-mode tokens refresh correctly.
- oauthFingerprint includes use_dcr (nil normalized to true) so toggling
DCR mode purges stale tokens.
- the web form only serializes manual endpoints when use_dcr is off.
Audited all MCP dialogs (form, global OAuth, per-user credentials, grants,
tools): OAuth dialogs handle completed/auth_url identically and read
config from stored server settings; runtime connect uses the stored token
via the refresher (no re-discovery).
Tests: manual auth-code + client_credentials endpoints, missing/SSRF
endpoints, and the full self/global/on-behalf authorization matrix.
|
||
|
Goon
|
06877365c4 | merge: sync dev into usage analytics branch | ||
|
Goon
|
4002bb4c13 | feat(usage): add event analytics dashboard | ||
|
Goon
|
f45bfa860c | feat(skills): add skill self-evolution metrics | ||
|
Duy /zuey/
|
269e2618ae |
feat: add archived run timeline
Squash merge PR #113 after resolving the project changelog conflict with current dev. Local checks passed: Go store/http/gateway/agent/pipeline tests, SQLite-tagged tests, both Go builds, web Vitest, and web build. PR CI run 26705098712 passed release-versioning, go, and web. |
||
|
Duy /zuey/
|
990fabf94b |
fix(config): persist shell deny group disables
Squash merge PR #112 after resolving the project changelog conflict with current dev. Local checks passed: config gateway tests, provider/http/tools deny-pattern tests, go build ./..., and go build -tags sqliteonly ./.... PR CI run 26704832350 passed release-versioning, go, and web. |
||
|
Duy /zuey/
|
9ab3d6dfe5 |
feat(tools): local-first document extraction for read_document
Squash merge PR #111 after resolving docs/changelog conflicts. Local checks covered tools/config and both Go builds; PR CI run 26704622503 passed release-versioning, go, and web. |
||
|
Goon
|
c99f7e1fa4 | feat(skills): add slash command activation | ||
|
Goon
|
d64a31ebdb | fix(usage): enforce caps on auxiliary llm calls | ||
|
Goon
|
9ff5a629d2 | feat(usage): add AI budget usage caps | ||
|
Duy /zuey/
|
56e227c403 |
feat(api): unblock goclaw-cli P6 trace follow and provider reconnect (#37)
* feat(api): unblock trace follow and provider reconnect * test(cron): stabilize run log execution test |
||
|
viettranx
|
085b89bf72 |
feat(agent): inject TTS [[tts]] directive guidance when auto mode is "tagged"
- Add TTSAutoMode field to LoopConfig, Loop, SystemPromptConfig - Wire from config.Tts.Auto through resolver to system prompt - Inject "## Voice Response" section explaining [[tts]] usage - Update AGENTS.md: voice only when user explicitly requests |
||
|
viettranx
|
8b8da3a3dd |
fix(mcp): unified grant revocation with per-agent tool group isolation
- Add GrantChecker interface with cache + event-bus invalidation for revalidating MCP tool grants at execution time (not just dispatch) - Move toolGroups from package-level global to per-Registry field; Clone() gives each agent Loop isolated tool groups, eliminating cross-agent race condition - BridgeTool.Execute rechecks grant validity before executing MCP methods; uses serverID field for cache key - Wire MCPGrantChecker through ResolverDeps to MCP Manager and LoopConfig - Add integration test for grant revocation scenarios - Update policy functions to accept *Registry parameter |
||
|
viettranx
|
33d188f01f |
feat(hooks/migration): auto-disable legacy command hooks on Standard boot
Phase 07 runtime migration. Post-Wave-1 Standard deployments that pre-dated the edition gate may still carry enabled command-type hook rows in the DB; those keep firing via the dispatcher after the UI stops letting users create new ones. This helper flips them off at startup so live traffic never fires a disabled-by-policy handler again. - internal/hooks/migration_command_autodisable.go: DisableLegacyCommandHooks(ctx, hookStore, edition). Standard only; Lite returns early. Master-scope context + List(Enabled=true) → skip non-command + source='builtin' rows → Update enabled=false. Per-row WARN log with hook_id + tenant_id for audit; INFO summary when n>0. Idempotent: second boot finds nothing. - cmd/gateway_managed.go: invoke after builtin.Seed, BEFORE handler registration so no HTTP/WS traffic races the migration. - Tests (fake store, 5 cases): * Standard with mixed rows → only command+source=ui gets disabled * http + script/builtin + command/builtin rows untouched * Idempotent second run → n=0 * Lite edition → n=0, nothing touched * nil store → n=0, no error * List error → propagated up Note: Phase 04/05 never seed command-typed builtins, but the carve-out is kept defensively so a future accidental builtin command row can't be disabled by this migration. |
||
|
viettranx
|
8ee540dc7e |
feat(hooks/builtin): embedded registry + UPSERT seed + store-layer readonly
Phase 04 of hooks Wave 1. Adds infrastructure for shipping canonical hook rows with the binary; Phase 05 plants the first real builtin (pii-redactor). - internal/hooks/builtin: loader (//go:embed yaml+js), Seed() with version reconciliation (newer embed overwrites DB; downgrade warn-only; preserves user's enabled toggle), AllowlistFor() for per-id mutable-field lookup. Stable UUIDv5 namespace 082ab084-a25f-52b4-a4a4-eb8a816bd9a8 keys rows across restarts so Seed is idempotent on every boot. - internal/hooks/store.go: ErrBuiltinReadOnly sentinel + WithSeedBypass ctx marker (seeder bypasses the guard; users cannot). - PG + SQLite hook stores: Update/Delete now reject any patch other than enabled=* on source='builtin' rows. Fail-closed on GetByID errors. - Dispatcher: SetBuiltinAllowlistLookup setter lets the gateway wire the registry-backed strict allowlist; unwired tests keep Phase 03 permissive default. Gateway installs a strip-all lookup first so a Load() failure fails closed instead of opening mutation to the permissive default. - i18n: MsgHookBuiltinReadOnly (en/vi/zh). - yaml.v3 promoted to direct dep. - Tests: loader parse + namespace stability + seed idempotency (3 boots → N rows), version bump, downgrade detection, operator BuiltinDisable escape hatch, PG + SQLite readonly guard round-trip. |
||
|
viettranx
|
b7592bcacf |
feat(hooks/script): integrate script handler — FireResult + source-tier gate + migration
Wave 1 Phase 03. Wires the Phase 02 Goja handler into the dispatcher, widens
the DB schema for script + builtin sources, and refactors Dispatcher.Fire to
return a FireResult so builtin-source hooks can mutate event input.
Dispatcher:
- FireResult{Decision, UpdatedToolInput, UpdatedRawInput} replaces the plain
Decision return. stdDispatcher.runSync keeps a local evMut copy so a
builtin-source hook's updatedInput flows to downstream hooks in the chain
and to the caller; non-builtin (source=ui) script mutations are dropped +
WARN-logged (defense-in-depth; source tier enforced at the dispatcher, not
only at the handler).
- applyBuiltinMutation + placeholder builtinAllowlistFor (rawInput, toolInput)
— Phase 04 overrides the allowlist from builtins.yaml.
- noopDispatcher returns FireResult{DecisionAllow}.
- Script mutation tests: TestDispatcher_ScriptMutation_BuiltinSourceApplies,
TestDispatcher_ScriptMutation_UISourceDenied.
Pipeline + callers (14 Fire sites refactored):
- FireHook wrapper returns FireResult; context_stage.go:52 applies
UpdatedRawInput → state.Input.Message; tool_stage.go:52 applies
UpdatedToolInput → tc.Arguments before ExecuteToolCall.
- delegate_bridge + delegate_tool keep the Decision branch; Updated* ignored.
- dispatcher_test / delegate_bridge_test / delegate_tool_hooks_test /
integration test helpers updated for the new shape.
Validation:
- edition_gate allows HandlerScript on every edition (sandboxed, no shell
escape surface).
- config.validateHandler rejects empty source, source > 32 KiB, goja compile
error; validateTimeout rejects on_timeout=ask|defer (reserved).
- Six new config tests cover the script path.
Migrations:
- PG migration 000053 relaxes handler_type + source CHECKs and drops
uq_hooks_{global,tenant,agent} — scripts routinely want many small hooks
per event. RequiredSchemaVersion → 53.
- SQLite SchemaVersion → 21; patch 20 is a SELECT 1 placeholder because
SQLite can't ALTER a CHECK — rebuildAgentHooksV21 runs outside the
migration tx (parallel to backfillV16) to rename/recreate the table with
widened CHECKs. schema.sql fresh-DB path matches.
- H9 fix: PGHookStore.Create + SqliteHookStore.Create honor caller-provided
cfg.ID (uuid.Nil falls back to UUIDv7), unblocking Phase 04 idempotent
UUIDv5 seed. TestCreateHonorsFixedID on both stores.
Config:
- config.HooksConfig{ScriptConcurrency, ScriptPerTenantConcurrency,
ScriptCacheSize, BuiltinDisable}; buildHookHandlers wires the script
handler with the caps from appCfg.Hooks.
Gates green: go build ./... + sqliteonly, go test -race -count=1 across
hooks/pipeline/sqlitestore.
|
||
|
viettranx
|
aa1b2f026a |
feat(hooks): WS RPC surface + tenant-scope guards + shared test runner
Exposes agent hooks over WebSocket (hooks.list/create/update/delete/ toggle/test/history) behind existing auth layer. Additions: - Gateway wiring: buildHookHandlers() factory reused by dispatcher and hooks.test runner, so UI test panel exercises production code paths - GetByID tenant-scope guard: non-master callers only see own tenant + global rows (matches List() behavior on both PG and SQLite) - i18n keys + catalogs (en/vi/zh) for hooks error messages - Protocol method constants for hooks.* and config.defaults |
||
|
viettranx
|
fb3552f7e5 |
fix(hooks/security): add SSRF-safe http client missed by phase 2 commit
Phase 2 (a587231b) advertised SSRF hardening for the http hook handler but the supporting `internal/security` package was never created, so the production HTTPHandler fell back to a bare http.Client and admin-config webhooks could probe loopback / link-local / RFC1918 / cloud-metadata. - internal/security/ssrf.go: Validate(rawURL) parses + resolves once, rejects loopback/link-local/private/multicast/unspecified + 169.254.169.254; NewSafeClient(timeout) returns an http.Client whose DialContext pins the resolved IP from context (defense-in-depth re-checks the dialed IP) and refuses redirects (CheckRedirect = ErrUseLastResponse) - internal/hooks/handlers/http_handler.go: call Validate before each request, attach pinned IP via security.WithPinnedIP(ctx, ip) - cmd/gateway_managed.go: construct HTTPHandler with Client: security.NewSafeClient(10*time.Second) - tests: 13 new ssrf_test.go cases (every block category + redirect refused + dial pinned); existing http_test.go retrofitted with security.SetAllowLoopbackForTest helper for httptest.NewServer - plan: phase-02 Step 2a marked done with reference to this commit Refs: GitHub Issue #875 |
||
|
viettranx
|
97f9784443 |
feat(hooks): phase 2 — handlers, pipeline wiring, ssrf-safe http client
- add command + http handlers (internal/hooks/handlers) with edition gating and Authorization-header decryption via crypto.Decrypt - add SSRF-safe dialer (internal/security/ssrf.go): DNS-once + pinned IP, blocks loopback / link-local / private ranges - wire dispatcher into pipeline stages: ContextStage fires SessionStart (async) + UserPromptSubmit (sync); ToolStage fires PreToolUse (sync, COW staging for updatedInput) + PostToolUse (async); FinalizeStage fires Stop (async) - bridge delegate events to dispatcher (SubagentStart / SubagentStop) - construct dispatcher in cmd/gateway_managed.go with both handlers and thread it through agent.ResolverDeps + delegateTool.SetHookDispatcher - unit tests for both handlers + integration tests covering HTTP allow with audit row, HTTP block, command Lite-only gate, delegate bridge subscribe - defer worker pool optimization (Phase 2 Step 3) to Phase 4 Refs: GitHub Issue #875 |
||
|
viettranx
|
4b658a2304 |
feat(tools): tenant-scoped allowed_paths configuration
- Add tenant-level filesystem path restrictions via system_configs table - Merge tenant paths with global skills directories in allowedWithTeamWorkspace() - Propagate tenant paths to subagents via RunContext - Seed allowed_paths from config.json to system_configs on startup - Fix TestStoreTask_RaceToClaimSameTask: use composite PK for team members |
||
|
viettranx
|
d77a3664db |
fix(cache): tenant-aware invalidation for builtin tools and skills
Tenant config changes for builtin tools and skills silently failed to invalidate cached agent Loops, leaving tenants stuck on stale tool/skill sets until the 10-minute TTL expired or an unrelated event wiped the cache. Master-level skill CRUD had the same gap in the opposite direction. - Add CacheInvalidatePayload.TenantID so events can scope to one tenant - Add Router.InvalidateTenant(tenantID) with prefix match on "tenantID:agentKey" cache keys; uuid.Nil is a no-op - Rework emitCacheInvalidate helpers in builtin_tools + skills handlers to carry tenant scope; add defense-in-depth uuid.Nil guards in four tenant-config handlers - Update TopicCacheBuiltinTools + TopicCacheSkills subscribers to branch on payload.TenantID (tenant event wipes that tenant, global event keeps the existing InvalidateAll path) - Wire emitCacheInvalidate into master skill CRUD paths (update, delete, toggle, upload, install-deps, rescan-deps, import) that previously only called BumpVersion - Document the system-owner bypass in requireTenantAdmin so handlers keep guarding uuid.Nil themselves |
||
|
viettranx
|
dabb1eaa11 |
feat(vault): workspace rescan endpoint with symlink-safe walker
Add POST /v1/agents/{agentID}/vault/rescan to backfill vault_documents
from filesystem. Walks agent workspace, registers missing/changed files,
publishes enrichment events for async summarize + embed + link classify.
Backend:
- SafeWalkWorkspace: symlink-safe walker with exclusion patterns,
resource limits (5K files, 500MB, 50MB/file), context deadline
- RescanWorkspace: hash-based dedup, path-based scope detection
(teams/{id}/ → team scope), idempotent via UpsertDocument ON CONFLICT
- Per-agent mutex (409 Conflict for concurrent rescans)
- Tenant-scoped workspace resolution (config.TenantWorkspace)
- Enrichment worker: semaphore-based parallel summarize (max 3 concurrent)
- Media files without summary skip LLM summarize, embed title+path only
- DRY: export InferTitle/InferDocType from vault pkg, remove from interceptor
- Auto-register text uploads in vault via onTextUploaded callback
Frontend:
- Rescan button in vault page header (FolderSync icon)
- Toast with result counts, 409 warning, error handling
- i18n strings for en/vi/zh
Security: skip all symlinks, boundary checks, per-file size limit,
tenant isolation via server-side workspace resolution.
|
||
|
viettranx
|
8d37dc45ea |
feat(pipeline): per-provider context window, cache/tenant/pipeline hardening
Ship Group A (bug fixes) + Group B (pipeline enhancements) from plans/260410-1009-openclaw-ts-feature-port — five changes that share cmd/ wiring and pipeline plumbing so they commit as a unit. cache: InMemoryCache now supports periodic sweep + max-size cap via variadic options. PermissionCache wires 60s sweep + 10k entry cap + Close() hook in gateway shutdown so long-running gateways don't leak per-user permission entries. Backward compatible for zero-arg callers. store: ContactCollector.seen cache key now includes tenant_id + channel_ instance so the same sender in different tenants (or different bot instances in the same tenant) no longer silently skip upserts against each other. Zero-tenant (Desktop) behaviour preserved. pipeline: EffectiveContextWindow is resolved once per run in ContextStage via a ResolveContextWindow callback (backed by providers.ModelRegistry) so PruneStage bills history against the actual model window instead of a stale static config. Nil resolver / unknown model fall back to Config.ContextWindow for backward compatibility. Locked to the model observed at context build time to prevent mid-run budget drift. pipeline: PipelineConfig.ReserveTokens carves out an optional safety buffer subtracted from the history budget so compaction fires slightly before the hard limit — protects against provider over-delivery and token counter drift on streaming responses. Zero (default) preserves legacy budget math. agent: ModelRegistry flows gateway → ResolverDeps → LoopConfig → Loop → pipeline adapter so resolver can look up per-model capabilities at run time without re-touching gateway internals. 20 regression tests across cache, contact collector, and pipeline cover the critical paths: cross-tenant isolation, cache sweep + eviction + Close idempotency, per-model window override + fallback, and reserve token buffer behaviour. All passing with -race on both go build ./... and go build -tags sqliteonly ./.... |
||
|
Plateau Nguyen
|
fb8afd41bf |
feat(channels): add Facebook Messenger and Pancake channel integrations (#731)
Add two new channel implementations for Facebook Fanpage (comment + Messenger auto-reply, first inbox DM) and Pancake/pages.fm (multi-platform inbox via Facebook, Zalo, Instagram, TikTok, WhatsApp, LINE). Key features: - Facebook: comment auto-reply, Messenger auto-reply, first inbox DM, HMAC-SHA256 webhook verification, multi-page webhook routing - Pancake: multi-platform inbox, outbound echo dedup with HTML normalization, race-condition-safe echo fingerprinting, platform-aware formatting - Bootstrap skip: pre-fill USER.md from channel metadata (Pancake) - SanitizeDisplayName across all channels (defense in depth) Code audit fixes: - Fix truncateForTikTok byte→rune slicing (UTF-8 corruption) - Fix empty message.ID shared dedup slot (silent message loss) - Fix DisplayName markdown injection in buildPrefilledUser - Consolidate duplicate ChannelMeta type (agent→bootstrap) - Compile-time interface assertions, alphabetical type constants - Per-message logs demoted to slog.Debug, errors.As for wrapped errors - UI: alphabetical channel ordering, complete config schemas - Remove deprecated WhatsApp bridge_url, fix nested error parsing |
||
|
|
8f56ddaa64 |
feat(v3): core architecture redesign — pipeline, memory, vault, evolution, providers, orchestration (#790)
* feat(v3): add core interface contracts and migration for v3 redesign
Foundation interfaces: TokenCounter, WorkspaceContext, DomainEventBus,
ProviderAdapter/Capabilities. Pipeline: Stage, RunState, MessageBuffer,
substates, Pipeline orchestrator. Memory: EpisodicStore, AutoInjector,
KG temporal extensions, consolidation workers. System integration:
PromptConfig, ToolCapability, Retriever. Orchestration: OrchestrationMode,
EvolutionMetrics/SuggestionStore. Migration 000037: episodic_summaries,
evolution tables, KG temporal columns. Schema version 36→37.
* refactor(plans): mark all v3 design phases complete with file references
* fix(v3): address code review findings on design contracts
- C1: add missing l0_abstract column to episodic_summaries migration
- C2: align EpisodicSummary ID/TenantID/AgentID to uuid.UUID
- H1: document tenant_id scoping requirement on EpisodicStore
- H2: add UNIQUE constraint on (agent_id, user_id, source_id) for dedup
- H4: clarify ProviderAdapter vs Provider relationship in doc
- M3: set state.ExitCode on BreakLoop/AbortRun in pipeline
- M6: store full PipelineConfig in Pipeline struct
- Edge: add WHERE embedding IS NOT NULL on HNSW index
* fix(v3): second-pass review fixes
- H1: use context.WithoutCancel for finalize + set ExitCode on ctx cancel
- H2: use utf8.RuneCountInString consistently in FallbackCounter
- H3: longest-prefix-match in ModelContextWindow (prevents wrong tokenizer)
- H4: return unsubscribe cleanup func from consolidation.Register
* feat(v3): implement DomainEventBus with worker pool, dedup, and retry
Worker pool processes events from buffered channel. SourceID-based dedup
prevents duplicate processing. Exponential backoff retry on handler error.
Panic recovery per handler. Graceful shutdown via Drain(). 8/8 tests pass
with race detector.
* feat(v3): implement ProviderAdapter for Anthropic, OpenAI, DashScope, Codex
Add CapabilitiesAware to all 6 providers. Create ProviderAdapter
implementations that delegate to existing buildRequestBody/parseResponse
for DRY. ClaudeCLI and ACP get capabilities only (subprocess transport).
DashScope wraps OpenAI adapter with StreamWithTools=false override.
* feat(v3): implement WorkspaceContext Resolver for 6 scenarios
Stateless resolver produces immutable WorkspaceContext at run start.
Handles personal/group/predefined/team-shared/team-isolated/delegation.
Wired into loop_context.go behind v3PipelineEnabled flag (additive,
v2 path unchanged). Includes delegation path boundary check,
master tenant bypass, and tenant slug path composition.
* feat(v3): implement tiktoken TokenCounter with BPE encoding + cache
Adds tiktoken-go for accurate cl100k_base/o200k_base token counting.
Per-message FNV-1a hash cache avoids re-encoding unchanged history.
Falls back to rune/3 heuristic for unknown models. NewTokenCounter
factory selects implementation at build time.
* feat(v3): promote 12 other_config JSONB fields to dedicated agent columns
Extract emoji, agent_description, thinking_level, max_tokens,
self_evolve, skill_evolve, skill_nudge_interval, reasoning_config,
workspace_sharing, chatgpt_oauth_routing, shell_deny_groups, and
kg_dedup_config from the catch-all other_config JSONB into proper
columns with DB-level types and defaults.
- Migration: PG (000037) + SQLite (schema v6→7) with backfill
- Go: AgentData struct + simplified Parse* methods
- Store: SELECT/INSERT/scan updated for both PG and SQLite
- Gateway: create/update handlers accept promoted fields
- HTTP: export/import with legacy backward compat
- Web UI: all 15 frontend files read/write from top level
* feat(v3): implement Knowledge Vault with unified search, wikilinks, and FS sync
Migration 000038 adds vault_documents (FTS+pgvector), vault_links, vault_versions
tables. VaultStore interface with PG implementation for document CRUD, hybrid
FTS+vector search, and bidirectional link management. All queries enforce
tenant_id isolation including JOIN-based scoping on link operations.
FS sync layer: SHA-256 content hashing, VaultInterceptor hooks into write_file/
read_file for auto-registration and lazy sync, fsnotify watcher with 500ms
debounce. Wikilink engine parses [[target]] syntax, resolves targets via
3-step strategy, and maintains vault_links on write.
VaultSearchService fans out queries across vault, episodic, and KG stores in
parallel with per-source score normalization and weighted merge. AutoInjector
and Retriever implementations for pipeline integration.
Three agent tools: vault_search (unified discovery), vault_link (explicit
linking), vault_backlinks (dependency tracing). Feature-flagged via
v3_vault_enabled agent setting.
* feat(v3): wire vault into gateway startup + add unit tests
Wire VaultStore embedding provider, VaultSearchService, VaultInterceptor
on read/write tools, and register vault_search/vault_link/vault_backlinks
tools in gateway_vault_wiring.go. All wiring gated by stores.Vault != nil.
Add 28 unit tests for ContentHash, ContentHashFile, and ExtractWikilinks
covering edge cases, unicode, display text, context windows, and offsets.
* feat(v3): implement stage-based pipeline loop with 8 pluggable stages
Decompose monolithic agent loop into internal/pipeline/ package:
- 6 stages: Context, Think, Prune+MemoryFlush, Tool, Observe+Checkpoint, Finalize
- Foundation types: Stage interface, RunState with 7 typed substates, MessageBuffer
- Pipeline orchestrator with setup/iteration/finalize 3-phase execution
- Callback-based PipelineDeps avoids circular import with agent package
- Feature-flagged via v3PipelineEnabled in Loop.Run()
- All 7 exit conditions preserved (no tools, max iter, truncation, loop kill,
read-only streak, tool budget, ctx cancel)
* feat(v3): wire pipeline callbacks to Loop methods + add 71 unit tests
Wire 15 of 17 PipelineDeps callbacks from Loop methods via closures:
- Context: LoadContextFiles, BuildMessages, EnrichMedia, InjectReminders
- Think: BuildFilteredTools, CallLLM (stream/sync)
- Prune: PruneMessages, CompactMessages
- Memory: RunMemoryFlush
- Finalize: SanitizeContent, FlushMessages, UpdateMetadata, BootstrapCleanup, MaybeSummarize
- Remaining: ExecuteToolCall, CheckReadOnly (deep loop.go integration)
Add comprehensive test suite (71 tests, all passing with -race):
- MessageBuffer: 10 tests (append, flush, replace, counts)
- Pipeline.Run: 14 tests (3-phase flow, exit conditions, ctx cancel)
- Stage tests: 47 tests (ThinkStage nudges/truncation, PruneStage budget,
ToolStage parallel/exit, ObserveStage content, CheckpointStage interval,
FinalizeStage cleanup)
* feat(v3): wire remaining 2 callbacks (ExecuteToolCall, CheckReadOnly)
Complete callback wiring — 17/17 PipelineDeps callbacks now active:
- ExecuteToolCall: resolves tool name, executes via registry, processes
result via existing processToolResult with loop detection bridge
- CheckReadOnly: delegates to checkReadOnlyStreak via bridge runState
- Bridge runState shares loop detection state between pipeline and agent
* fix(v3): eliminate data race in tool execution + capture injected messages
- Remove parallel tool execution path — serialize all tool calls to avoid
data races on shared bridgeRS (loop detector, media results, deliverables)
- Loop kill checked after each tool (mid-batch early exit)
- BuildFilteredTools: capture and append injected tool-awareness messages
- Rename test to reflect sequential execution
* feat(v3): wire ResolveWorkspace, safe parallel tools, ContextStage tests
- Wire ResolveWorkspace callback via workspace.NewResolver() with
ResolveParams from Loop fields (no longer a nil stub)
- Re-add safe parallel tool execution: split into ExecuteToolRaw
(parallel I/O) + ProcessToolResult (sequential state mutation)
with opaque rawData pass-through (no double execution)
- Add 12 unit tests for ContextStage (8) + MemoryFlushStage (3)
- Split tool callbacks to loop_pipeline_tool_callbacks.go (under 200 lines)
- Capture buildFilteredTools injected messages
* feat(v3): add episodic memory store + temporal KG columns
Phase 1 — Episodic Store:
- Migration 000039: episodic_summaries table with pgvector, FTS, L0 abstracts
- EpisodicStore PG impl: CRUD, hybrid FTS+vector search, ExistsBySourceID,
PruneExpired. Idempotent via source_id UNIQUE constraint.
Phase 2 — Temporal KG:
- Migration 000040: valid_from/valid_until on kg_entities + kg_relations,
partial indexes for current-facts queries, epoch→timestamptz backfill
- ListEntitiesTemporal: current-only, point-in-time, or include-expired modes
- SupersedeEntity: atomic expire-old + insert-new in single transaction
Schema version bumped to 40.
* fix(v3): review fixes for episodic store + temporal KG
- C1: Fix column name mismatch turn_count vs message_count in Go SQL
- C2: Remove redundant migration 000040 (000037 already adds temporal KG columns)
- H1: Use time.Time not int64 for TIMESTAMPTZ columns in SupersedeEntity
- H2: Add tenant_id scoping to Get/Delete for tenant isolation
- M2: Fix scanEntityTemporal to convert TIMESTAMPTZ→UnixMilli correctly
- L1: Remove unused uuid import from episodic_search.go
- Schema version corrected to 39 (only 000039 is new)
* feat(v3): implement consolidation pipeline with 3 event-driven workers
Event chain: session.completed → EpisodicWorker → episodic.created →
SemanticWorker → entity.upserted → DedupWorker
- EpisodicWorker: reuses compaction summary or calls LLM, generates L0
abstract (extractive), idempotent via source_id check
- SemanticWorker: extracts KG facts from episodic summary via existing
Extractor, sets temporal valid_from, publishes entity.upserted
- DedupWorker: runs DedupAfterExtraction on new entity IDs (terminal)
- L0 abstract: sentence-based extraction (~50 tokens), no LLM needed
- All workers registered via DomainEventBus.Subscribe()
* feat(v3): implement progressive loading with L0 auto-inject + unified search
- AutoInjector: searches episodic store, builds L0 prompt section (~200 tokens),
skips trivial messages via stopword filter
- L1Cache: in-memory LRU (500 entries, 1h TTL) for structured overviews
- UnifiedSearch: cross-tier search merging episodic + document results by score
- ContextStage integration: AutoInject callback appends memory section to system prompt
- MemorySection field added to ContextState for observability
* feat(v3): add memory_expand tool for L2 episodic retrieval
New tool: memory_expand(id) returns full episodic summary with metadata.
Complements memory_search L0/L1 results with deep L2 access.
Nil-safe: returns error message when episodic store not available.
Gateway wiring + memory_search depth param + kg_search temporal param
deferred to runtime integration phase.
* feat(v3): complete Phase 5 — tool extensions + gateway wiring
- memory_search: add depth param + episodic tier search merged with docs
- kg_search: add as_of temporal param, use ListEntitiesTemporal
- memory_expand: registered in gateway startup
- Gateway: Episodic field in Stores, PGEpisodicStore in factory,
embedding provider wired, tools connected to episodic store
* fix(v3): Phase 3 review fixes — tenant isolation + AutoInject args
- C1: Add tenant_id filter to ftsSearch, vectorSearch, List queries
(prevents cross-tenant episodic memory leaks)
- C2: Fix AutoInject callback signature — agent/tenant captured by
closure, only userMessage + userID passed explicitly
- H1: Add tenant_id to List query
* feat(v3): wire per-agent v3 flags from DB into dual-mode gate
Parse v3_pipeline_enabled, v3_memory_enabled, v3_retrieval_enabled from
agent other_config JSONB via ParseV3Flags(). Resolver now sets all flags
on LoopConfig so the existing gate in loop_run.go reads from DB.
- V3Flags struct + ParseV3Flags() + ValidateV3Flags() in store layer
- v3MemoryEnabled/v3RetrievalEnabled added to Loop, LoopConfig, PipelineConfig
- Auto-inject gated on V3RetrievalEnabled (was unconditional)
- Structured perf logging for v3 pipeline runs
- v3 flag validation on both WS agent.update and HTTP PUT endpoints
* feat(v3): wire AutoInjector into pipeline for L0 memory auto-inject
Create AutoInjector at gateway startup from episodic store, pass through
ResolverDeps → LoopConfig → Loop. Pipeline adapter builds AutoInject
callback capturing agent/tenant context via closure.
ContextStage already gates on V3RetrievalEnabled + AutoInject != nil.
* feat(v3): add tool metadata map + capability-based deny rules
Registry gains per-tool ToolMetadata map with RegisterWithMetadata()
and GetMetadata() (infers defaults from tool name when not explicit).
PolicyEngine gains DenyCapability() for RBAC integration — tools with
denied capabilities filtered at step 8 after existing 7-step pipeline.
* fix(v3): add RWMutex to PolicyEngine capability deny fields
DenyCapability() and SetRegistry() now guarded by sync.RWMutex.
FilterTools reads snapshot under RLock. Prevents data race when
capability rules are modified concurrently with tool filtering.
* feat(v3): implement delegate tool for inter-agent task delegation
New `delegate` tool wraps existing agent_links infrastructure
(CanDelegate, DelegateTargets). Supports async (fire-and-forget)
and sync (block with timeout) modes. Permission checked via
AgentLinkStore. Events emitted: delegate.sent/completed/failed.
DelegateRunFunc injected by gateway to avoid circular dependency.
* feat(v3): complete 3 deferred implementations
1. OrchestrationMode resolution: ResolveOrchestrationMode() checks
team membership → delegate links → spawn (priority order).
2. PG EvolutionMetricsStore: RecordMetric, QueryMetrics, aggregate
tool/retrieval metrics, TTL cleanup. All queries tenant-scoped.
3. BridgePromptBuilder: implements PromptBuilder interface by
delegating to existing BuildSystemPrompt(). Appends v3 memory
L0 section when enabled. Ready for template engine swap later.
* fix(v3): address code review findings on commits 5-6
- C1: CanDelegate now tenant-scoped (fail-closed on missing tenant)
- H1: Sync delegate timeout capped at 600s
- H2: Async goroutine gets 10min deadline (prevents leaks)
- H3: JSONB casts use COALESCE/NULLIF guards (handles missing fields)
- M1/M2: Remove dead code (formatVaultSection, memoryL0ToStrings)
* fix(teams): stop auto-creating agent_links for team members
Teams use agent_team_members table directly — agent_links caused
context confusion between team dispatch and delegation systems.
- Remove autoCreateTeamLinks() calls from team create + member add
- Remove link cleanup from member remove
- Remove dead autoCreateTeamLinks() function
- Append DELETE to migration 000039: clear team-created agent_links
* fix(v3): tenant isolation for all agent_links queries + PromptBuilder Instructions
- DelegateTargets, GetLinkBetween, SearchDelegateTargets,
SearchDelegateTargetsByEmbedding, DeleteTeamLinksForAgent all now
scoped by tenant_id (fail-closed on missing tenant)
- BridgePromptBuilder now maps Instructions/InstructionContent to
AGENTS.md context file (was silently dropped)
* feat(v3): wire orchestration mode + evolution metrics into agent loop
- Orchestration mode: resolver resolves mode from team/links, tool filter
hides delegate/team_tasks based on mode, prompt builder injects delegation
targets section
- Evolution metrics: non-blocking goroutine records tool execution metrics
(name, success, duration) via EvolutionMetricsStore in both v2 loop and
v3 pipeline paths (sequential + parallel)
- Fix review findings: tenant ID propagated via store.WithTenantID in
background goroutine, 5s timeout prevents goroutine leak
* feat(v3): implement suggestion engine with pluggable analysis rules
- PG EvolutionSuggestionStore: CRUD for agent_evolution_suggestions table
- SuggestionEngine: aggregates 7-day metrics, runs rules, deduplicates
pending suggestions per type before creating new ones
- 3 initial rules: LowRetrievalUsage (usage_rate<0.2), ToolFailure
(success_rate<0.1), RepeatedTool (>100 calls/week → suggest skill)
- EventSuggestionCreated event type added to eventbus
- Cron wiring deferred to gateway startup integration pass
* feat(v3): implement auto-adapt guardrails with apply/rollback
- AdaptationGuardrails: max delta per cycle, min data points, locked
params, rollback-on-drop percentage
- ApplySuggestion: applies threshold suggestions to agent other_config
JSONB, stores baseline for rollback
- RollbackSuggestion: restores baseline values from suggestion params
- EvaluateApplied: compares post-apply metrics to baseline, auto-rolls
back when quality drops beyond threshold
- Scope limited to retrieval params only (never security settings)
* feat(v3): wire evolution stores + daily/weekly cron for suggestions
- Add EvolutionMetrics + EvolutionSuggestions to Stores struct + PG factory
- Wire EvolutionMetricsStore into ResolverDeps (cmd/gateway_managed.go)
- Add gateway_evolution_cron.go: daily suggestion analysis + weekly
evaluation/rollback for applied suggestions
- Cron runs as background goroutine with 5-min timeout per cycle
* fix(v3): address code review findings on evolution engine
- C1: persist baseline parameters before marking suggestion as applied
(was building map but never saving — rollback would always fail)
- H1: add tenant_id isolation to UpdateSuggestionStatus, GetSuggestion,
and new UpdateSuggestionParameters method
* test(v3): add unit tests for orchestration, suggestions, guardrails, prompt
- orchestration_mode_test: orchModeDenyTools (4 modes) + ResolveOrchestrationMode
(4 scenarios with mock stores)
- suggestion_rules_test: LowRetrievalUsage, ToolFailure, RepeatedTool with
threshold boundary tests (at/below/above min data points)
- evolution_guardrails_test: DefaultGuardrails values + CheckGuardrails
(insufficient data, locked params, zero-min fallback)
- prompt_builder_orchestration_test: BridgePromptBuilder orchestration section
presence/absence across 4 scenarios + target content verification
* test(v3): add integration tests for evolution metrics + suggestions
- Test helper: shared PG connection with sync.Once migration, per-test
tenant+agent seed with cleanup
- Evolution metrics: RecordMetric, AggregateToolMetrics (success rate),
Cleanup (TTL deletion)
- Evolution suggestions: full CRUD, UpdateSuggestionParameters (baseline
persist), tenant isolation (cross-tenant read blocked)
- Pipeline E2E: seed 25 failed tools + 55 low-usage retrievals, verify
SuggestionEngine creates suggestions, verify dedup on second run
- Fix: migration 039 de-duped (episodic_summaries already in 037)
- Fix: NULL reviewed_by scan via sql.NullString
* feat(v3): add HTTP API handlers for evolution, vault, episodic, orchestration, v3-flags
5 new handler files exposing v3 backend stores as REST endpoints:
- evolution_handlers.go: metrics query/aggregate + suggestions CRUD
- vault_handlers.go: cross-agent document listing + search + links
- episodic_handlers.go: episodic summaries list + hybrid search
- orchestration_handlers.go: computed mode + delegate targets (read-only)
- v3_flags_handlers.go: per-agent v3 feature flag get/toggle
Store fixes from code review:
- episodic FTS: use inline to_tsvector (no stored tsv column)
- episodic: conditional user_id filter in List + Search (admin view)
- episodic: add tenant_id to ExistsBySourceID + PruneExpired
- evolution: require tenant_id in context (no struct fallback)
- evolution: check RowsAffected on suggestion updates
- vault: optional agent_id filter in ListDocuments (cross-agent)
* feat(v3): add web UI for evolution tab, v3 settings, vault page, episodic memory
Agent Detail enhancements:
- V3 Settings section: pipeline/memory/retrieval flag toggles
- Orchestration section: mode badge + delegate targets display
- Evolution section: added metrics + suggestions v3 flag toggles
- Evolution tab: Recharts metrics charts + suggestion review table
with approve/reject/rollback actions + guardrails card
New pages:
- /vault: Knowledge Vault document registry with cross-agent listing,
hybrid search dialog, document detail with wikilinks
- Memory page: added Episodic Memory tab with summary cards,
expandable details, key topic badges, and hybrid search
Infrastructure:
- HttpClient: added patch() method
- Query keys: v3Flags, orchestration, evolution namespaces
- 4 new hooks: use-v3-flags, use-orchestration, use-evolution-metrics,
use-evolution-suggestions, use-vault, use-episodic
- i18n: vault namespace (en/vi/zh), agents + memory keys updated
- Reused formatRelativeTime from lib/format.ts (eliminated 3 duplicates)
* refactor(http): add bindJSON helper and migrate all decode call sites
Replace 36 json.NewDecoder(r.Body).Decode + error blocks with bindJSON
across 20 HTTP handler files. Standardizes decode error responses to
structured writeError format. Fixes unchecked decode in handleIndexAll.
* refactor(store): adopt sqlx for PG scan operations (Phase 1+2)
Add jmoiron/sqlx v1.4.0 with camelToSnake json tag mapper.
Migrate scan-heavy PG store methods to sqlx Get/Select:
- tracing.go: GetTrace, ListTraces, ListChildTraces, GetTraceSpans, GetCostSummary
- heartbeat.go: Get, ListDue, ListLogs
- providers.go: GetProvider, GetProviderByName, ListProviders, ListAllProviders
- mcp_servers.go: GetServer, GetServerByName, ListServers
- pairing.go: ListPending, ListPaired
- agents_export_queries.go: 5 export functions
- agents_export_team_queries.go: exportTeamMembers, ExportAgentLinks
All writes (INSERT/UPDATE/DELETE), execMapUpdate, and dynamic WHERE
builders remain raw SQL. Zero behavior change.
* refactor(store): adopt sqlx for SQLite scan operations (Phase 3)
Migrate SQLite store scan methods to sqlx Get/Select:
- providers.go: GetProvider, GetProviderByName, ListProviders, ListAllProviders
- tenants.go: GetTenant, GetTenantBySlug, ListTenants, GetTenantUser, ListUsers, ListUserTenants
- mcp_servers.go: GetServer, GetServerByName, ListServers
Create sqlx_scan_structs.go with sqliteTime-aware scan structs
(providerRow, tenantRow, tenantUserRow, mcpServerRow) to handle
SQLite TEXT timestamp parsing via StructScan.
* refactor(store): migrate PG bulk scan operations to sqlx (Phase 4)
Migrate scan-heavy methods across 6 PG store files:
- tenant_store.go: GetTenant, GetTenantBySlug, ListTenants, GetTenantUser,
ListUsers, ListUserTenants — removed 3 scan helpers
- teams.go: ListTeams, GetTeam, ListMembers, ListMembersByTenant
- teams_tasks_activity.go: ListComments, ListEvents, ListFollowUps
- pending_message_store.go: ListPending, ListByHistoryKey
- skills_grants.go: ListAgentGrants
- config_permissions.go: CheckPermission
~20 scan ops converted. Files with encryption post-processing,
pq.Array, pgvector, or dynamic SQL kept raw.
* refactor(store): extract shared CamelToSnake mapper, add UUIDArray usage note
- Move camelToSnake to internal/store/column_mapper.go (DRY)
- Both pg and sqlitestore packages now import shared CamelToSnake
- Add planned-use comment on UUIDArray type
* refactor(cli): migrate commands from config.json to HTTP API, add providers/setup/TUI
- Add unified HTTP client (gateway_http_client.go) with auth, error parsing, typed generics
- Rewrite agent list/add/delete to use gateway HTTP API instead of config.json
- Rewrite channels list to HTTP API, add channels add/delete subcommands
- Replace models command with full providers CRUD (list/add/update/delete/verify)
- Add setup wizard command (provider → agent → channel post-onboard flow)
- Add Bubble Tea TUI behind build tag (tui/!tui with noop fallback)
- Update onboard next-steps to mention goclaw setup
- Add build-tui Makefile target
- Fix URL path injection (url.PathEscape on all user-supplied path segments)
- Fix UTF-8 truncation in skills description display
* refactor(store): add explicit db struct tags, fix sqlx mapper for heartbeat scan error
Switch sqlx mapper from NewMapperFunc (which only applies CamelToSnake to
field names, not tag values) to NewMapperFunc("db", CamelToSnake) with
explicit db:"column_name" tags on all store structs.
Root cause: NewMapperFunc("json", fn) sets mapFunc but not tagMapFunc,
so camelCase json tags like "agentId" were used as-is instead of being
converted to "agent_id", causing "missing destination name" scan errors.
Fix: use db struct tags as the source of truth for column mapping.
Every DB entity field gets db:"column_name", nested JSON configs and
runtime-only structs get db:"-".
* test(store): add integration tests for 13 store interfaces (70 tests)
Cover Tier 1 (critical) + Tier 2 (security) stores with integration tests
running against pgvector pg18. Coverage from 2.4% to ~54%.
Stores tested: Session, Agent, Team/Task, Memory, KnowledgeGraph, Vault,
MCP Server, API Key, ConfigPermission, Contact.
Infrastructure: fixture builders (seedTeam, seedMCPServer, etc.),
mock EmbeddingProvider, multi-tenant helpers, expanded cleanup.
* fix(store): resolve NULL scan bugs in MCP server and task metadata
- mcp_servers: COALESCE nullable TEXT columns (display_name, command,
url, api_key, tool_prefix) to prevent sqlx scan failures
- mcp_servers_access: COALESCE nullable JSONB columns in ListAgentGrants
(tool_allow, tool_deny, config_overrides) to prevent silent row drops
- teams_tasks: default task metadata to '{}' instead of nil to satisfy
NOT NULL constraint on CreateTask
- sqlx_helpers: export InitSqlx for integration test setup
* feat(pipeline): fix v3 pipeline context injection, tracing, KG temporal filters
- Pipeline context: add InjectContext + LoadSessionHistory callbacks to
ContextStage, propagate enriched ctx via state.Ctx for iteration stages
- Pipeline tracing: wrap makeCallLLM with emitLLMSpanStart/End, wrap
makeExecuteToolCall/Raw with emitToolSpanStart/End
- Token counter: switch pipeline from FallbackCounter to TiktokenCounter
- KG temporal: add valid_until IS NULL filter to all entity/relation
queries (list, search, vector, FTS, traversal CTE, stats)
- Skills: add SkillEmbedder interface for future hybrid BM25+vector search
- Cache: remove unused tenantResolve dead code from PermissionCache
- Store: fix NULL scan bugs in tracing metadata and agent skill_nudge
- Test: add TestStoreKG_TemporalFilter integration test
- UI: add v3 version badge, evolution section, memory/traces improvements
* refactor(store): migrate KG store from raw sql.Rows to sqlx StructScan
Migrate 6 knowledge graph store files from manual rows.Scan() to
pkgSqlxDB.GetContext/SelectContext with intermediate scan row structs.
- Add entityRow, relationRow, traversalRow, dedupCandidateRow structs
with json.RawMessage for jsonb and time.Time for timestamptz columns
- Add toEntity()/toRelation() converters (UnixMilli + json.Unmarshal)
- Add sqlxTx() helper for wrapping *sql.Tx with sqlx mapper
- Fix ScanDuplicates passing time.Now().Unix() to TIMESTAMPTZ column
- Fix ListEntitiesTemporal missing tenant scope (scopeClause)
- Fix SupersedeEntity missing tenant scope and tenant_id on INSERT
- Fix DedupCandidate.CreatedAt using Unix() instead of UnixMilli()
- Update agents_export_queries.go to reuse new scan row structs
- Net -160 lines of manual scan boilerplate removed
* refactor(store): migrate memory, skills, agents, sessions, mcp, cron, vault stores to sqlx
Batch migration of 19 store files from raw rows.Scan() to
pkgSqlxDB.GetContext/SelectContext with intermediate scan row structs.
Groups migrated:
- Memory: memory_docs, memory_admin, memory_search, memory_embedding_cache
- Episodic: episodic_search, episodic_summaries
- Skills: skills, skills_admin, skills_embedding, skills_export_queries
- Agents: agents (backfill+shares), agents_context, agents_export_team_standalone
- Sessions: sessions_list (List, ListPaged, ListPagedRich)
- MCP: mcp_servers_access, mcp_export_queries
- Cron: cron_exec (GetRunLog)
- Vault: vault_documents (ListDocuments, ftsSearch, vectorSearch)
- Tenant: tenant_configs (ListDisabled, ListAll)
7 new scan row files created. Net -510 lines of manual scan boilerplate.
INSERT/UPDATE/DELETE and scalar COUNT queries kept as raw SQL.
* fix(store): fix 3 sqlx scan struct db tag issues found by audit
- Fix vault FTS alias mismatch: `AS rank` → `AS score` (critical: runtime scan error)
- Fix episodic key_topics type: json.RawMessage → pq.StringArray (TEXT[] column)
- Fix agentShareRow.CreatedAt: string → time.Time, wire to output struct
* feat(providers): implement Wave 2 provider resilience and intelligence
9-phase implementation covering:
- Request middleware chain with composable body transformers
- OpenAI prompt caching, service tier, and fast mode middlewares
- Error classification (9 categories) with two-tier failover
- Model registry with forward-compat resolvers (Anthropic + OpenAI)
- Embedding providers (OpenAI + Voyage) with 1536-dim validation
- Cooldown/probe system with per-provider:model state tracking
- Markdown-aware chunking shared across 5 channels
- Session recall via FTS + pgvector on episodic summaries
- Dreaming/promotion pipeline for long-term memory consolidation
Migrations: 000040 (episodic search index), 000041 (promoted_at column)
Schema version: 39 → 41
* feat(providers): wire model registry into gateway provider construction
Create InMemoryRegistry with Anthropic + OpenAI forward-compat resolvers
at gateway startup. Pass to all Anthropic and OpenAI providers created
from both config and DB sources.
* feat(consolidation): wire DomainEventBus and consolidation pipeline
Create DomainEventBus at gateway startup, thread through resolver →
LoopConfig → Loop → PipelineDeps. Emit session.completed event after
each run finalization. Register consolidation pipeline (episodic →
semantic → KG dedup → dreaming) with event bus subscriptions.
* fix(store): fix episodic key_topics pq.Array, ON CONFLICT, and migration 040 immutability
- episodic_summaries.go Create: json.Marshal(KeyTopics) → pq.Array (text[] column)
- episodic_search.go scanEpisodic/scanEpisodicRow: json.RawMessage → pq.StringArray
- episodic_summaries.go Create: ON CONFLICT add WHERE source_id IS NOT NULL for partial index
- migration 040: add immutable_array_to_string wrapper (array_to_string is STABLE in PG)
* test(store): add 17 integration tests for skills, cron, episodic, tenant configs
- Skills store: 6 tests (CRUD, grants, tenant isolation)
- Cron store: 4 tests (job CRUD, run log sqlx scan, pagination, tenant isolation)
- Episodic store: 4 tests (summary CRUD, list, FTS search, tenant isolation)
- Tenant configs: 3 tests (tool/skill disable, list, tenant isolation)
- Test helper: add cleanup for skills, cron, episodic tables
* fix(permissions): use cron-specific permission check for cron tool (#725)
* fix(security): harden exec path exemption matching (#721)
- Add absolute path exemption for dataDir/skills-store/ (fixes skill
scripts using absolute paths like /app/data/skills-store/ being denied)
- Strip surrounding quotes before prefix matching (LLMs often quote paths)
- Reject path traversal ("..") in exempt fields to prevent escape
- Switch from "any field exempt → skip" to per-field matching: only exempt
if ALL fields that match the deny pattern are individually exempt
- Closes pipe/comment bypass vectors where an exempt path in one argument
would exempt the entire command including non-exempt paths
Includes 27 test cases covering: legitimate access, quoted paths,
path traversal, unicode bypass, pipe/comment bypass, mixed args.
* fix(permissions): use cron-specific permission check for cron tool
Cron tool was hardcoded to check `file_writer` configType via
CheckFileWriterPermission(), ignoring the `cron` configType that
the UI actually saves when granting cron permissions. This caused
agents in group chats to be denied cron access even with correct
permission configured.
Add ConfigTypeCron constant and CheckCronPermission() that checks
`cron` configType first, falling back to `file_writer`.
---------
Co-authored-by: Viet Tran <viettranx@gmail.com>
* fix(chat): load message history on first conversation click (#730)
* fix(chat): load message history when selecting existing conversation from clean state
The skipNextHistoryRef was unconditionally set when sessionKey transitioned
from empty to non-empty. This prevented loadHistory() from running when
clicking an existing conversation from the initial /chat page. The skip
was only intended for the new-chat send flow where the optimistic message
is already displayed.
Guard the skip with expectingRunRef so it only activates when a message
send is in flight.
Closes #729
* docs: add UI diff evidence for PR #730
Before/after screenshots and HTML comparison report showing
first conversation click behavior fix.
* feat(whatsapp): port native WhatsApp channel with whatsmeow from dev
Cherry-pick
|
||
|
viettranx
|
63f0c0fe45 |
feat(agent): centralized tenant user identity resolution for credentials
Add CredentialUserID context key that resolves channel contacts to merged tenant users for credential lookups (SecureCLI, MCP). Keeps UserID unchanged for session/workspace scoping. Resolves group senders, group contacts, and unresolved DMs via ContactStore with 60s TTL cache. |
||
|
viettranx
|
39bb90bd60 |
refactor(permissions): remove auto-add file writer, add config type constants
- Remove auto-add logic that granted file_writer permission to the first group/guild member who chatted with the bot - Add ConfigTypeFileWriter and ConfigTypeHeartbeat constants, replace all hardcoded config_type strings across callers - Add bootstrap exception: /addwriter and !addwriter allow first writer to be added when no writers exist yet - Optimize writer commands: reuse cached ListFileWriters result for both permission check and last-writer guard, reducing DB queries per command - Add freshness directive to file writer system prompt so bot prioritizes current list over stale references in conversation history |
||
|
Kai (Tam Nhu) Tranandviettranx
|
3ca3bb2062 |
feat: add capability-aware reasoning effort controls (#593)
* feat(reasoning): add capability-aware effort resolution - resolve requested reasoning levels against exact model capabilities - persist requested effort on agents and expose effective effort in traces - add backend tests for provider models, agent store, and resolution logic Refs #591 * feat(ui): gate reasoning controls by model capabilities - only show supported reasoning levels when provider model metadata is available - preserve expert reasoning selections during async model loading - surface effective reasoning details in trace dialogs and localized copy Refs #591 * docs(api): document capability-aware reasoning controls - describe exact-match capability lookup and downgrade behavior - update provider model metadata and trace response documentation - refresh the generated OpenAPI spec for the new reasoning fields Refs #591 * feat: add provider-first reasoning controls * docs: refresh PR 593 UI evidence callouts * refactor: deduplicate reasoning normalize functions and remove PR evidence - Export NormalizeReasoningEffort/NormalizeReasoningFallback from providers package; store package now delegates instead of duplicating - Store reasoning fallback constants alias providers canonical definitions - Export deriveLegacyThinkingLevel from types/provider.ts; remove local copies from agent-advanced-dialog and provider-overview - Remove unused _providerType param from useProviderModels hook - Fix reasoning debug log to fire for all cases with a reason (not just non-off efforts) - Remove docs/pr-593-evidence/ binary screenshots from repo --------- Co-authored-by: viettranx <viettranx@gmail.com> |
||
|
viettranx
|
2e869f4ece |
feat(kg): add tsvector FTS, entity deduplication, and embedding fixes
- Replace ILIKE search with PostgreSQL tsvector/GIN full-text search for KG entities (migration 000031) - Add entity deduplication system with dual-threshold strategy: auto-merge at 0.98+ similarity with Jaro-Winkler name match, flag 0.90-0.98 as candidates for manual review - Add ScanDuplicates for bulk on-demand duplicate detection - Add MergeEntities with advisory lock and tenant-scoped relation re-pointing (delete-then-update to avoid ON CONFLICT on UPDATE) - Wire dedup inline after KG extraction pipeline - Fix BackfillKGEmbeddings: was failing silently due to context.Background() missing tenant_id; now runs cross-tenant - Fix BackfillKGEmbeddings: break on error → continue with failed ID tracking and max consecutive error cap - Add EmbedEntity helper; UpsertEntity now generates embeddings in background goroutine - Add HTTP endpoints: POST /kg/dedup/scan, GET /kg/dedup, POST /kg/merge, POST /kg/dedup/dismiss - Add web UI: Dedup dialog with Scan All button, side-by-side entity comparison, merge/dismiss actions - Add Jaro-Winkler similarity algorithm with 34 unit tests - Update IngestExtraction to return upserted entity IDs - Bump RequiredSchemaVersion to 31 |
||
|
viettranx
|
b088144fd9 |
fix(sqlite): bootstrap not running on first chat due to per-connection PRAGMA gap
Root cause: pool.go applied busy_timeout PRAGMA via db.Exec() which only affects one connection in the pool. Other connections had no busy_timeout, causing immediate SQLITE_BUSY errors during concurrent startup operations (agent creation, WebSocket connect, health checks). This silently aborted context file seeding — BOOTSTRAP.md, USER.md, AGENTS.md all missing from system prompt on first interaction. Fix (3 layers): 1. pragmaConnector: wraps sql.Driver to apply PRAGMAs (busy_timeout, WAL, etc.) on every new connection, not just one. All SQLite queries benefit. 2. CacheInvalidateFunc: clears ContextFileInterceptor cache after seeding so LoadContextFiles sees newly seeded files on the first turn. 3. fallbackBootstrap: if DB seed still fails, injects embedded templates in-memory so the first turn still gets onboarding. Clears after use. |
||
|
viettranx
|
d3bf16d2de |
refactor(bootstrap): separate profile and seeding callbacks, consolidate per-user state
- Split EnsureUserFilesFunc into EnsureUserProfileFunc (profile + workspace) and SeedUserFilesFunc (context file seeding) for single-responsibility - Merge userWorkspaces + userFilesSeeded sync.Maps into unified userSetups struct to prevent desync between workspace and seeding state - Add skipIfAnyExist param to SeedUserFiles to encapsulate the "seed only for brand-new users" logic within the bootstrap package - Extract getOrCreateUserSetup helper for clean per-user initialization - Add bootstrap state tests covering all 4 system prompt branches - Keep legacy EnsureUserFilesFunc as fallback for backward compatibility |
||
|
Viet Tran
|
e183b459c9 |
feat: SQLite desktop edition — full desktop app with team tasks (#505)
* feat(store): add SQLite backend foundation with build-tag injection
Add sqlitestore package (//go:build sqlite) as alternative to PostgreSQL:
- pool.go: WAL mode, busy_timeout, 4 concurrent read connections
- helpers.go: ? param helpers, JSON array storage, nullable/update utils
- scope.go: tenant scope with ? placeholders (vs PG's $N)
- schema.sql: 1296-line flattened DDL from 29 PG migrations (51 tables)
- schema.go: embedded schema with transactional apply + version tracking
- factory.go: NewSQLiteStores() stub (stores wired in Phase 2)
Build-tag split for store initialization:
- cmd/gateway_stores_pg.go (//go:build !sqlite) — default PG-only
- cmd/gateway_stores_sqlite.go (//go:build sqlite) — runtime PG/SQLite switch
- cmd/gateway_setup.go: extracted wireTracingAndCron() shared helper
Config: GOCLAW_STORAGE_BACKEND + GOCLAW_SQLITE_PATH env vars.
Security: goclaw.db added to DenyPaths (exec, read_file, filesystem tools).
* feat(edition): add centralized edition package for feature tier limits
New internal/edition/ package — single source of truth for all edition limits:
- Edition struct with MaxAgents, MaxTeams, MaxChannels, KGEnabled, TeamFullMode, etc.
- Standard (default, all features) and Lite (desktop, 5 agents, 1 team) presets
- atomic.Pointer for thread-safe Current()/SetCurrent()
Wiring:
- cmd/gateway.go: GOCLAW_EDITION env override (lite/standard) at startup
- cmd/gateway_stores_sqlite.go: auto-set Lite when backend=sqlite
- /v1/edition HTTP endpoint for UI comparison modal (public, no auth)
* feat(sqlitestore): implement Phase 2A core stores + sqliteonly build tag
Implement 12 SQLite store backends (4200+ LOC) mirroring existing PG stores:
- SessionStore, AgentStore, ProviderStore, TracingStore, SnapshotStore
- ConfigSecretsStore, SystemConfigStore, TenantStore, HeartbeatStore
- BuiltinToolStore, BuiltinToolTenantConfigStore, SkillTenantConfigStore
All stores wired in factory.go. Remaining stores (Memory, Cron, Skills,
Teams, etc.) left nil — gateway handles gracefully.
Add sqliteonly build tag for PG-free desktop builds:
- go build . → PG only
- go build -tags sqlite . → PG + SQLite (runtime switch)
- go build -tags sqliteonly . → SQLite only (no pgx dependency)
Key SQLite adaptations:
- ? placeholders (not $N), json_extract/json_each/json_array_length
- DISTINCT ON → GROUP BY + Go dedup, ANY($1) → IN (?,?...)
- rows.Err() checks on all scan loops, execMapUpdateWhereTenant in helpers
- GetOrCreateUserProfile uses RowsAffected() instead of PG's xmax trick
* feat(sqlitestore): implement Phase 2B feature stores
Port 8 additional store backends to SQLite (19 new files, ~6000 LOC):
Cron: cron.go, cron_crud.go, cron_exec.go, cron_scheduler.go
- Job scheduling with cache, ListDue, MarkRunning, MarkComplete
Skills: skills.go, skills_crud.go, skills_content.go, skills_grants.go
- CRUD, grants, content management. LIKE search (no FTS/vector in Lite)
MCP: mcp_servers.go, mcp_servers_access.go, mcp_user_credentials.go
- Server CRUD, agent/user grants, encrypted credentials
Channels: channel_instances.go, pairing.go, pending_messages.go, contacts.go
- Channel management, device pairing, message queue, contact store
Teams: teams.go, teams_tasks.go, teams_tasks_lifecycle.go, teams_tasks_activity.go
- Team/task CRUD, lifecycle transitions, activity log, progress tracking
- JSON array for members/blocked_by (replaces PG text[])
Remaining nil stores: Memory, AgentLinks, KnowledgeGraph, Activity,
SecureCLI, APIKeys, ConfigPermissions — gateway handles gracefully.
* fix(sqlitestore): fix critical arg ordering + data races in Phase 2B stores
C1: ListTasks arg mismatch — limit+1 in userID slot, wrong results
C2: SearchTasks missing duplicate userID arg for (? = '' OR t.user_id = ?)
H3: cron_crud.go cacheLoaded written without mutex → use InvalidateCache()
H4: cron_exec.go discarded ExecContext error → log warning
H1: Add rows.Err() checks in cron_crud, cron_scheduler, cron_exec, pairing
* feat(sqlitestore): implement Phase 2C — Memory, Activity, APIKeys, ConfigPermissions
Complete remaining essential stores (6 new files, ~900 LOC):
Memory: memory.go, memory_docs.go, memory_search.go
- Document/chunk CRUD, LIKE-based search (no vector in Lite edition)
- Embedding methods return empty results gracefully
Activity: activity.go — simple activity logging
APIKeys: api_keys.go — API key CRUD with SHA-256 hash lookup
ConfigPermissions: config_permissions.go — permission rules with TTL cache
24/27 stores now wired. Only AgentLinks, KnowledgeGraph, SecureCLI
remain nil (disabled in Lite edition by design).
Total: 49 files, 11,435 LOC in internal/store/sqlitestore/
* fix(sqlitestore): fix variable shadow in GetDocument + handle chunk delete error
C1: GetDocument scopeClause used := inside if/else blocks, shadowing
outer err variable — query errors silently swallowed. Fixed by renaming
to tcErr matching PG pattern.
H1: IndexDocument chunk deletion ExecContext error was discarded, could
cause duplicate chunks. Now returns error on failure.
* feat(desktop): Phase 3 — Wails v2 desktop app shell with chat UI
Desktop app (ui/desktop/) using Wails v2 + React 19 + Tailwind CSS 4:
Go backend:
- main.go: Wails entry point with embedded frontend assets
- app.go: gateway embedding via goroutine, health check, Wails bindings
- keyring.go: OS keyring secrets with file fallback
- cmd/gateway_export.go: exports RunGateway() for desktop embedding
React frontend:
- WS v3 client: protocol handshake, exponential backoff, call queue
- Chat system: Zustand stores, RAF-batched streaming, 10 event handlers
- Components: MessageBubble, MarkdownRenderer (rehype-sanitize),
CodeBlock, ToolCallBlock, ThinkingBlock, ActivityIndicator, InputBar
- Layout: AppShell (2-column), Sidebar with agent/session list, TopBar
- Onboarding wizard (5 steps): welcome, gateway, provider, agent, ready
- Magic Blue theme (dark/light), Inter + JetBrains Mono typography
Build: all 3 variants pass (PG, sqlite, sqliteonly)
* chore: ignore Wails build artifacts (wailsjs, build, package.json.md5)
* fix(desktop): correct onboarding provider list and agent creation API contract
- ProviderStep: expand from 3 to 16 providers in 4 groups (Popular, Cloud, Local, Regional)
with correct provider_type values and api_base defaults
- AgentStep: fix API payload — use agent_key (slug), provider (name string),
agent_type=predefined with description in other_config
- use-agents: fix field mapping — agent_key, display_name from backend response
- Clean up failed provider on verify error
* fix(desktop): add missing providers — Bailian Coding, Z.ai Coding, Ollama Cloud
* fix(desktop): match web dashboard brand colors + persist onboarding in store
- Replace Magic Blue theme with web dashboard's warm blue OKLCH palette
- Move onboarding state from localStorage to Zustand persist store
- Add "Run Setup Wizard" option in TopBar settings menu to re-trigger onboarding
- Fix light mode theme activation (explicit :root:not(.dark) overrides)
* fix(desktop): apply dark theme before first paint + better error messages
- Add class="dark" default on <html> + inline script to read persisted theme
before React hydrate (prevents light flash on dark mode)
- Improve provider test error message for network failures
* feat(desktop): add GoClaw logo from web UI to onboarding + topbar
* fix(desktop): add Vite proxy for dev mode to avoid CORS gateway errors
- Proxy /v1, /ws, /health to localhost:18790 in Vite dev server
- Use relative URLs in dev mode (import.meta.env.DEV) so proxy handles CORS
- Production build uses direct gateway URL from Wails binding
* fix(desktop): GoClaw dock icon + fix duplicate provider slug on re-test
- Convert goclaw-icon.svg to 1024x1024 PNG for macOS dock icon
- Fix provider creation: handle existing slug by finding and updating
- Track build/appicon.png in git (exclude only build/bin/)
* fix(sqlitestore): UUID text/blob mismatch in scopeClause breaks all queries
scopeClause passed uuid.UUID (16-byte array) to SQLite ? placeholder,
but tenant_id column stores TEXT strings. SQLite compared BLOB vs TEXT
→ no match → all scoped queries returned empty results.
Fix: pass scope.TenantID.String() to ensure TEXT comparison.
Also includes:
- macOS dock icon (.icns from GoClaw logo)
- Onboarding auto-detect existing providers/agents
- Debug logging for token + API URL
* fix(desktop): CORS for dev mode + direct gateway URL
- Wails dev server (port 34115) doesn't proxy API calls, causing 405
- Frontend now connects directly to gateway URL from Wails binding
- Added GOCLAW_DESKTOP=1 env → enables CORS middleware on gateway
- desktopCORS wraps mux with Access-Control-Allow-* headers + OPTIONS
- Simplified provider test flow: list-then-create/update
* fix(desktop): split onboarding into 6 steps matching web UI flow
Web UI flow: create provider → select model + verify → create agent.
Desktop was incorrectly trying to verify before creating provider.
Changes:
- ProviderStep: now only creates/saves provider (no verify)
- NEW ModelVerifyStep: loads models from provider, test connection
- AgentStep: receives pre-selected model, shows read-only model field
- OnboardingWizard: 6 steps with auto-detect skip logic
- Auto-detect: has agents→Ready, has providers→ModelVerify, nothing→Provider
* feat(desktop): add Combobox component, use searchable model selector
- New Combobox: searchable dropdown with custom value support
- ModelVerifyStep: replace plain input/select with Combobox
- Models loaded from GET /v1/providers/{id}/models API
- Allows typing custom model name if API returns empty list
* fix(desktop): rename icon to iconfile.icns matching Wails convention
* refactor(desktop): overhaul UI/UX, fix onboarding, fix event handling
Desktop app major refactor:
UI/UX:
- Chat-focused layout with floating panels on dotted canvas
- Sidebar: agent list + sessions grouped by date (no resource counts)
- Modern input bar: rounded pill with attach/send buttons
- User bubble: card style matching web UI (not solid color)
- Thinking block: collapsible, max-height, proper label
- ErrorBoundary wrapping app
- Dock icon: regenerated with transparent bg + dark rounded frame
Onboarding:
- 3-step flow matching web UI (Provider → Model → Agent)
- Proper create vs update (check DB before POST)
- SetupStepper with step circles + connectors
- Agent presets from web UI (Fox Spirit, Artisan, Astrologer)
- Auto-detect existing setup via use-bootstrap-status hook
Event handling (verified from Go source):
- chunk: payload.content (not payload.chunk)
- thinking: payload.content (not payload.thinking)
- tool.call: payload.id/name (not toolId/toolName)
- tool.result: payload.is_error/content (not error field)
- run.completed: usage.prompt_tokens/completion_tokens
- New: block.reply, run.retrying handlers
Backend fixes:
- SQLite scanTime helper for modernc.org/sqlite text timestamps
- X-GoClaw-User-Id header in desktop API client (not X-User-ID)
- user_id: system (owner role in desktop single-user mode)
- CORS: allow X-GoClaw-User-Id header
- SQLite busy timeout: 5s → 10s
- Snapshot SQL: cross-DB compatible (FILTER→CASE, ::BIGINT→CAST)
- Promise.allSettled for bootstrap status (one fail doesn't block other)
* feat(desktop): chat polish, SQLite fixes, summoning modal, bootstrap guard
Chat Polish (Phase 1):
- ToolCallBlock: Wrench/Zap icons, phase badges, arg summary, grouped rendering
- ThinkingBlock: auto-expand on stream, Brain icon, cursor pulse
- MessageBubble: isStreaming prop, streaming cursor, grouped tool calls
- ImageLightbox: fullscreen overlay, gallery nav, keyboard shortcuts, download
- MediaBlock: grid layout, click-to-open lightbox, hover overlay
- ActivityIndicator: phase-specific icons (Brain/Wrench/RefreshCw)
- ChatCanvas: track lastAssistantId for streaming, EmptyState with prompts
- Filter [System] nudge messages and tool-role messages from chat history
SQLite Fixes:
- sessions_list: fix time.Time scan failure (3 sites) using sqliteTime scanner
- snapshots: fix ON CONFLICT expression mismatch with unique index
- snapshots: fix GetLatestBucket using nullSqliteTime
- pool: explicit PRAGMAs (busy_timeout=15s, WAL, synchronous=NORMAL)
- schema: seed master tenant (was missing, causing FK violations)
- schema: incremental migration framework (version-gated patches)
Onboarding:
- SummoningModal: port from web UI with framer-motion animations
- AgentStep: show summoning modal after create, continue button
- App: auto-detect empty DB and reset onboarded flag
- ChatCanvas: loading spinner while agent loads after onboarding
Bootstrap Guard:
- After auto-cleanup of BOOTSTRAP.md, check if USER.md is still empty
- Inject reminder if agent cleared BOOTSTRAP but didn't fill USER.md
Session Management:
- Load chat history on session click (was missing useEffect)
- Fix race condition: don't clear messages on session switch (atomic replace)
- SidebarFooter: center New Chat button text
* fix(desktop): session management, delete confirm, event listener race fix
- New Chat: only clears state, no empty session creation (sendMessage auto-creates)
- Delete session: hover X button with ConfirmDialog confirmation
- Event listener: use sessionKeyRef instead of closure to prevent stale events
- Remove "skip to dashboard" link from onboarding
- Add reusable ConfirmDialog + ConfirmDeleteDialog common components
* feat(desktop): settings view with tabbed layout (Phase 4)
- SettingsView: tab container with header, close button, canvas-dots bg
- SettingsTabBar: 9 tabs (Appearance, Providers, Agents, MCP, Skills, Tools, Cron, Traces, About)
- AppearanceTab: dark/light theme toggle, language + timezone placeholders
- AboutTab: version, edition limits, runtime info
- ui-store: activeView, settingsTab, openSettings(), closeSettings()
- AppShell: switch between chat and settings views
- SidebarFooter: gear icon opens settings (was "Run Setup Wizard")
- Keyboard: Cmd+, opens settings, Escape closes
- Branding: "GoClaw Lite" in sidebar header
- Agent status: online (green) instead of idle for desktop
- Tab content wrapped in solid bg card with border
* feat(desktop): provider management CRUD in settings (Phase 5)
- use-providers hook: list, create, update, delete, verify via HTTP API
- ProviderList: list view with Add button, empty state
- ProviderRow: status dot, type badge, edit/delete actions
- ProviderFormDialog: create/edit with type selector, masked API key, test connection
- Wire ProviderList into Settings Providers tab
* fix(desktop): remove Test Connection from provider form (requires model + provider ID)
* feat(desktop): agent management CRUD in settings (Phase 6)
- AgentData/AgentInput types matching web UI contracts
- use-agent-crud hook: list, create, update, delete, resummon (5 agent limit)
- AgentCard: emoji, status dot, provider/model/type badges, edit/delete/resummon
- AgentFormDialog: provider Combobox + model Combobox (from /v1/providers/{id}/models)
- Create: agent_type selector, "Check & Create" verifies model before create
- Edit: no type change, no re-verify needed
- Personality textarea for predefined agents
- AgentList: grid, edition limit warning, create triggers SummoningModal
- Delete uses ConfirmDeleteDialog (type name to confirm)
- Sidebar agent list refreshes after CRUD
* feat(desktop): agent detail panel with full config (Phase 6 polish)
Agent detail panel (fullscreen overlay covering sidebar):
- PersonalitySection: emoji editor, display name, description, status select, default toggle, agent key display
- ModelBudgetSection: provider/model Combobox with verify-before-save, context window, max tool iterations
- EvolutionSection: self_evolve toggle with info callout (predefined agents only)
- Sticky save bar: backdrop blur, cancel/save buttons, spinner on save
- Save blocked if provider/model changed but not verified
AgentList: card click opens detail panel, create dialog separate
AgentFormDialog: create-only, Check & Create with verify
* fix(desktop): resummon confirm, summoning z-index, save bar UX
- Resummon requires confirm dialog before triggering
- SummoningModal z-index z-50 → z-[70] (above detail panel z-[60])
- Save bar: show error inline, "Verify model first" when blocked
* feat(desktop): agent detail quality polish — files tab, memory config, rich cards
Types (synced with web UI):
- AgentData: added owner_id, workspace, restrict_to_workspace, frontmatter, context_window, max_tool_iterations as required fields
- MemoryConfig, CompactionConfig interfaces
- BootstrapFile type for WS file operations
AgentCard (matching web UI agent-card.tsx):
- Star icon for default agent
- Animated pulse badge for summoning status
- Self-evolve sparkle indicator (orange when active)
- Frontmatter/expertise with line-clamp-3
- Context window display (e.g. "200K ctx")
- Safe emoji extraction, UUID name detection
AgentDetailPanel:
- Tab navigation: Overview + Files tabs
- Overview: Personality + ModelBudget + Memory + Evolution sections
- Files tab: WS-based file editor (agents.files.list/get/set)
- File sidebar with selection
- Textarea editor with dirty tracking
- Save button with spinner
- Sticky save bar with backdrop blur (overview only)
- Resummon with confirm dialog
MemorySection (matching web UI memory-section.tsx):
- Enable/disable toggle
- 6 config fields: max_results, min_score, max_chunk_len, chunk_overlap, vector_weight, text_weight
- "Using global defaults" when disabled
* feat(desktop): agent files — hide USER/HEARTBEAT, add Edit with AI
- Hide USER.md, USER_PREDEFINED.md, HEARTBEAT.md from files tab (managed by bootstrap/cron)
- Add "Edit with AI" button → RegenerateDialog → POST /v1/agents/{id}/regenerate
- Auto-select first file on load
- Show file size in sidebar
- Pass agentId to files tab for regenerate API call
* fix(desktop): remove memory config section (no embedding in SQLite)
* fix(desktop): remove misleading bytes display from file sidebar
* fix(desktop): replace native checkboxes/selects with custom Switch + Combobox
- New Switch component matching Radix UI toggle style
- Replace all native <input type="checkbox"> with Switch in agent/provider forms
- Replace native <select> for status with Combobox
- All interactive elements have cursor-pointer
* fix(desktop): evolution callout colors — use opacity-based for both themes
* fix(desktop): global cursor-pointer for all interactive elements
* fix(desktop): improve dark mode contrast for text and status colors
- text-secondary: 0.62 → 0.68 lightness (better readability on dark bg)
- text-muted: 0.52 → 0.58 lightness (was below WCAG AA 4.5:1 minimum)
- success: 0.45 → 0.55 (green was too dim on dark bg)
- warning: 0.65 → 0.70 (slightly brighter)
- idle: 0.52 → 0.58 (match text-muted)
* fix(desktop): revert color values to exact web UI match (0.62/0.52/0.45)
* fix(desktop): agent card badge colors — match web UI badge variants exactly
* feat(desktop): add MCP servers + Builtin Tools settings tabs
Phase 7 implementation:
MCP Tab:
- Full CRUD with 5-server edition limit
- Form dialog with transport-conditional fields (stdio/SSE/streamable-http)
- Test Connection with inline success/error feedback
- Agent grants dialog (grant/revoke per agent)
- Tools discovery dialog (view server tools)
- KeyValueEditor with sensitive field masking (auth/token/secret)
Tools Tab:
- Category-grouped list of 41 seeded builtin tools
- Toggle enable/disable with optimistic update
- Specialized settings forms: web_fetch extractor chain, media provider chain
- Generic JSON editor fallback for other tools
- Provider/model Combobox selection for media tools
Common:
- RefreshButton component with 500ms min spin animation
- KeyValueEditor with password masking for sensitive keys
Fixes:
- SQLite builtin_tools scan: use scanTimePair() for timestamps
- Session click while in settings: now closes settings view
- Dark mode text contrast: bumped text-secondary/text-muted lightness
- Light mode text contrast: darkened text-secondary/text-muted
- Focus ring thickness: ring-2 → ring-1 globally
- Misleading "memory layering (Postgres)" log label
* feat(desktop): add Skills tab, agent skill grants, emoji avatar, agent form redesign
Skills:
- Skills settings tab with upload ZIP, toggle, delete, runtime check
- Agent skill grants section in agent detail panel (toggle per agent)
- SQLite SkillManageStore interface compliance fixes
Agent form:
- 2-column layout, wider modal (max-w-3xl)
- 6 personality presets (Fox Spirit, Artisan, Astrologer, Researcher, Writer, Coder)
- Separate Verify Model + Summon buttons
- Always predefined type (removed open option)
Fixes:
- SQLite time.Time scan: mcp_servers, mcp_grants, activity_logs
- Combobox portal with scroll/resize tracking
- AgentAvatar shows emoji from other_config
- uploadFile sends X-GoClaw-User-Id header + generic type
- isApiClientReady guard prevents ErrorBoundary crash
- Verify model field: valid (not success)
* fix(sqlitestore): comprehensive timestamp scan sweep + UI fixes
SQLite timestamp sweep (13 files, 25+ sites):
All time.Time direct scans replaced with sqliteTime/scanTimePair/nullSqliteTime.
Files: activity, teams, teams_tasks, teams_tasks_activity, config_permissions,
agents_access, tracing_spans, tracing_scan, channel_instances, tenants,
pending_messages, api_keys, heartbeat.
UI fixes:
- Agent switching: clear active session + chat when agent changes
- MCP table: vertical align middle on row cells
* fix(sqlitestore): fix json.RawMessage scan + sqliteVal for dynamic updates
- mcp_servers_access: scan json columns via string intermediates (SQLite
TEXT → json.RawMessage incompatible, use string then convert)
- helpers: add sqliteVal() to auto-marshal map/slice/struct to JSON
string in execMapUpdate/execMapUpdateWhereTenant — fixes agent save
500 error when updating other_config, tools_config, etc.
- Add AgentMcpSection: toggle MCP server grants per agent in detail panel
- Clean up debug logging from McpGrantsDialog
* feat(desktop): add i18n (react-i18next) + toast system + language/timezone pickers
i18n:
- Install react-i18next + i18next
- Copy 12 web locale namespaces (en/vi/zh) + desktop.json namespace
- Create i18n/index.ts with browser language detection + localStorage persist
- Replace ~300 hardcoded strings across 40+ components with t() calls
- Language picker in ChatTopBar (top-right) + Settings > Appearance
- Timezone picker in ChatTopBar with search + Intl.supportedValuesOf fallback
- All 6 agent presets fully translated (vi/zh) with prompts from locale files
- Agent Key never translated (uses stable English slugs)
Toast:
- Zustand toast store (success/error/warning/default, 4s auto-dismiss)
- Toaster component (bottom-right, z-100, slide-in animation)
- Toast calls in all CRUD hooks (agents, providers, MCP, skills, tools)
* feat(desktop): add Cron Jobs + Traces settings tabs with syntax highlighting
Cron Jobs (Phase 8):
- WS RPC hook (cron.list/create/delete/toggle/run/runs)
- CronList table with schedule formatting, status badges, run/toggle/delete
- CronFormDialog with slug name, agent selector, 3-way schedule (every/cron/once)
- CronRunsDialog for execution history
Traces (Phase 9):
- HTTP REST hook (GET /v1/traces with pagination + agent filter)
- TraceList table with duration, tokens, spans, relative time
- TraceDetailDialog with metadata, collapsible input/output, flat span list
- Syntax-highlighted JSON/code previews via react-syntax-highlighter
* fix(sqlitestore): fix tracing scan — endTime *time.Time → nullSqliteTime
Both trace and span scan functions used *time.Time for nullable end_time
column, which fails on SQLite TEXT timestamps. Changed to nullSqliteTime
with Valid check before assigning pointer.
* fix(desktop): fix cron schedule type label — map 'at' kind to 'once' i18n key
* fix(desktop): prevent flash on cron/traces refresh — keep data while refetching
* feat(desktop): add file attachment rendering — FileButton + FilePreviewDialog
- FileButton: compact attachment button with emoji icon, filename, size, download
- FilePreviewDialog: modal with type-detected preview (image/video/audio/markdown/code/text)
- MarkdownRenderer: override a/img for /v1/files/ links → FileButton + resolved URLs
- MediaBlock: non-image files render as FileButton instead of plain links
- api.ts: add getBaseUrl() for file URL resolution
* fix(desktop): resolve file URLs in chat — toFileUrl for media_refs + relative paths
- use-chat: add toFileUrl() to convert raw paths to /v1/files/{basename} URLs
for both run.completed media and history media_refs
- MarkdownRenderer: detect relative file paths (./path/file.ext) via isFileLink
in addition to /v1/files/ links, resolve all to gateway URL
* fix(desktop): authenticated file serving — media cache + blob URLs
Security fix: all /v1/files/ requests now use Bearer auth via fetchFile().
No more raw <a href> or <img src> with unauthenticated file URLs.
- Add media-cache.ts (blob cache with 5-min TTL, dedup inflight fetches)
- Add use-media-url.ts hook (returns cached blob URL for authenticated media)
- Add AuthImage component (loads images via auth blob) + downloadFile helper
- Update MarkdownRenderer: file images use AuthImage, downloads use downloadFile
- Update FileButton: authenticated download via blob
- Update FilePreviewDialog: authenticated fetch for text/preview content
- Clean filename display (strip timestamps + query params)
* fix(desktop): fix file attachments — use media_refs.id for URL, auth all media
Root cause: media_refs from backend has {id, mime_type, kind} but no path/url.
The id IS the filename basename. Fixed toFileUrl to use ref.id as fallback.
Also: AuthImage for lightbox, authenticated audio/video in MediaBlock,
clean filename display (strip timestamps).
* fix(desktop): file serving, traces, and media rendering improvements
Backend:
- Store MediaRef.Path in loop_finalize for direct file serving
- Use full path (not basename) in gateway_managed event signing
- Add fuzzyMatchInDir for LLM-hallucinated filenames
- Add findInWorkspace support for agent dirs and ws/ directory
- Add POST /v1/files/sign endpoint for client-side URL signing
- Always save LLM span input_preview (not just verbose mode)
- Truncate previews from tail (keep recent context), limit 2000 chars
- Add exact filename hint to create_image/video/audio tool results
Desktop frontend:
- File attachments: FileButton, FilePreviewDialog, AuthImage with blob cache
- Media cache: sign URLs via API for non-ft URLs, Bearer auth fallback
- Download via Wails SaveFile binding (native Save As dialog)
- OpenFile + DownloadURL Wails Go bindings
- Traces: rewrite with span tree hierarchy, formatTokens (90.8K),
formatDuration with start/end fallback, expandable spans
- Traces: export via DownloadURL, copy with checkmark state
- Code preview: JSON = oneDark syntax highlight, text = light pre block
- MarkdownRenderer: baseUrl prop for relative image resolution
- ImageLightbox: preventDefault on keyboard nav (no macOS beep)
- ErrorBoundary: reload instead of re-render on retry
- Combobox: compact sizing (py-1.5, text-sm)
- FileButton: fix nested button HTML violation
* feat(desktop): team tasks kanban board + edition policy + file preview fixes
Phase 11 implementation:
Backend:
- TeamActionPolicy interface — lite/full edition gating for team_tasks tool
- Filter blocked actions from schema enum + early guard in Execute()
- System prompt: edition-specific team member guidance
- Skip skill_manage/publish_skill registration + seeding in lite
- teams.create: require at least 1 member
- files.go: 2-layer path isolation (workspace boundary + tenant scope)
Desktop UI:
- Kanban board with 6 status columns + framer-motion layout animation
- Task detail modal with collapsible description/result sections
- Team create dialog with styled member checkboxes
- Sidebar: teams section with create button + Lite edition badge
- Chat TaskPanel: compact active tasks with real-time WS updates
- Real-time: debounced get-light fetch (300ms) + progress patch (1s)
- Edition comparison modal (Lite vs Standard feature table)
- Custom dropdown filter replacing native select
- i18n: teams namespace (en/vi/zh)
Fixes:
- WS params: camelCase (teamId, taskId, sessionKey) matching backend
- FileButton: span wrapper (HTML nesting) + createPortal for dialog
- FilePreviewDialog: defensive filename normalization for extension check
- use-chat: prevSessionRef init null — fix blank chat on view switch
- Scrollbar: 3px auto-hide
- Agent click in sidebar returns to chat view
* feat: Update frontend build assets, change the default agent thinking level to 'low', and update Go module dependencies.
|
||
|
viettranx
|
5f1658cfad |
fix(gateway): prevent double /v1/files/ prefix in media URLs
OnEvent handler mutated MediaResult.Path in-place on the shared RunResult.Media slice, corrupting paths for downstream consumers (announce queue, outbound channels). Clone the media slice before signing URLs. Add defense-in-depth guard in mediaToMarkdown to skip prefix when path already starts with v1/files/ or v1/media/. |
||
|
viettranx
|
d3a4398e78 |
fix(gateway): sign file URLs in run.completed events for real-time display
OnEvent callback only handled map[string]string payloads, but run.completed uses map[string]any (contains usage map + media slice). File URLs in content were never signed, and media paths were sent as raw local paths without ?ft= hash tokens. Switch on payload type to handle both shapes: sign content URLs and convert media local paths to /v1/files/basename?ft=hash. |
||
|
viettranx
|
7aa47d1baf |
refactor(security): remove cross-tenant bypass, enforce strict tenant isolation
- Add RoleOwner ("owner") to permissions hierarchy (above admin)
- Replace all IsCrossTenant/client.IsCrossTenant() permission guards with
IsOwnerRole/client.IsOwner() role-based checks (20+ locations)
- Remove cross-tenant bypass from tenantClauseN SQL helper — always adds
AND tenant_id = $N (fail-closed)
- Add explicit unscoped store methods for startup: ListAllProviders,
ListAllInstances, ListAllEnabled, GetByIDUnscoped, GetTeamUnscoped
- Remove CrossTenant from auth layer: owner users get role "owner" with
concrete tenant ID (MasterTenantID fallback), no bypass flag
- Remove crossTenant field from WS Client; connect response sends
is_owner instead of cross_tenant
- Simplify event filter from 3-mode to 2-mode (tenant-scoped + owner sees system events)
- Fix channel instance loader to use inst.TenantID for agent lookup
- Fix agent embedding goroutine to capture tenant from caller context
- Fix provider cache invalidation to use event.TenantID
- Separate skills store: ListAllSystemSkills (system only) vs
ListAllSkills (system + tenant-scoped), StoreMissingDeps restricted
to system skills only
- Frontend: isCrossTenant → isOwner, add "owner" to UserRole type,
RequireAdmin uses hasMinRole for owner > admin hierarchy
|
||
|
Viet Tran
|
cd022699f6 |
feat: multi-tenant isolation — complete implementation (#359)
* feat(security): multi-tenant user data isolation (Plan 1)
Comprehensive user data isolation for non-owner system users:
- API key identity binding: owner_id column forces user_id on auth,
prevents spoofing via X-GoClaw-User-Id header
- Sessions: ownership checks on list/preview/patch/delete/reset,
non-admin users see only their own sessions
- Cron: user_id filtering on list, ownership checks on mutations
- Server-side WS event filtering: agent/chat/session/cron/team events
scoped per-user instead of broadcast to all clients
- Web UI role guards: RequireAdmin on 15 admin-only pages, role
propagated from WS connect response to auth store
- Tracing/activity: user_id enforcement for non-admin HTTP callers
- Teams: HasTeamAccess membership checks on get/delete/list
- Skills: fail-closed ownership check (deny non-admin if store
doesn't support owner lookup)
- HTTP auth: requireAuthBearer now enforces owner_id + user context
for file/media downloads (was missing)
- Dead code: removed delegation_history, handoff_routes tables and
all related handlers/store code
- New: team_user_grants table for user-to-team access control
Migration 000026: api_keys.owner_id + team_user_grants + DROP legacy tables
* feat(security): multi-tenant foundation — tenants table, tenant_id propagation, permission cache (Plan 2)
Add tenant isolation infrastructure across the entire gateway:
Schema (migration 000027):
- Create tenants + tenant_users tables with master tenant seed
- Add tenant_id column to 30 user-scoped tables (NOT NULL DEFAULT master)
- api_keys.tenant_id nullable (NULL = system-level cross-tenant key)
- Create builtin_tool_tenant_configs + skill_tenant_configs for per-tenant overrides
- Drop custom_tools table (agent loop integration never wired)
Store layer:
- TenantStore interface + PGTenantStore (CRUD tenants + tenant_users)
- TenantID field on AgentData + APIKeyData
- tenant_id in agents/api_keys/skills SQL (Create, Get, List)
Context propagation:
- WithTenantID/TenantIDFromContext (uuid.Nil = fail-closed)
- WithCrossTenant/IsCrossTenant (owner/system admin flag)
Auth tenant resolution:
- HTTP: resolveAuthBearer sets TenantID/CrossTenant on all 5 auth paths
- WS: handleConnect sets tenantID/crossTenant on Client
- API key 2-tier: NULL = cross-tenant (system), set = tenant-scoped
Runtime isolation:
- Event bus: TenantID field on Event, fail-closed filter in event_filter.go
- Cron: tenant context injected in RunJob handler
- Subagent: tenant validation prevents cross-tenant spawn
- Security logging: tenant_id in auth resolution logs
Tenant management:
- WS RPC: 7 methods (tenants.list/get/create/update, tenants.users.*)
- HTTP: 7 endpoints (/v1/tenants/*)
- Slug validation + path traversal prevention
- Role validation (owner/admin/operator/member/viewer)
Infrastructure:
- PermissionCache: 4 sub-caches (tenant resolve, role, agent access, team access)
- tenant_paths.go: filesystem path helpers with master-tenant backward compat
- i18n: MsgInvalidRole key + translations (en/vi/zh)
Dead code removed: custom_tools store, HTTP handler, DynamicToolLoader (-828 lines)
* feat(security): tenant query filtering + workspace isolation (Plan 3)
Add WHERE tenant_id filtering to all 30+ tenant-scoped store queries,
wire workspace filesystem isolation, and harden restrict_to_workspace.
Store query filtering:
- Add tenantClauseN/tenantIDForInsert/requireTenantID helpers
- Filter all SELECT/INSERT/UPDATE/DELETE by tenant_id for non-cross-tenant
- Refactor SessionStore.GetOrCreate and CronStore.AddJob/ListJobs to
accept context.Context for tenant propagation
- System skills (is_system=true) bypass tenant filter for all tenants
- Special cases: GetByKey (channels), GetByHash (auth) skip filter
Workspace isolation:
- Resolver computes tenant-scoped workspace + dataDir for non-master tenants
- Add WithTenantSlug/TenantSlugFromContext to context propagation
- Add TenantStore + Workspace to ResolverDeps
- Force effectiveRestrict() to always return true (multi-tenant security)
- Remove restrict_to_workspace from agentAllowedFields
UI cleanup:
- Remove custom-tools pages, types, routes, constants (backend removed in Plan 2)
- Clean tool-name-select component of custom tools references
* feat(security): session ctx propagation + execMapUpdate tenant guard (Plan 4)
Session store:
- Add ctx to AddMessage, SetSessionMetadata, SetAgentInfo, List, Save
- List now filters by tenant_id for non-cross-tenant callers
- Save uses ExecContext for cancellation support
- All ~15 callers updated to pass ctx
execMapUpdate tenant guard:
- Remove deleted_at IS NULL from execMapUpdateWhereTenant (only agents has soft-delete)
- Migrate 8 callers to execMapUpdateWhereTenant: agent_links, channel_instances,
mcp_servers, secure_cli, tracing, teams, skills_crud, cron_update
- Add ctx to UpdateSkill, UpdateJob interfaces + all callers
Deferred: cron scheduler global cache (correct by design — system process),
browser per-tenant isolation (separate plan).
* refactor(store): add context.Context to all SessionStore interface methods
Complete ctx propagation across all 24 SessionStore methods for:
- Future tenant-aware DB operations
- Request cancellation/timeout support
- Distributed tracing capability
Updated ~15 files including all callers in agent loop, gateway methods,
heartbeat ticker, tools, and CLI commands.
* fix(security): remove context.Background() shadowing in gateway handlers
Critical fix from code review: gateway agent handlers (create, update,
delete, identity, files, links, teams) were creating ctx := context.Background()
which shadowed the handler's ctx that carries tenant_id. This breaks
tenant-scoped agent queries for non-master tenants.
- Remove ctx shadowing in 7 agent handler files
- Add ctx param to resolveAgentUUID/resolveAgentInfo helpers
- Use store.WithCrossTenant in resolver (system-level operation)
* feat(security): tenant-scoped UNIQUE constraints for multi-tenant isolation
Update UNIQUE indexes to include tenant_id, allowing same names across tenants:
- agents: (agent_key) → (tenant_id, agent_key) WHERE deleted_at IS NULL
- sessions: (session_key) → (tenant_id, session_key)
- skills: (slug) → (tenant_id, slug)
- mcp_servers: (name) → (tenant_id, name)
- channel_contacts: (channel_type, sender_id) → (tenant_id, channel_type, sender_id)
Code changes:
- GetByKey now filters by tenant_id (same pattern as GetByID)
- ON CONFLICT clauses updated for sessions and skills
- Channel consumer uses WithCrossTenant for agent resolution
- Down migration restores original constraints
* fix(security): close remaining tenant isolation gaps from final audit
Critical fixes:
- gateway_setup: WithCrossTenant for default agent lookup at startup (C6)
- channel_contacts: ON CONFLICT updated to (tenant_id, channel_type, sender_id) (Q15)
- agents.Delete: tenant filter on DELETE (Q1)
High priority fixes:
- agents: List, GetDefault, ShareAgent, RevokeShare, ListShares, CanAccess,
ListAccessible, Update unset-default — all now tenant-scoped
- skills_crud: DeleteSkill now takes ctx, verifies tenant ownership
- mcp_servers, channel_instances, secure_cli: Delete methods tenant-scoped
- WithCrossTenant added to: gateway team notifications, team_tool_cache,
pending_messages GetDefault
* fix(migration): add tenant_id to usage_snapshots unique index
Update idx_usage_snapshots_unique to include tenant_id, preventing
cross-tenant upsert collisions when different tenants have agents
with same provider/model/channel combination.
* feat(security): cron tenant guard + browser per-tenant isolation
Phase 3 — Cron API tenant guard:
- Add ctx to 5 CronStore methods (GetJob, RemoveJob, EnableJob, RunJob, GetRunLog)
- All API-facing cron ops now filter by tenant_id (prevents cross-tenant CRUD)
- RemoveJob/EnableJob return "not found" on tenant mismatch (no enumeration)
- GetRunLog JOINs cron_jobs for tenant filtering
- UpdateJob internal reads scoped by tenant (defense-in-depth)
- Scheduler-internal methods (GetDueJobs, refreshJobCache) unchanged (system-level)
Phase 4 — Browser per-tenant isolation:
- Per-tenant incognito browser contexts via rod Incognito() (separate cookie jars)
- All page access (Snapshot, Screenshot, Navigate, Click, Type, etc.) validated
via getPageForTenant — blocks cross-tenant access by targetID
- OpenTab creates pages in tenant's incognito context
- ListTabs scoped to tenant's incognito context
- ConsoleMessages validates page ownership
- Stop/reconnect properly cleans up incognito contexts
* feat(security): isolation gaps + per-tenant config (Plan 5)
Part A — Isolation Gap Fixes:
- Merge migration 028 into 027: add tenant_id to llm_providers +
config_secrets, fix UNIQUE constraints for paired_devices +
channel_instances
- providers.go: tenant filtering on all CRUD queries
- config_secrets.go: ON CONFLICT (key, tenant_id)
- pairing_store: add ctx to all 7 interface methods, remove hardcoded
MasterTenantID, update ~15 channel caller files
- Session cache: prefix keys with tenantID to prevent cross-tenant
collision. DB queries (loadFromDB, Save, Delete, LastUsedChannel)
add tenant filter
- config_permissions cache: prefix keys with tenantID
- Cron ListJobs: fail-closed when tenant context missing
Part B — Per-Tenant Configuration:
- Provider Registry: compound key tenantID/name with fallback to
master tenant. GetForTenant/ListForTenant/RegisterForTenant
- Resolver: uses tenant-aware provider lookup + disabled tools query
- Agent loop: filter disabled tools from LLM tool definitions
- Builtin tool tenant configs: store interface + PG implementation +
PUT/DELETE HTTP endpoints
- Skill tenant configs: store interface + PG + ListAccessible LEFT
JOIN to exclude disabled skills per tenant
- OAuth: DBTokenSource with tenantID field for tenant-scoped token
refresh
- All HTTP provider handlers use RegisterForTenant/UnregisterForTenant
* feat(security): channel tenant propagation + MCP per-user credentials (Plan 6)
- Propagate tenant_id from channel_instances through BaseChannel →
InboundMessage → agent loop context (fixes 5-point break in tenant flow)
- Inject tenant context in WS router dispatch for all gateway methods
- Add MCP per-user credential overrides (api_key, headers, env) with
AES-256-GCM encryption and HTTP API endpoints
- Rewrite MCP pool with tenant-scoped keys, slot semaphore, idle eviction,
and credential rotation support (Evict per tenant+server)
- Bypass pool for users with custom credentials (separate connections)
- Fix MCP APIKey never passed to connections (inject as Authorization header)
* fix(security): close remaining tenant isolation gaps from Plan 1-6 audit
- Add tenant_id to 6 missing tables: agent_context_files,
skill_agent_grants, mcp_agent_grants, team_tasks, spans,
embedding_cache (migration 027)
- Fix tid==uuid.Nil fallback to fail-closed (return error) in 8 update
methods: agent_links, teams, skills, channel_instances, secure_cli,
cron, mcp_servers, tracing
- Add tenant filter to bare DELETEs: DeleteLink, DeleteTeam
- Add tenant filter to queries: ListChildTraces, GetMonthlyAgentCost,
CountAgentGrantsByServer, ListAccessible (MCP), ReviewRequest,
ResolveGroupTitles, buildTraceWhere
- Fix missing tenant_id in INSERTs: CreateSkill, GrantToUser,
ReviewRequest grant INSERTs
- Add tenant filter to api_keys: List, Revoke, Delete
- Fix cron scanJob/RemoveJob/EnableJob fallthrough patterns
* fix(security): inject tenant context into channel handler entry points
Channel handlers used context.Background() which lost tenant context,
causing store operations to either fail-closed or default to master
tenant. Now all 10 handler entry points inject tenant from BaseChannel.
* fix(security): tenant filters for teams, tasks, skills (Plan 6b audit)
- Teams: add tenant filter to GetTeamForAgent, ListMembers,
ListIdleMembers, KnownUserIDs (JOIN agent_teams for tenant check)
- Teams: add tenant_id to GrantTeamAccess INSERT, tenant filter to
RevokeTeamAccess, ListTeamGrants, HasTeamAccess
- Team tasks: add tenant_id to CreateTask INSERT, fail-closed
UpdateTask, tenant filter on all 7 query/delete methods
- Skills: add tenant filter to RevokeFromAgent, ListAgentGrants
- Skills: add ctx param + tenant filter to ToggleSkill
- History: annotate context.Background() locations with TODOs for
future tenant injection (requires PendingHistory struct refactor)
* fix(security): add tenant_id to 4 missing team tables + fix INSERTs
Add tenant_id column to: agent_team_members, team_task_comments,
team_task_events, team_task_attachments (migration 027).
Fix INSERT statements to include tenant_id: AddMember,
AddTaskComment, RecordTaskEvent, AttachFileToTask.
* fix(migration): cast UUID literals in tenant_users seed + usage_snapshots index
PostgreSQL doesn't auto-cast string to UUID in SELECT and expression
index contexts. Add explicit ::uuid casts to prevent migration failure.
* docs: add multi-tenant architecture guide for integrators
Comprehensive solution doc covering auth model, WS protocol, event
system, data isolation, API reference, and integration patterns.
Target audience: developers building custom frontends or SaaS on GoClaw.
* feat(ui): multi-tenant awareness + tenant admin page (Plan 7)
Backend:
- Enrich WS connect response with tenant_name, tenant_slug, cross_tenant
- Add tenants.mine WS method (any user, returns own memberships)
- Parse tenant_hint in connect params for browser pairing multi-tenant
- Wire tenantStore to MethodRouter for connect-time tenant lookup
Frontend:
- Auth store: tenantId, tenantName, tenantSlug, isCrossTenant, availableTenants
- WS client: capture tenant fields from connect, send tenant_hint
- WS provider: auto-fetch tenants.mine on connect
- useTenants() shared hook for all tenant-aware components
- Tenant indicator in sidebar connection status
- Tenant admin page (/admin/tenants) with list + create dialog
- Tenants nav in sidebar (cross-tenant admin only)
- i18n: tenants namespace (en/vi/zh)
- Type updates: tenant_id on AgentData, ApiKeyData
* refactor(ui): move tenant selector into user menu dropdown in topbar
Replace simple logout button with a Radix Popover user menu showing:
- User ID display
- Tenant selector (when multi-tenant: list all tenants with check mark)
- Logout button
Remove tenant indicator from connection-status.tsx (now in topbar).
Tenant switch saves slug to localStorage and reloads for reconnect.
* feat(ui): add logout confirmation dialog
Show destructive confirm dialog before logout via ConfirmDialog
component. Added logoutConfirm i18n key for en/vi/zh.
* fix(ui): security hardening — hide admin nav, fix route guard, fix refresh
- Hide System nav group for non-admin roles in sidebar (was visible to all)
- Replace RequireAdmin with RequireCrossTenant guard on /admin/tenants route
- Add RequireCrossTenant component to require-role.tsx
- Fix refresh button animation: use isFetching instead of isLoading
- Clean up connection-status.tsx (remove tenant indicator, now in topbar)
* feat: cross-tenant admin tenant scope selector
Backend: add tenant_scope connect param. Cross-tenant clients can
narrow their scope to a specific tenant (slug). applyTenantScope()
sets client.tenantID and clears crossTenant flag.
UI: user menu shows "All Tenants" option for cross-tenant admins.
Selecting a tenant saves slug to localStorage as tenant_scope,
reload reconnects with narrowed scope. "All Tenants" clears scope.
* feat: provisioning API key scope + tenant detail page (Plan 8)
Backend:
- Add operator.provision scope for limited tenant management
- Add HasScope() method to gateway Client
- Allow provision-scoped keys to create tenants + add users
- Allow provision-scoped keys to create tenant-bound API keys
Frontend:
- Tenant detail page with user management (list, add, remove)
- Clickable tenant list rows navigate to detail
- i18n: tenant detail keys (en/vi/zh)
- Route /admin/tenants/:id with RequireCrossTenant guard
* fix: tenant scope keeps admin privileges + UI pattern fixes
Backend:
- applyTenantScope keeps crossTenant=true (retains admin features)
- Router: scoped cross-tenant injects WithTenantID (filters data)
while keeping admin role for method access
UI:
- Fix "All Tenants" check mark (compare against nil UUID string)
- Fix tenant label when scope active (show selected tenant name)
- Use ConfirmDialog for user removal (was hand-rolled)
- Add DialogDescription to add-user dialog (Radix a11y)
- Fix table min-w-[600px] consistency
- Fix column header mismatch (was "role", should be "created")
* fix(ui): clean up tenant detail header — remove redundant info panel
Remove duplicate slug/status/created panel. Info now shown in
PageHeader description (slug + date). Status badge removed (redundant
with description). Cleaner, consistent with other admin pages.
* fix(ui): redesign tenant detail with info cards + user cards
* feat(ui): tenant selection gate — require tenant before app access
- Add tenantSelected flag to auth store (persisted via localStorage)
- WS provider auto-selects: single-tenant user auto, cross-tenant
admin defaults to "All Tenants", zero-tenant user blocked
- RequireAuth gate: redirect to /select-tenant when connected but
no tenant selected
- New TenantSelectorPage: centered card layout matching login page,
"All Tenants" amber card for cross-tenant admin, per-tenant cards
with role badges, no-access state with logout button
- i18n: selectTenant, noAccess keys (en/vi/zh)
* fix(security): scope events for cross-tenant admin with tenant_scope
Event filter was checking !crossTenant before filtering — scoped
cross-tenant admins (crossTenant=true + tenantID set) bypassed
tenant event filtering. Now checks tenantID != Nil regardless of
crossTenant flag, ensuring scoped admins only see their chosen
tenant's events.
* fix(security): HTTP API now respects tenant_scope for gateway token
Root cause: UI uses HTTP API (/v1/agents, /v1/mcp/servers, etc.)
for data fetching. HTTP auth middleware with gateway token always
set CrossTenant=true with no tenant filtering. tenant_scope only
worked for WS connection, not HTTP requests.
Fix:
- HTTP client sends X-GoClaw-Tenant-Scope header from localStorage
- HTTP auth resolves header slug → tenant UUID via tenantStore
- requireAuth: CrossTenant + TenantID → WithTenantID (scoped)
- Wire InitTenantStore(pgStores.Tenants) in gateway startup
* feat(security): tenant-aware provider registry, event filter, and membership validation
- Refactor providers.Registry: Get(ctx, name) / List(ctx) extract tenant
from context via injected TenantFromCtx func (avoids circular import)
- Event filter: fail-closed 3-mode tenant filtering
Mode 1: unscoped admin sees all
Mode 2: scoped admin sees tenant events + system events
Mode 3: regular user sees only own tenant (fail-closed)
- WS connect: resolveTenantHint validates membership via GetUserRole
with PermissionCache (30s TTL, bus invalidation)
- BroadcastForTenant helper for tenant-scoped event emission
- Session list: add TenantID to SessionListOpts from context
- Cron handleRun: preserve tenant in background goroutine context
- GOCLAW_LOG_LEVEL env var (debug|info|warn|error) for Docker/K8s
- Cache debug logging: tenant_cache, permission_cache, api_key_cache
- Friendly verify error: timeout → user-readable message
- Verify timeout: 15s → 30s
* feat(ui): setup wizard improvements + agent preset enrichment
- Setup: skip link with confirm dialog, language selector (en/vi/zh)
- Setup: card padding fix (py-0 gap-0 on Card, py-5 on CardContent)
- Setup: remove duplicate skip link from layout
- Step Model: verify countdown timer (30s), stops on result
- Step Agent: default Fox Spirit preset, selected state styling,
hide agent key/name inputs, auto-derive from preset, emoji in config
- Summoning modal: elapsed timer (m:ss format)
- Agent presets: enriched prompts with human-like quirks
Fox Spirit: playful personality, care reminders
Artisan: portrait/banner/ads/logo expertise
Astrologer: reference sites (astro.com, cafeastrology, labyrinthos)
- i18n: "triệu hồi linh hồn" fix, all 3 locales updated
* feat(ui): API Key tenant support + card layout + provider chain fix
- API Key create: tenant selector for cross-tenant admin, provision scope
- API Key create: redesigned dialog with scope cards, Radix Select, icons
- API Key list: card layout with badges (status, tenant, scopes)
- API Key: shortcut in user menu (topbar)
- API Key: keep "API Key" untranslated across all locales
- Provider chain: empty state fix — skip legacy entry when provider
not found in current tenant
- i18n: form.cancel key added to all 3 locales
* fix(ui): add bottom padding to all page layouts + misc improvements
- Add pb-10 to all 24 page containers to prevent content touching
bottom edge of viewport
- Various UI polish from user modifications (summoning colors,
layout icon, agent cards, sidebar adjustments)
* feat(ui): MCP user credentials dialog + builtin tool tenant toggle
- MCP: per-user credentials dialog (api_key, headers, env KV editor)
with status badges, delete all, save
- MCP: "My Credentials" button on each server row
- Builtin Tools: per-tenant enable/disable override toggle
with "Using default" / "Enabled/Disabled for tenant" badges
and reset-to-default button
- Setup: larger logo (h-16) and bolder title (text-4xl font-bold)
- i18n: all keys added to en/vi/zh for both features
* fix(ui): API key card spacing + remove pagination border
- Card padding: px-4 py-3.5 (was px-3 py-2), rows spaced with gap-2
- Scopes on separate row from dates for readability
- Card gap: space-y-2.5 between cards
- Pagination: add className prop, remove border-t on API keys page
- Badge/icon sizes bumped to text-xs / h-3.5 (was text-[10px] / h-3)
* fix(security): comprehensive tenant isolation audit — SQL, events, cache, skills, files
Defense-in-depth hardening across 12 audit phases:
- SQL: add tenant_id WHERE to teams_tasks lifecycle/activity/followup/progress/embedding (~30 functions)
- Events: broadcastTeamEvent + task_ticker + subagent announce now carry TenantID
- Cache: agentKeyCache scoped by tenant (agent keys per-tenant, not globally unique)
- Skills: SkillStore interface accepts ctx, SQL filter (is_system OR tenant_id=$N), per-tenant list cache, GrantToAgent includes tenant_id, tenant-scoped file storage
- Files: StorageHandler/FilesHandler/TeamAttachments/teamWorkspaceDir use config.TenantDataDir/TenantTeamDir
- Security: HMAC signed file tokens (file_token.go) replace gateway token in URLs
- Audit: AuditEventPayload carries TenantID for async subscriber tenant scoping
- InboundMessage: subagent/dispatch/validation/session_send propagate TenantID
- Pending messages: DeleteStale scoped by tenant
* fix(security): skip gateway token in URLs with signed file tokens
toFileUrl() now skips appending ?token=GATEWAY_TOKEN when the URL
already contains ?ft= (HMAC signed file token). Prevents gateway
token exposure via browser history, logs, and referrer headers.
* fix(security): stop persisting auth tokens in session media URLs
mediaToMarkdown() now stores clean paths (/v1/files/path) without
any auth tokens. Previously embedded ?token=GATEWAY_TOKEN (or ?ft=)
into markdown which gets persisted in session messages DB.
Frontend toFileUrl() adds auth at render time — tokens never stored.
* fix(security): migration 027 strips leaked gateway tokens from session URLs
Adds cleanup step to tenant foundation migration: removes ?token=xxx
from persisted media URLs in session messages. Old code embedded the
gateway token; new code stores clean paths only.
* fix(security): sign file URLs at delivery time, not persist time
Add SignFileURLs() utility that finds /v1/files/ and /v1/media/ URLs
in content and appends HMAC signed ?ft= tokens before delivery.
Applied at 4 delivery points:
- WS agent events (OnEvent callback in gateway_managed.go)
- WS chat.history response
- WS sessions.preview response
- HTTP /v1/chat/completions response
Sessions store clean paths only. Tokens are generated per-delivery
with 1h TTL — never persisted in DB. Frontend toFileUrl() skips
appending gateway token when ?ft= is already present.
* fix: file token verify path must match signed path (/v1/files/ prefix)
SignFileURLs() signs the full URL path "/v1/files/{path}" but the
verify in files.go auth() was using "/{path}" (without prefix).
HMAC mismatch caused all signed file tokens to return 401.
* fix(security): scope storage size cache per-tenant
sizeCache was a single global entry — all tenants shared one cached
size. Changed to sync.Map keyed by tenantBaseDir so each tenant gets
its own cached size calculation.
* feat(ui): redesign API keys page — table layout + code snippet dialog
Replace card-based API keys list with table layout matching MCP Servers
pattern. Add "API Key Usage" dialog with tabbed code snippets (cURL,
TypeScript, Go) showing gateway connection examples with syntax
highlighting and copy-to-clipboard.
* fix(builtin-tools): seed media tools disabled, fix tenant toggle, add unconfigured warning
- Seed media tools with Enabled=false and no default provider settings
(user must configure provider chain before enabling)
- Fix provider chain form ghost entries: validate provider exists in
tenant before showing (parseInitialEntries new-format path)
- Fix double toggle: show only tenant override OR global toggle, not both
- Fix list API: merge tenant_enabled from builtin_tool_tenant_configs
into response when tenant-scoped (was always null)
- Add ListAll() to BuiltinToolTenantConfigStore for full override map
- Add amber warning banner for enabled media tools missing provider config
* feat(mcp): require_user_credentials setting + KeyValueEditor for user creds
- Add require_user_credentials setting in mcp_servers.settings JSONB
- Backend: skip MCP server in LoadForAgent when user lacks credentials
- Frontend: toggle in MCP form dialog, persisted in settings field
- Redesign MCP user credentials dialog: replace raw Textarea with
KeyValueEditor (sensitive key masking for auth/token/secret fields)
- Add settings to mcpServerAllowedFields for HTTP update
* fix(security): restrict cross-tenant to owner IDs, config to owners only
- Gateway token + non-owner user ID: admin role but tenant-scoped
(no cross-tenant access). Fallback: only "system" is owner when
GOCLAW_OWNER_IDS not configured (fail-closed).
- Config page (WS config.* methods): wrapped with requireCrossTenant
middleware — non-owner admins get permission denied
- Config sidebar link: hidden for non-cross-tenant users
- Logout: clear tenant_id and tenant_hint from localStorage
(prevents tenant scope leak to next user session)
- Refactor: LOCAL_STORAGE_KEYS.TENANT_ID/TENANT_HINT constants
* fix(ui): chat bubble contrast, login logo, tenant no-access UX
- Chat bubble: use --chat-bubble-user CSS var (darker orange, L=0.50/0.52)
with text-white for WCAG AA contrast (~5.5:1)
- Login page: logo h-20 w-20, title text-3xl font-bold
- Tenant selector no-access: shield icon + hint text explaining
user needs admin to add them to a tenant
- Sidebar: GoClaw text uses text-sidebar-primary (brand color)
* feat(contacts): merge/unmerge contacts to tenant users
Add API and UI for linking channel contacts to tenant_users identity,
enabling cross-channel user identification within a tenant.
Backend:
- POST /v1/contacts/merge — link contacts to existing or new tenant_user
- POST /v1/contacts/unmerge — remove merged_id from contacts
- GET /v1/contacts/merged/{id} — list contacts by tenant_user
- GET /v1/tenant-users — list users for current tenant
- Add display_name + metadata columns to tenant_users (migration 27)
- All endpoints enforce tenant isolation via context tenant_id
Frontend:
- Checkbox multi-select on contacts table
- Selection toolbar with Merge/Unmerge buttons
- Merge dialog: link to existing user or create new
- Link2 icon indicator for merged contacts
- i18n: en/vi/zh translations for merge section
* fix(security): add tenant_id to span and embedding_cache inserts
SpanData struct was missing TenantID field — all span inserts failed
with NOT NULL constraint violation after migration 027 dropped defaults.
Fix captures tenant_id from context at emit time (6 call sites in
loop_tracing.go + subagent_tracing.go), then includes it in both
CreateSpan() and BatchCreateSpans() SQL (25→26 columns).
Also fixes embedding_cache writeEmbeddingCache() which was missing
tenant_id in its batch INSERT — same class of bug.
Both use MasterTenantID fallback for backward compatibility.
* feat: Introduce tenant switcher UI and enhance multi-tenant architecture documentation.
* fix(security): enforce tenant scoping, fix session isolation and UI cleanup
- Force cross-tenant admins to always have a concrete tenant_id (default
MasterTenantID) instead of unscoped WithCrossTenant — prevents mismatch
between session listing (no filter) and writes (MasterTenantID fallback)
- Make agent router tenant-aware: Get(ctx, agentID) resolves agent for
the caller's tenant, preventing cross-tenant agent cache collisions
- Fix context.Background() in title goroutine and summarization — now
uses tenant-aware context (WithoutCancel) so titles and compaction
persist to the correct tenant
- Add read-only SessionStore.Get() method; replace GetOrCreate in auth
checks (preview/patch/delete/reset) to prevent phantom session creation
- Inject tenant from channel instance into inbound message processing
- Remove "All Tenants" option from tenant selector, topbar switcher,
and ws-provider auto-select — admin must always operate within a tenant
- Fix contacts page selection toolbar layout shift (always rendered)
- Widen MCP credentials sensitive header regex to catch API_KEY etc.
* fix(security): propagate tenant_id in consumer handlers and background ops
- InjectTeamDispatch: use context.WithoutCancel instead of context.Background
to preserve tenant_id while avoiding cancel propagation from HTTP/WS handlers
- handleTeammateMessage/handleSubagentAnnounce: inject tenant_id from msg
- Add nil guard for outcome.Result to prevent panic on agent-not-found
- Use BroadcastForTenant for EventTeamTaskFailed/Completed/LeaderProcessing
- Remove unnecessary WithCrossTenant in autoSetFollowup (ctx already scoped)
- resolveAgentByKey: accept ctx param for tenant-scoped agent lookup
- pending_messages: use request ctx instead of cross-tenant for GetDefault
* fix(security): tenant-scope EnsureContact and PendingHistory DB operations
- All channel EnsureContact calls now use tenant-scoped ctx instead of
context.Background (whatsapp, slack, discord, telegram, feishu, zalo)
- PendingHistory: add tenantID field, thread through constructors
- All PendingHistory DB ops (load, flush, compact, delete) use tenantCtx()
- Normalize timeouts: 10s for simple queries, 15s for batch writes
* feat(teams): auto-attach media, retry completed tasks, improve tool messages
- Auto-attach workspace media from any tool (create_image/audio/video) to
team tasks via loop-level hook, not just write_file interceptor
- Store absolute paths in team_task_attachments instead of relative
- Extend retry action to support completed tasks (reopen for follow-up)
- Context-aware comment result messages with next-action guidance for
leader vs member roles and task status
- All tool results include task_id for agent follow-up actions
- Use #N "subject" format instead of raw UUIDs in tool messages
* feat(multi-tenant): tenant isolation for media, events, providers and UI
- Tenant-scoped media store, event filter, provider registry
- Tenant header propagation in WS/HTTP clients
- UI: tenant-aware chat messages, markdown renderer improvements
- Protocol: tenant error codes and event definitions
* docs: add multi-tenant architecture documentation
* fix(teams): store absolute paths in team_task_attachments
- AutoAttachWorkspaceFile: use cleanPath consistently instead of raw absPath
- executeAttach: resolve relative paths to absolute via team workspace
- AfterWrite interceptor already uses filepath.Clean (verified)
* fix(teams): attachment download handles both absolute and relative paths
filepath.Join with an absolute att.Path discards the teamBase prefix,
causing path traversal check to fail and download to serve wrong file.
Now checks IsAbs first — uses path directly for new absolute entries,
falls back to legacy join for old relative entries.
* fix(teams): attachment download validates against workspace root not tenant dir
Absolute paths stored in DB don't match TenantTeamDir structure
(master tenant has no tenants/ prefix). Now validates absolute paths
against dataDir (workspace root) instead. Legacy relative paths still
resolve via TenantTeamDir as before. IDOR check on att.TeamID ensures
cross-team isolation.
* fix(teams): attachment download uses workspace root, not data dir
Files are stored under GOCLAW_WORKSPACE/teams/ but handler was passed
dataDir (GOCLAW_DATA_DIR) — completely different directory. Now passes
workspace. Legacy relative paths resolve via {workspace}/teams/{teamID}/{chatID}/{path}.
* fix(security): use HMAC-signed file tokens for attachment downloads
Replace gateway token exposure (?token=) with HMAC-signed short-lived
file tokens (?ft=) for team task attachment downloads — same mechanism
used by chat file URLs.
Backend:
- team_attachments auth: accept ?ft= signed token (priority 1), Bearer (priority 2)
- teams_tasks RPC: sign download_url with HMAC at delivery time
- Add fileTokenSecret to TeamsMethods, thread through wireChannelRPCMethods
Frontend:
- Use server-signed download_url from attachment data instead of ?token=
- Remove useAuthStore dependency from task-detail-dialog
* fix(security): decouple file token signing from gateway token
- Generate random 256-bit HMAC key at startup (crypto/rand, memory-only)
- All file signing/verification uses FileSigningKey() instead of gateway token
- Remove ?token= query param fallback from /v1/files/, /v1/media/, attachments
- Only ?ft= signed tokens and Bearer header accepted for file access
- Reduce file token TTL from 1h to 5min
- Frontend: remove gateway token from all file URLs and imports
- Note: tokens invalidate on restart (acceptable for 5min TTL + WS reconnect)
* fix(ui): use signed download_url for task attachments
- Add download_url to TeamTaskAttachment type
- Use a.download_url (server-signed ?ft=) instead of bare URL
* fix(security): tenant-scope team workspace paths + show user/tenant in topbar
- WorkspaceDir callers now use config.TenantWorkspace() to resolve
tenant-scoped base dir (non-master tenants get workspace/tenants/{slug}/)
- Fixes: all tenants previously wrote to global /app/workspace/teams/
without filesystem-level isolation
- Affected: loop.go (agent run), team_tasks_mutations.go (task creation)
- teams_workspace.go already correct (uses TenantTeamDir)
- UI topbar: show "userId (tenantName)" in user menu
* feat(ui): redesign task detail dialog with improved UX
- Split monolithic 343-line component into 5 focused files
- New header: subject as title, identifier + status badges above
- Metadata grid with soft bg-muted/30 background, priority icons
- Attachments: card-style with mime-type icons + proper download Button
- Description/Result: markdown rendering via MarkdownRenderer
- Comments: avatar circles + markdown rendering for content
- All sections collapsible with chevron + count badge
- Timeline: vertical dot-line pattern, collapsed by default
- Fix kanban card hover layout shift (opacity instead of display toggle)
* fix(security): tenant-scoped workspace paths and tool cache isolation
- Scope team workspace paths to tenant directory
- Add tenant isolation to tool cache and task reads
- Shell deny pattern improvements
- Agent resolver and context file tenant scoping
- Sidebar tenant/user display fix
- Add tests for workspace, boundary, and context file interceptor
* fix(ui): tenant visibility fallback, merge coming-soon, task detail tweaks
- Tenants page: try tenants.list (owner), fall back to tenants.mine
for regular users; hide create button for non-owners
- Merge contacts dialog: add coming-soon banner (i18n en/vi/zh),
disable form and submit button
- Task detail: collapse attachments/comments by default,
guard download_url before rendering
|
||
|
viettranx
|
13139e08ca |
feat: extractive memory fallback when LLM flush fails
Add regex-based extraction as safety net when LLM memory flush returns NO_REPLY or fails. Extracts decisions, key facts, URLs, file paths, dates, and user preferences from conversation history before compaction. - Wire extractiveMemoryFallback into runMemoryFlush (NO_REPLY + LLM error) - Add memStore to Loop for direct memory writes - Use store.MemoryUserID(ctx) for correct shared memory scoping - Limit regex input to last 20 messages for performance - Drop FTS language config (tsv column is stored with 'simple', changing query language without matching stored config breaks search) Closes #328 |
||
|
viettranx
|
90716a2694 |
fix(kg): complete semantic search implementation and shared KG consistency
- Add batch embedding generation in IngestExtraction (was missing, making vector search dead code) - Add BackfillKGEmbeddings for existing entities without embeddings - Add KGUserID() context helper to prevent entity duplication in shared KG mode - Add IsSharedKG handling to DeleteEntity, DeleteRelation, ListEntities, ListAllRelations, PruneByConfidence - Replace O(n²) bubble sort with slices.SortFunc in hybridMergeEntities - Fix vectorSearchEntities duplicate param (reuse same $N for SELECT and ORDER BY) - Update KG tool and memory interceptor to use KGUserID instead of MemoryUserID - Remove unrelated MemoryStore additions from Loop/Resolver (should be separate PR) - Fix comments: "team_id" → "agent-level sharing" to match actual implementation |
||
|
Tyler VoandClaude Opus 4.6
|
1015c9f73c |
feat: add pgvector semantic search to KG entities + team KG sharing
- Add embedding column to kg_entities with HNSW index - Generate entity embeddings during IngestExtraction - Hybrid search in SearchEntities (ILIKE 0.3 + vector 0.7) - Add team_id scope for shared KG across team members - Add IsSharedKG/WithSharedKG context helpers Closes #327 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Viet Tran
|
84650c5c14 |
feat(teams): attachments refactor, semantic search, improved prompting (#310)
* feat(teams): refactor attachments, remove team_message, add task comments UI Major team system refactoring: - Drop team_workspace_files, team_workspace_file_versions, team_workspace_comments, team_messages tables; replace team_task_attachments with path-based schema - Add denormalized comment_count/attachment_count on team_tasks for dashboard perf - Auto-track file writes as task attachments via WorkspaceInterceptor - Remove team_message tool entirely (tool, store, i18n, builtin_tools, MCP bridge) - Members communicate via task comments; approve/reject use comments for audit trail - Add commented/new_task notification types to TeamNotifyConfig - Enrich task completion announce with member comments - User-created tasks stay pending (backlog) — no auto-assign to leader - Configurable member request tasks (member_requests.enabled in team settings) - Structured task description template in TEAM.md for v2 leads - HTTP attachment download endpoint with IDOR + path traversal protection - Web UI: count badges on task list, comments section with input, download button - Team settings UI: completed/commented/new_task toggles, member requests section * feat(teams): priority dispatch, compact prompting, realtime comments - Priority dispatch: DispatchUnblockedTasks dispatches only 1 task per owner per round (highest priority first). Fixes cancel bug where CancelSession killed innocent queued tasks. - Prompt rework: Replace verbose Task Decomposition (25 lines) with compact Task Planning (8 lines). Add explicit UUID warning and sequencing guidance for weak models (Qwen, MiniMax). - Recent comments in dispatch: buildRecentCommentsSummary appends 3 most recent comments to re-dispatched tasks (reject, retry, stale). - Enrich comment event payload with TaskNumber, Subject, CommentText (truncated 500 runes, UTF-8 safe). - UI: Board subscribes to TEAM_TASK_COMMENTED for realtime comment_count badge updates. Task detail dialog auto-refreshes comments on event. - Tool description hint: guide models to write self-contained task descriptions with clear objectives and context. * perf(teams): add ListRecentTaskComments with SQL LIMIT Dispatch only needs 3 most recent comments — avoid fetching all. New ListRecentTaskComments(ctx, taskID, limit) uses ORDER BY DESC LIMIT N then reverses to chronological order. * feat(teams): add subject embedding for semantic task search + improve prompting - Add vector(1536) embedding column to team_tasks with HNSW index - Implement hybrid search: FTS (0.3) + cosine similarity (0.7) with graceful fallback - Auto-generate embeddings on task create/update, backfill existing tasks on startup - Wire embedding provider into PGTeamStore via gateway_setup - Change FTS from OR to AND with prefix matching for precise keyword search - Reduce search page size from 30 to 5 to save tokens - Rename migration 000023 → 000024, bump RequiredSchemaVersion to 24 - Update TEAM.md hints: prefer search over list, batch task creation with blocked_by - Add anti-pattern examples to prevent sequential task creation |
||
|
viettranx
|
6595c0185a |
refactor: merge file writers into config permissions + agent hard delete
Migration 000023: - Migrate group_file_writers data into agent_config_permissions (config_type='file_writer') - Add FK CASCADE/SET NULL on 16 tables for agent hard delete - Clean up soft-deleted zombie agents - ALTER scope VARCHAR(100) → VARCHAR(255) - Partial unique index on agent_key (allows reuse after delete) Store layer: - Remove GroupWriterCache, GroupFileWriterData, 5 AgentStore methods - Add CheckFileWriterPermission (fail-open), ListFileWriters (cached) - Add scope filter to ConfigPermissionStore.List() Tools: Replace GroupWriterCache with ConfigPermissionStore in read_file, write_file, edit, cron, context_file_interceptor Agent loop: buildGroupWriterPrompt uses ListFileWriters + metadata parsing Channels: Telegram commands rewritten to use configPermStore.Grant/Revoke/List HTTP API: Writer endpoints rewired to configPermStore (same response shapes) Hard delete: agents.go DELETE FROM instead of soft delete |
||
|
viettranx
|
08a2d95c0c |
feat: agent heartbeat system — periodic proactive check-ins (#245)
Phase 1 (Core):
- Migration 000022: agent_heartbeats, heartbeat_run_logs, agent_config_permissions tables
- HeartbeatStore + ConfigPermissionStore interfaces with PG implementations
- HeartbeatTicker: background poll → active hours filter → queue-aware skip → run → smart suppression → deliver/log
- Heartbeat tool: status/get/set/toggle/set_checklist/get_checklist/test/logs actions
- Permission check with wildcard scope matching + TTL cache (60s)
- RPC methods: heartbeat.get/set/toggle/test/logs/checklist.get/checklist.set
- HEARTBEAT.md routed via context file interceptor (read/write for both open + predefined agents)
- Session keys: agent:{id}:heartbeat or agent:{id}💓{ts} (isolated)
- PromptMinimal for heartbeat sessions (like cron/subagent)
- Event broadcasting + cache invalidation via bus (heartbeat + config_perms)
- Gateway wiring: ticker init, event wiring, graceful shutdown
Phase 2 (Integration):
- wakeMode: CronPayload.WakeHeartbeat triggers heartbeat after cron job completes
- Queue-aware: Scheduler.HasActiveSessionsForAgent() skips busy agents
- Stagger: deterministic FNV offset spreads heartbeats across interval
- lightContext: RunRequest.LightContext skips context files, only injects checklist
- System prompt distinguishes cron (user-scheduled tasks) vs heartbeat (autonomous monitoring)
|
||
|
viettranx
|
27fb900510 |
refactor(tools): remove workspace_read/workspace_write, use file tools for team workspace
Remove dedicated workspace tools in favor of making existing file tools (read_file, write_file, list_files, edit) team-workspace-aware. - Delete workspace_tool_read.go and workspace_tool_write.go - Clean up workspace_dir.go: export WorkspaceDir, remove dead code (workspaceRelPath, sanitizeFilePath, inferMimeType, templates, etc.) - Remove workspace tool registration from gateway_managed.go - Remove workspace tool references from policy, subagent, MCP bridge - Add PathAllowable/PathDenyable to types.go for interface abstraction |
||
|
Goon
|
75c570e951 |
feat(security): credentialed exec + HTTP RBAC + API key cache (#197)
- Secure CLI credential injection via AES-256-GCM encrypted env vars - API key management with fine-grained RBAC scopes - resolveAuth/requireAuth middleware across all 25+ HTTP handlers - In-memory API key cache with TTL, negative caching, pubsub invalidation - Sandbox-first execution (fails if unavailable, no silent fallback) - Credential scrubbing, constant-time token comparison, Admin-only CLI creds - SQL migration 000020: secure_cli_binaries + api_keys tables - 14 unit tests for cache and RBAC with race detector Closes #197 |
||
|
Viet Tran
|
9a9744077e |
refactor(teams): v2 system cleanup — remove legacy tools, fix followup, add events API (#210)
Major refactoring of the team system with multiple improvements:
## Removed legacy delegation tools
- Delete `delegate.go`, `delegate_async.go`, `delegate_sync.go`, `delegate_events.go`,
`delegate_policy.go`, `delegate_prep.go`, `delegate_state.go`, `delegate_search_tool.go`
- Delete `evaluate_loop_tool.go`, `handoff_tool.go`
- Remove all references and registrations from tool manager and policy
- Clean up TEAM_PLAYBOOK_IDEAS.md and TEAM_SYSTEM.md (moved to docs)
## Rename await_reply → ask_user
- Rename action `await_reply` → `ask_user`, `clear_followup` → `clear_ask_user`
- Rename functions `executeAwaitReply` → `executeAskUser`, `executeClearFollowup` → `executeClearAskUser`
- Update system prompt with stronger wording to prevent model misuse
- Model was confusing "await_reply" with general waiting; "ask_user" is unambiguous
## Fix auto-followup false positives
- Add `HasActiveMemberTasks(ctx, teamID, excludeAgentID)` store method
- Guard `autoSetFollowup()` in consumer: skip when lead has active member tasks
- Prevents auto-followup when lead is orchestrating teammates (not waiting for user)
## Task identifier zero-padding
- Change format from `T-1-xxxx` → `T-001-xxxx` (3-digit minimum)
## Refactor workspace WS handlers to filesystem-only
- Rewrite `teams.workspace.list/read/delete` to use pure filesystem (os.ReadDir/ReadFile/Remove)
- Remove DB dependency from workspace WS handlers
- Consistent with storage handler and workspace tools
- Simplify TeamWorkspaceFile type and frontend hook
## Add team events listing API
- New WS method `teams.events.list` with team_id, limit, offset params
- New HTTP endpoint `GET /v1/teams/{id}/events` with bearer auth
- New `ListTeamEvents(ctx, teamID, limit, offset)` store method
- JOIN with team_tasks for team-wide event filtering
## Extract team access policy
- New `team_access_policy.go` — centralized team tool access control
## Migration 000019: team_id columns
- Add team_id foreign key columns to relevant tables
## Other improvements
- Add team_id propagation through agent loop, tracing, sessions
- Update i18n locale files (en/vi/zh) for new tool labels
- Update frontend builtin-tools page and require-setup component
- Bump RequiredSchemaVersion for migration 000019
|
||
|
Goonandviettranx
|
5e2fa395c7 |
feat(providers): add ACP provider for external coding agents (#190)
* feat(providers): add ACP provider for orchestrating external coding agents (#189) Implement native Go ACP (Agent Client Protocol) client as a new Provider. Enables GoClaw to orchestrate any ACP-compatible agent (Claude Code, Codex CLI, Gemini CLI) as a subprocess via JSON-RPC 2.0 over stdio. - Add bidirectional JSON-RPC 2.0 transport over stdio pipes - Add subprocess process pool with idle TTL reaping and crash recovery - Add ACP session lifecycle (initialize, session/new, session/prompt) - Add tool bridge for agent-initiated fs/terminal/permission requests - Add workspace sandboxing, shell deny patterns, and env var filtering - Wire config-based and DB-based provider registration paths - Export DefaultDenyPatterns from tools package for reuse * feat(providers): add changelog entry for ACP provider integration * fix(tools): prevent workspace traversal bypass via /tmp/ fallback in resolveMediaPath Reject paths containing ".." in the isInTempDir fallback to prevent workspace escape where traversal path still resolves inside /tmp/. * fix(tools): block workspace-sibling paths in resolveMediaPath /tmp/ fallback When workspace is inside /tmp/, traversal paths like workspace/../X resolve to /tmp/ siblings that pass isInTempDir. Reject paths inside the workspace parent directory to prevent this escape. * feat(providers): add ACP provider web UI and live reload via pubsub Web UI for creating/editing ACP providers with dedicated form fields (binary, args, idle TTL, permission mode, work directory). ACP providers now update immediately without gateway restart via cache invalidation pubsub pattern. Frontend: - New ACPSection form component with i18n (en/vi/zh) - Provider form dialog integration with ACP state management - ACP type badge on providers list page - Settings field added to provider TypeScript types Backend: - ACP models handler (claude/codex/gemini) without API key requirement - Binary path validation + LookPath verification in verify handler - Provider CRUD emits cache.invalidate events via msgBus - Subscriber in gateway_managed.go re-registers ACP providers from DB - ACP core improvements from code review (helpers, jsonrpc, process, terminal, tool_bridge) --------- Co-authored-by: viettranx <viettranx@gmail.com> |
||
|
Viet TranandNam Nguyen Ngoc
|
1a42dc93a6 |
feat(teams): team system v2 with bug fixes, workspace scope, versioning, and prompt optimization (#183)
* feat(workspace): add team shared workspace for file collaboration
- Add workspace_write and workspace_read tools for agents to share files across team members
- Create team_workspaces DB table with migration 000017 (file metadata, pinning, tags)
- Implement PostgreSQL store layer for workspace CRUD operations
- Add RPC handlers for workspace list/read/delete from web UI
- Build React workspace tab with file listing, content preview, and delete
- Propagate workspace channel/chatID scope through delegation chain
- Auto-allow workspace tools in agent tool policy when agent belongs to a team
- Inject team workspace guidance into system prompt for team agents
- Add /reset command handler for clearing session history
- Harden MCP bridge context middleware to reject headers when no gateway token
- Add i18n strings for workspace UI in en/vi/zh locales
* feat(teams): add comprehensive task management with followup reminders and recovery
- Add task followup/reminder system with auto-set on lead agent reply and auto-clear when user responds on channel
- Add task recovery ticker to re-dispatch stale/pending tasks periodically
- Add task scopes, filtering by status/channel/chatID, and task events
- Add WS RPC handlers for task CRUD, assignments, comments, events, and bulk operations (teams_tasks.go)
- Add task detail dialog, settings UI for followup config, and scope filtering in web dashboard
- Add migrations 000018 (team_tasks_v2) and 000019 (task_followup)
- Extend team_tasks_tool with await_reply, clear_followup actions
- Auto-complete/fail team tasks when delegate agent finishes
- Add workspace file listing and team tool manager enhancements
* docs(teams): add team system architecture and playbook ideas documentation
- Add TEAM_SYSTEM.md with full architecture design covering task management, shared workspace, and delegation engine subsystems
- Add TEAM_PLAYBOOK_IDEAS.md outlining future team coordination layers (playbook, member capabilities, auto-learned patterns)
- Document data models, status flows, tool actions, followup reminder system, task ticker, execution locking, and workspace scope model
* fix(teams): resolve 6 critical bugs in team task system
- Fix unblock SQL: check array_length after array_remove (not before)
- Enforce single-team leadership in team creation
- Add requireLead() for approve/reject tool actions
- Validate cross-team dependency references in blocked_by
- Add team_id to handoff route for multi-team isolation
- Set blocked_by DEFAULT '{}' to prevent NULL array issues
* refactor(workspace): use stable userID as scope key instead of connection UUID
Workspace scope changed from (team_id, channel, chat_id) to (team_id, userID).
Fixes workspace fragmentation across WS tab refreshes and reconnections.
* feat(teams): add V1/V2 versioning with feature gating and optimized prompts
- IsTeamV2() helper gates advanced features (locking, followup, review, audit)
- V2 tool actions rejected for V1 teams with clear error message
- Ticker, gateway consumer, delegation hooks respect version flag
- TEAM.md renders v1/v2 sections conditionally
- Tool descriptions and params optimized (~38% token reduction)
- UI: version toggle in settings, V2 Beta badge, conditional rendering
- i18n: version modal keys for en/vi/zh
* fix(migration): use VARCHAR(255) for user ID columns and add metadata JSONB
- assignee_user_id, user_id, actor_id: TEXT → VARCHAR(255)
- Add metadata JSONB to team_task_comments and team_task_attachments
---------
Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
|