# GoClaw Gateway PostgreSQL multi-tenant AI agent gateway with WebSocket RPC + HTTP API. ## Language Always respond in the same language as the user's prompt. If the user writes in Vietnamese, respond in Vietnamese. If in English, respond in English. Match the user's language naturally. ## Tech Stack **Backend:** Go 1.26, Cobra CLI, gorilla/websocket, pgx/v5 (database/sql, no ORM), golang-migrate, go-rod/rod, telego (Telegram) **Web UI:** React 19, Vite 6, TypeScript, Tailwind CSS 4, Radix UI, Zustand, React Router 7. Located in `ui/web/`. **Use `pnpm` (not npm).** **Desktop UI:** React 19, Vite 6, TypeScript, Tailwind CSS 4, Zustand, Framer Motion. Located in `ui/desktop/frontend/`. **Use `pnpm`.** **Desktop App:** Wails v2 (`//go:build sqliteonly`). Located in `ui/desktop/`. Embeds gateway + React frontend in single binary. **Database:** PostgreSQL 18 with pgvector (standard). SQLite via `modernc.org/sqlite` (desktop/lite). Raw SQL with `$1, $2` (PG) or `?` (SQLite) positional params. Nullable columns: `*string`, `*time.Time`, etc. ## Project Structure ``` cmd/ CLI commands, gateway startup, onboard wizard, migrations internal/ ├── agent/ Agent loop (think→act→observe), router, resolver, input guard ├── bootstrap/ System prompt files (SOUL.md, IDENTITY.md) + seeding + per-user seed ├── bus/ Event bus system ├── cache/ Caching layer ├── channels/ Channel manager: Telegram, Feishu/Lark, Zalo, Discord, WhatsApp │ └── whatsapp/ Native WhatsApp via whatsmeow (v3) ├── config/ Config loading (JSON5) + env var overlay ├── consolidation/ Memory consolidation workers (episodic, semantic, dreaming) (v3) ├── crypto/ AES-256-GCM encryption for API keys ├── cron/ Cron scheduling (at/every/cron expr) ├── edition/ Edition system (Lite, Standard) with feature gating ├── eventbus/ Domain event bus with worker pool, dedup, retry (v3) ├── gateway/ WS + HTTP server, client, method router │ └── methods/ RPC handlers (chat, agents, sessions, config, skills, cron, pairing) ├── hooks/ Hook system for extensibility ├── http/ HTTP API (/v1/chat/completions, /v1/agents, /v1/skills, etc.) ├── i18n/ Message catalog: T(locale, key, args...) + per-locale catalogs (en/vi/zh) ├── knowledgegraph/ Knowledge graph storage and traversal ├── mcp/ Model Context Protocol bridge/server ├── media/ Media handling utilities ├── memory/ Memory system (pgvector) ├── oauth/ OAuth authentication ├── orchestration/ Orchestration primitives: BatchQueue[T] generic, ChildResult, media conversion (v3) ├── permissions/ RBAC (admin/operator/viewer) ├── pipeline/ 8-stage agent pipeline (context→history→prompt→think→act→observe→memory→summarize) ├── providers/ LLM providers: Anthropic (native HTTP+SSE), OpenAI-compat (HTTP+SSE), DashScope (Alibaba Qwen), Claude CLI (stdio+MCP bridge), ACP (Anthropic Console Proxy), Codex (OpenAI) ├── providerresolve/ Provider adapter + model registry with forward-compat resolver ├── sandbox/ Docker-based code execution sandbox ├── scheduler/ Lane-based concurrency (main/subagent/cron) ├── sessions/ Session management ├── skills/ SKILL.md loader + BM25 search ├── store/ Store interfaces + implementations (PostgreSQL, SQLite) │ ├── base/ Shared store abstractions: Dialect interface, helpers (NilStr, BuildMapUpdate, BuildScopeClause) │ ├── pg/ PostgreSQL implementations (database/sql + pgx/v5) │ └── sqlitestore/ SQLite implementations (modernc.org/sqlite) ├── tasks/ Task management ├── tokencount/ tiktoken BPE token counting ├── tools/ Tool registry, filesystem, exec, web, memory, subagent, MCP bridge, delegate ├── tracing/ LLM call tracing + optional OTel export (build-tag gated) ├── tts/ Text-to-Speech (OpenAI, ElevenLabs, Edge, MiniMax) ├── updater/ Desktop auto-update checker (Lite edition) ├── upgrade/ Database schema version tracking ├── vault/ Knowledge Vault with wikilinks, hybrid search, FS sync ├── workspace/ WorkspaceContext resolver for 6 scenarios pkg/protocol/ Wire types (frames, methods, errors, events) pkg/browser/ Browser automation (Rod + CDP) migrations/ PostgreSQL migration files ui/web/ React SPA (pnpm, Vite, Tailwind, Radix UI) ui/desktop/ Wails v2 desktop app (React frontend + embedded gateway) ``` ## Key Patterns - **Store layer:** Interface-based (`store.SessionStore`, `store.AgentStore`, etc.) with shared Dialect pattern in `store/base/`. PostgreSQL (`pg/`) and SQLite (`sqlitestore/`) implementations use `database/sql` + `pgx/v5/stdlib` + sqlx, raw SQL, `BuildMapUpdate()` and `BuildScopeClause()` helpers - **Agent types:** `open` (per-user context, 7 files) vs `predefined` (shared context + USER.md per-user) - **Agent identity:** Dual-identity pattern (agent_key vs UUID) applies to agents, teams, tenants. Rule: UUID for DB/FK/events, agent_key for logs/paths/UI. See `docs/agent-identity-conventions.md` - **Context files:** `agent_context_files` (agent-level) + `user_context_files` (per-user), routed via `ContextFileInterceptor` - **Providers:** Anthropic (native HTTP+SSE), OpenAI-compat (HTTP+SSE), DashScope (Alibaba Qwen), Claude CLI (stdio+MCP bridge), ACP (Anthropic Console Proxy), Codex (OpenAI). All use `RetryDo()` for retries. Loads from `llm_providers` table with encrypted API keys. ProviderAdapter enables pluggable implementations with ModelRegistry forward-compat resolver. Shared SSEScanner in `providers/sse_reader.go` for streaming providers - **Pipeline:** 8-stage loop (context→history→prompt→think→act→observe→memory→summarize) with pluggable callbacks, always-on execution path - **DomainEventBus:** Typed events with worker pool, dedup, retry. Used by consolidation pipeline and memory workers - **3-tier memory:** Working (conversation) → Episodic (session summaries) → Semantic (KG). Progressive loading L0/L1/L2 with auto-inject for L0 - **Knowledge Vault:** Document registry + [[wikilinks]] + hybrid search, query layer above existing stores, FS sync, unified search - **Context propagation:** `store.WithAgentType(ctx)`, `store.WithUserID(ctx)`, `store.WithAgentID(ctx)`, `store.WithLocale(ctx)`, `store.WithTenantID(ctx)` - **Request middleware:** Composable chain (cache, service tier, request guards), zero-alloc fast path for hot operations - **Self-evolution:** Metrics → suggestions → auto-adapt. 3 progressive stages: metrics collection, suggestion analysis, guardrail-protected apply/rollback - **Orchestration:** Delegate tool for inter-agent task delegation with agent_links, 3 delegation modes (auto/explicit/manual), token-aware work distribution. BatchQueue[T] generic for result aggregation - **WebSocket protocol:** Frame types `req`/`res`/`event`. First request must be `connect` - **Config:** JSON5 at `GOCLAW_CONFIG` env. Secrets in `.env.local` or env vars, never in config.json - **Security:** Rate limiting, input guard (detection-only), CORS, shell deny patterns, SSRF protection, path traversal prevention, AES-256-GCM encryption. All security logs: `slog.Warn("security.*")` - **Telegram formatting:** LLM output → `SanitizeAssistantContent()` → `markdownToTelegramHTML()` → `chunkHTML()` → `sendHTML()`. Tables rendered as ASCII in `
` tags
- **i18n:** Web UI uses `i18next` with namespace-split locale files in `ui/web/src/i18n/locales/{lang}/`. Backend uses `internal/i18n` message catalog with `i18n.T(locale, key, args...)`. Locale propagated via `store.WithLocale(ctx)` — WS `connect` param `locale`, HTTP `Accept-Language` header. Supported: en (default), vi, zh. New user-facing strings: add key to `internal/i18n/keys.go`, add translations to all 3 catalog files. New UI strings: add key to all 3 locale dirs. Bootstrap templates (SOUL.md, etc.) stay English-only (LLM consumption).

## Running

```bash
go build -o goclaw . && ./goclaw onboard && source .env.local && ./goclaw
./goclaw migrate up                 # DB migrations
# Integration tests (requires pgvector pg18 on port 5433)
docker run -d --name pgtest -p 5433:5432 -e POSTGRES_PASSWORD=test -e POSTGRES_DB=goclaw_test pgvector/pgvector:pg18
TEST_DATABASE_URL="postgres://postgres:test@localhost:5433/goclaw_test?sslmode=disable" \
  go test -v -tags integration ./tests/integration/

# Layered tests
make test-invariants  # P0 - tenant isolation (blocking)
make test-contracts   # P1 - API schemas (requires server)
make test-scenarios   # P2 - user journeys (requires server)
make test-critical    # P0 + P1 (pre-merge)

cd ui/web && pnpm install && pnpm dev   # Web dashboard (dev)

# Desktop (Wails + SQLite)
cd ui/desktop && wails dev -tags sqliteonly  # Dev mode with hot reload (direct)
make desktop-dev                             # Same as above via Makefile
make desktop-build VERSION=0.1.0             # Build .app (macOS) or .exe (Windows)
make desktop-dmg VERSION=0.1.0               # Create .dmg installer (macOS only)
```

## CI/CD & Releases

### Workflows

| Workflow | Trigger | Purpose |
|----------|---------|---------|
| `ci.yaml` | push main, PR→main/dev | Go build+test+vet, Web build |
| `release.yaml` | tag `v[0-9]+.[0-9]+.[0-9]+` | Binaries + Docker (4 variants + web) + Discord |
| `release-beta.yaml` | tag `v*-beta*` / `v*-rc*` | Beta binaries + Docker + GitHub prerelease |
| `release-desktop.yaml` | tag `lite-v*` | Desktop app (macOS+Windows), auto prerelease for `-beta`/`-rc` tags |

### Creating Releases

**Standard release** — manual tag push after merging `dev` → `main`:
```bash
git tag v3.0.0 && git push origin v3.0.0
```

**Beta release** (from dev):
```bash
git tag v2.67.0-beta.1 && git push origin v2.67.0-beta.1   # standard beta
git tag lite-v1.2.0-beta.1 && git push origin lite-v1.2.0-beta.1  # lite beta
```

**Desktop release:**
```bash
git tag lite-v1.1.0 && git push origin lite-v1.1.0   # stable
git tag lite-v1.1.0-beta.1 && git push origin lite-v1.1.0-beta.1  # beta (prerelease)
```

### Docker Images

Published to GHCR (`ghcr.io/nextlevelbuilder/goclaw`) and Docker Hub (`digitop/goclaw`).

| Variant | Tag | Contents |
|---------|-----|----------|
| latest | `:latest`, `:vX.Y.Z` | Backend + web UI + Python |
| base | `:base`, `:vX.Y.Z-base` | Backend only, no UI/runtimes |
| full | `:full`, `:vX.Y.Z-full` | All runtimes + skills pre-installed |
| otel | `:otel`, `:vX.Y.Z-otel` | Latest + OpenTelemetry tracing |
| web | `-web:latest` | Standalone web UI (Nginx) |
| beta | `:beta`, `:vX.Y.Z-beta.N` | Beta builds from dev |

### Tag Pattern Safety

- `release.yaml`: tag-triggered (`v[0-9]+.[0-9]+.[0-9]+`) — clean semver only, no beta/rc
- `release-beta.yaml`: tag-triggered (`v*-beta*`, `v*-rc*`) — never matches clean semver
- `release-desktop.yaml`: tag-triggered (`lite-v*`) — `lite-` prefix prevents overlap
- No workflow triggers overlap — each tag pattern is distinct. Merging to `main` only triggers CI, not release

## Desktop Edition (Lite)

- **Build tag:** `//go:build sqliteonly` — desktop binary includes only SQLite, no PostgreSQL
- **Edition system:** `internal/edition/edition.go` — `Lite` preset auto-selected for SQLite backend. Check `edition.Current()` for feature limits
- **Entry point:** `ui/desktop/main.go` + `ui/desktop/app.go` — Wails bindings, embedded gateway
- **Secrets:** OS keyring (`go-keyring`) with file fallback at `~/.goclaw/secrets/`
- **Data dir:** `~/.goclaw/data/` (SQLite DB, configs)
- **Workspace:** `~/.goclaw/workspace/` (agent files, team workspace)
- **Port:** 18790 (localhost only), configurable via `GOCLAW_PORT`
- **WS params:** All WS method params use **camelCase** (`teamId`, `taskId`, `sessionKey`) — match Go struct `json:"..."` tags
- **Version:** `cmd.Version` set via `-ldflags` at build time. Frontend calls `wails.getVersion()`
- **Auto-update:** `internal/updater/updater.go` checks GitHub Releases for `lite-v*` tags. Frontend `UpdateBanner` shows notification
- **Releases:** Tag `lite-v*` triggers `.github/workflows/release-desktop.yaml` → builds macOS (arm64+amd64) + Windows → GitHub Release
- **Install scripts:** `scripts/install-lite.sh` (macOS), `scripts/install-lite.ps1` (Windows PowerShell)
- **Lite limits:** 5 agents, 1 team, 5 members, 50 sessions. No channels, heartbeat, file storage UI, skill self-manage, KG, RBAC, multi-tenant
- **Tool gating:** `TeamActionPolicy` in `internal/tools/team_action_policy.go` — lite blocks comment/review/approve/reject/attach/ask_user. `skill_manage`/`publish_skill` not registered in lite
- **File serving:** 2-layer path isolation in `internal/http/files.go` — workspace boundary (all editions) + tenant scope (standard only with RBAC)

## Plan Verification Rules

Apply before finalizing any multi-phase plan. Trust-but-verify between scout → planner → final plan.

1. **Verify factual claims against code** — re-grep/re-count every number, path, endpoint. Don't copy from scout summaries.
2. **Re-scout on scope change** — if phase promotes from deferred → active, re-scout. Don't reuse brainstorm summary.
3. **Grep delete scope deep** — `grep -rn '' .` whole repo. Stubs often have refs in catalogs/routing/switch cases. Enumerate ALL sites in todo.
4. **Verify external API endpoints via `docs-seeker`** — before writing endpoint into plan. Sibling APIs often use different roots.
5. **Cross-phase gates explicit** — "Phase N-1 merged + tests green" in phase Context. Execution order alone ≠ enforcement.
6. **Scout desktop and web separately** — `ui/desktop/frontend/` ≠ `ui/web/`. Different structure, i18n namespaces, test framework presence.
7. **i18n keys ordering** — add key + 3 catalogs as explicit todo step BEFORE handler code. Missing key = runtime crash.
8. **Signature-change callers enumeration** — grep + list all callers explicitly. "Update all callers" insufficient.
9. **Zero-coverage characterization test = blocker step** — write byte/request-body fixture test BEFORE migration. Not "recommended".
10. **Context key style convention** — check existing `context.go` pattern before introducing new key types. Mixed = code smell.
11. **Alias/shim coverage** — enumerate ALL exported symbols via `go doc `. Add compile-time signature guards.
12. **Verify pass MANDATORY after rewrite** — spawn fresh Explore/grep to audit planner output. Don't trust self-validation.
13. **No fabricated Go identifiers** — every type/function/package referenced in plan must be grep-verified to exist (`grep -rn '^func \|^type ' `). Plausible-sounding APIs (`Keyring`, `StartSpan`, `Validator`, `security.SSRF`) are RED FLAGS — conventions differ per codebase. Rule of thumb: if you can't cite `file_path:line_number` for a symbol, you're fabricating. Apply especially when planner says "reuse existing X" — re-verify X exists before writing into plan.
14. **Plausible-but-wrong API families** — watch for: OTel-style `StartSpan/EndSpan` (codebase may be emit-based); wrapper types like `Keyring/Vault/Manager` (codebase may use free functions); `internal/security` or `internal/auth` packages (often scattered instead of centralized). When unsure, `go doc ` lists actual exported surface.

**Pattern to avoid:** user asks → planner writes → report "done".
**Safer pattern:** user asks → scout → planner writes → audit-verify → report.

**Concrete red-team practice:** After planner completes, run `code-reviewer`/`brainstormer` agent in audit mode with explicit instruction "spot-check 15+ factual claims against live codebase". Caught fabricated `internal/security/`, `crypto.Keyring`, `tracing.StartSpan` in Agent Hooks plan (see `plans/260414-2229-agent-hooks-system/reports/audit-260414-2310-plan-review.md`).

## Post-Implementation Checklist

After implementing or modifying Go code, run these checks:

```bash
go fix ./...                        # Apply Go version upgrades (run before commit)
go build ./...                      # Compile check (PG build)
go build -tags sqliteonly ./...     # Compile check (Desktop/SQLite build)
go vet ./...                        # Static analysis
go test -race ./tests/integration/  # Integration tests with race detector
```

Go conventions to follow:
- Use `errors.Is(err, sentinel)` instead of `err == sentinel`
- Use `switch/case` instead of `if/else if` chains on the same variable
- Use `append(dst, src...)` instead of loop-based append
- Always handle errors; don't ignore return values
- **Migrations (dual-DB):** PostgreSQL and SQLite have **separate migration systems**. When adding schema changes: (1) PG: add SQL in `migrations/` + bump `RequiredSchemaVersion` in `internal/upgrade/version.go`. (2) SQLite: update `internal/store/sqlitestore/schema.sql` (full schema for fresh DBs) + add incremental patch in `schema.go` `migrations` map + bump `SchemaVersion` constant. **Always update both** — missing SQLite migrations cause desktop edition to crash on startup
- **i18n strings:** When adding user-facing error messages, add key to `internal/i18n/keys.go` and translations to `catalog_en.go`, `catalog_vi.go`, `catalog_zh.go`. For UI strings, add to all locale JSON files in `ui/web/src/i18n/locales/{en,vi,zh}/`
- **SQL safety:** When implementing or modifying SQL store code (`store/pg/*.go`), always verify: (1) All user inputs use parameterized queries (`$1, $2, ...`), never string concatenation — prevents SQL injection. (2) Queries are optimized — no N+1 queries, no unnecessary full table scans. (3) WHERE clauses, JOINs, and ORDER BY columns use existing indices — check migration files for available indexes
- **DB query reuse:** Before adding a new DB query for key entities (teams, agents, sessions, users), check if the same data is already fetched earlier in the current flow/pipeline. Prefer passing resolved data through context, event payloads, or function params rather than re-querying. Duplicate queries waste DB resources and add latency
- **Solution design:** When designing a fix or feature, identify the root cause first — don't just patch symptoms. Think through production scenarios (high concurrency, multi-tenant isolation, failure cascades, long-running sessions) to ensure the solution holds up. Prefer explicit configuration over runtime heuristics. Prefer the simplest solution that addresses the root cause directly
- **Tenant-scope guards on admin writes:** `RoleAdmin` is not a tenant check. Writes to **global** tables (no `tenant_id` column — e.g. `builtin_tools`, disk config, package mgmt) must gate with `http.requireMasterScope` / WS `requireMasterScope(requireOwner(...))`. Writes to **tenant-scoped** tables must gate with `http.requireTenantAdmin` + SQL `WHERE tenant_id = $N`. Shared predicate: `store.IsMasterScope(ctx)`. See `CONTRIBUTING.md` → "Tenant-scope guards" for the full decision table and anti-patterns.

## Mobile UI/UX Rules

When implementing or modifying web UI components, follow these rules to ensure mobile compatibility:

- **Viewport height:** Use `h-dvh` (dynamic viewport height), never `h-screen`. `h-screen` causes content to hide behind mobile browser chrome and virtual keyboards
- **Input font-size:** All ``, `