mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-14 16:17:25 +00:00
Duy /zuey/
e589545ff5
* feat(packages): unify Packages & CLI Credentials into tabs + per-grant env overrides
Merge /cli-credentials screen into /packages as a tab, redesign Packages page
with Radix Tabs (System/Python/Node/GitHub/CLI Credentials) + sticky Runtimes
header. Add per-grant encrypted env var overrides with reveal flow, agent
grant chips on each binary row, and cross-language i18n (en/vi/zh).
Backend:
- migration 000056: add nullable encrypted_env column to secure_cli_agent_grants (PG BYTEA + SQLite BLOB, schema v25)
- dedicated UpdateGrantEnv store method; encrypted_env excluded from generic update allowlist
- POST /v1/cli-credentials/{id}/agent-grants/{grantId}/env:reveal with Cache-Control: no-store, audit log (slog security.cli_credential.env.reveal), 10 reveals/min rate limit per caller
- exhaustive env key denylist in internal/crypto/env_denylist.go (PATH, HOME, LD_PRELOAD, DYLD_/GOCLAW_/LD_ prefixes, etc.)
- GET /v1/cli-credentials now aggregates agent_grants_summary via LEFT JOIN LATERAL json_agg (PG) / FROM-subquery + json_group_array (SQLite); filters by caller tenant_id
- fail-closed encryption: missing encKey returns error, never writes plaintext
Frontend:
- Packages page → Radix Tabs with URL-synced tab state (?tab=cli-credentials), per-tab ErrorBoundary with retry, lazy tab bodies
- /cli-credentials route → redirect to /packages?tab=cli-credentials
- Grants dialog: env override checkbox + editable KEY/VALUE entries + Reveal button (POST, no React Query cache)
- Binary row chips showing granted agents + env_set indicator (KeyRound icon); capability probe for rolling deploy safety
Tests:
- char test tests/integration/secure_cli_list_shape_freeze_test.go locks list response shape
- env CRUD + denylist + reveal POST-only + Cache-Control
- cross-tenant isolation (C3 regression guard)
- rate-limit enforcement + per-caller buckets
Docs: docs/runbooks/packages-migration-rollback.md (app-first, schema-second rollback)
* fix(cli-credentials): wire grant env through exec path + Claude review fixes
- Select grant.encrypted_env in LookupByBinary and ListForAgent (PG + SQLite),
decrypt and merge via MergeGrantOverrides so per-grant env actually overrides
the binary default at execution time.
- Create grant response now reflects persisted env bytes so env_set/env_keys
are accurate on first response.
- Validate binaryID as UUID in env:reveal handler; audit logs use UUID.
- Expand FE denylist to match internal/crypto/env_denylist.go and add prefix
check (DYLD_, GOCLAW_, LD_).
- Remove dead grantUpdateRequest struct.
- Document empty-map env_vars semantic and the LIMIT 20 summary cap.
* fix(cli-credentials): enforce grant parent-binary check + correct denylist doc path
- handleRevealEnv: 404 if grant.binary_id != URL binaryID, enforcing the URL hierarchy.
- Fix file-header docstring to point at internal/crypto/env_denylist.go (matches inline comment).
* test(integration): fix CI build failures
- mcp_grant_revoke_test.go: drop duplicate contains helper; use strings.Contains.
- secure_cli_cross_tenant_isolation_test.go: remove (referenced non-existent APIs).
- secure_cli_agent_grants_env_test.go: drop unused store import.
- secure_cli_reveal_rate_limit_test.go: drop unused database/sql import.
* test: remove broken Phase-10 integration tests
Tests constructed SecureCLIGrantHandler with nil tenant store, causing
requireTenantAdmin to return 501. These were scaffolding-only tests
that never passed. Core functionality validated by four passing Claude
review rounds.
* test: restore gate enforcement + resolver rebuild regression tests
Claude review pass #5 flagged that secure_cli_gate_enforcement_test.go
and the resolver rebuild test in mcp_grant_revoke_test.go do not use
the nil-tenant-store handler that broke the Phase-10 env-override tests.
Restored from origin/dev with minor fixes:
- mcp_grant_revoke_test.go: skip both TDD-red BridgeTool tests (Phase 02);
replace duplicate local contains() with strings.Contains
- secure_cli_gate_enforcement_test.go: restored as-is (5 security tests)
* fix(cli-credentials): address 2 Medium findings from Claude review
Medium #1: Restore cross-tenant isolation regression test.
- Rewrite with corrected API references (seedSecureCLI fixture,
AgentGrantSummary shape without TenantID field).
- Scope: store-layer tests only. SQL-enforced isolation via
b.tenant_id + LEFT JOIN LATERAL g.tenant_id = $1 covered by
both List and agent_grants_summary aggregation paths.
- HTTP-layer tests deferred — require gateway-token auth scaffolding.
Medium #2: Inject env:reveal rate limiter into handler instance.
- Removed package-level envRevealLimiter singleton.
- Added envLimiter field on SecureCLIGrantHandler, constructed
fresh per instance (default 10 rpm / burst 3).
- Added SetEnvRevealLimiter(rpm, burst) for deterministic tests.
- Prevents cross-test state leakage under t.Parallel().
* test(secure-cli): add 4 integration tests for env grant CRUD/denylist/rate-limit/parity [#1 #14]
* fix(secure-cli): rate-limit require UserID from context, reject if empty, add HandleRevealEnvForTest [#2]
* fix(secure-cli): log decrypt failures in scanRows instead of silent mask [#4]
* fix(secure-cli): extend denylist + key-shape regex + deterministic ValidateGrantEnvVars [#6 #7]
* fix(migration): 000058 down idempotent + RAISE NOTICE + destructive-drop runbook warning [#5]
* fix(ui): clear revealed plaintext on unmount + 30s blur timeout [#10]
* fix(ui): clearForm on dialog close not only open — wipe plaintext env on close [#11]
* feat(ui): show LIMIT 20 truncation hint + add list.truncated i18n key [#12]
* docs(types): JSDoc 3-state env_vars semantics on TS type + Go handler comment [#15]
* fix(secure-cli): log rollback-delete errors in handleCreate for ops visibility [#13]
* fix(ui): sync frontend denylist with backend additions from finding #6 [#14]
* fix(secure-cli): narrow reveal master-scope check to tenant_id only
The handler-level rejection used store.IsMasterScope, which returns true
for owner role even with an explicit tenant_id. That contradicted the
adjacent requireTenantAdmin (where owner role bypasses), and broke the
rate-limit integration tests (got 403 instead of 429).
Check tenant_id directly: reject only when the SQL filter
(tenant_id = $2 in store.Get) would not bind to a real tenant — i.e.
uuid.Nil or MasterTenantID. Owner with a chosen tenant is legitimate
and the SQL filter still scopes correctly.
Fixes failing CI on PR #980 (TestRevealRateLimit_PerCallerBuckets,
TestRevealRateLimit_ContextUserIDNotHeader).
Multi-Tenant AI Agent Platform
Multi-agent AI gateway built in Go. 20+ LLM providers. 7 channels. Multi-tenant PostgreSQL.
Single binary. Production-tested. Agents that orchestrate for you.
Documentation • Quick Start • Twitter / X
🌐 Languages: 🇨🇳 简体中文 · 🇯🇵 日本語 · 🇰🇷 한국어 · 🇻🇳 Tiếng Việt · 🇵🇭 Tagalog · 🇪🇸 Español · 🇧🇷 Português · 🇮🇹 Italiano · 🇩🇪 Deutsch · 🇫🇷 Français · 🇸🇦 العربية · 🇮🇳 हिन्दी · 🇷🇺 Русский · 🇧🇩 বাংলা · 🇮🇱 עברית · 🇵🇱 Polski · 🇨🇿 Čeština · 🇳🇱 Nederlands · 🇹🇷 Türkçe · 🇺🇦 Українська · 🇮🇩 Bahasa Indonesia · 🇹🇭 ไทย · 🇵🇰 اردو · 🇷🇴 Română · 🇸🇪 Svenska · 🇬🇷 Ελληνικά · 🇭🇺 Magyar · 🇫🇮 Suomi · 🇩🇰 Dansk · 🇳🇴 Norsk
Core Features
- 8-Stage Agent Pipeline — context → history → prompt → think → act → observe → memory → summarize. Pluggable stages, always-on execution
- 4-Mode Prompt System — Full / Task / Minimal / None with section gating, cache boundary optimization, and per-session mode resolution
- 3-Tier Memory — Working (conversation) → Episodic (session summaries) → Semantic (knowledge graph). Progressive loading L0/L1/L2
- Knowledge Vault — Document registry with wikilinks, hybrid search (FTS + pgvector), filesystem sync
- Agent Teams & Orchestration — Shared task boards, inter-agent delegation (sync/async), 3 orchestration modes (auto/explicit/manual)
- Self-Evolution — Metrics → suggestions → auto-adapt with guardrails. Agents refine their own communication style
- Multi-Tenant PostgreSQL — Per-user workspaces, per-user context files, encrypted API keys (AES-256-GCM), RBAC, isolated sessions
- 20+ LLM Providers — Anthropic (native HTTP+SSE with prompt caching), OpenAI, OpenRouter, Groq, DeepSeek, Gemini, Mistral, xAI, MiniMax, DashScope, Claude CLI, Codex, ACP, and any OpenAI-compatible endpoint
- 7 Messaging Channels — Telegram, Discord, Slack, Zalo OA, Zalo Personal, Feishu/Lark, WhatsApp
- Production Security — 5-layer permission system, rate limiting, prompt injection detection, SSRF protection, AES-256-GCM encryption
- Single Binary — ~25 MB static Go binary, no Node.js runtime, <1s startup, runs on a $5 VPS
- Observability — Built-in LLM call tracing with spans and prompt cache metrics, optional OpenTelemetry OTLP export
Desktop Edition (GoClaw Lite)
A native desktop app for local AI agents — no Docker, no PostgreSQL, no infrastructure.
macOS:
curl -fsSL https://raw.githubusercontent.com/nextlevelbuilder/goclaw/main/scripts/install-lite.sh | bash
Windows (PowerShell):
irm https://raw.githubusercontent.com/nextlevelbuilder/goclaw/main/scripts/install-lite.ps1 | iex
What's Included
- Single native app (Wails v2 + React), ~30 MB
- SQLite database (zero setup)
- Chat with agents (streaming, tools, media, file attachments)
- Agent management (max 5), provider config, MCP servers, skills, cron
- Team tasks with Kanban board and real-time updates
- Auto-update from GitHub Releases
Lite vs Standard
| Feature | Lite (Desktop) | Standard (Server) |
|---|---|---|
| Agents | Max 5 | Unlimited |
| Teams | Max 1 (5 members) | Unlimited |
| Database | SQLite (local) | PostgreSQL |
| Memory | FTS5 text search | pgvector semantic |
| Channels | — | Telegram, Discord, Slack, Zalo, Feishu, WhatsApp |
| Knowledge Graph | — | Full |
| RBAC / Multi-tenant | — | Full |
| Auto-update | GitHub Releases | Docker / binary |
Building from Source
# Prerequisites: Go 1.26+, pnpm, Wails CLI (go install github.com/wailsapp/wails/v2/cmd/wails@latest)
make desktop-build # Build .app (macOS) or .exe (Windows)
make desktop-dmg VERSION=0.1.0 # Create .dmg installer (macOS only)
make desktop-dev # Dev mode with hot reload
Desktop Releases
Desktop uses independent versioning with lite-v* tags:
git tag lite-v0.1.0 && git push origin lite-v0.1.0
# → GitHub Actions builds macOS (.dmg + .tar.gz) + Windows (.zip)
# → Creates GitHub Release with all assets
Architecture
Quick Start
Prerequisites: Go 1.26+, PostgreSQL 18 with pgvector, Docker (optional)
From Source
git clone -b main https://github.com/nextlevelbuilder/goclaw.git && cd goclaw
make build
./goclaw onboard # Interactive setup wizard
source .env.local && ./goclaw
Note: The default branch is
dev(active development). Use-b mainto clone the stable release branch.
With Docker
# Generate .env with auto-generated secrets
chmod +x prepare-env.sh && ./prepare-env.sh
# Add at least one GOCLAW_*_API_KEY to .env, then:
make up
# Web Dashboard at http://localhost:18790 (built-in)
# Health check: curl http://localhost:18790/health
# Optional: separate nginx for custom SSL/reverse proxy
# make up WITH_WEB_NGINX=1 → Dashboard at http://localhost:3000
make up creates a Docker network, embeds the correct version from git tags, builds and starts all services, and runs database migrations automatically.
Common commands:
make up # Start all services (build + migrate)
make down # Stop all services
make logs # Tail logs (goclaw service)
make reset # Wipe volumes and rebuild from scratch
Optional services — enable with WITH_* flags:
| Flag | Service | What it does |
|---|---|---|
WITH_BROWSER=1 |
Headless Chrome | Enables browser tool for web scraping, screenshots, automation |
WITH_OTEL=1 |
Jaeger | OpenTelemetry tracing UI for debugging LLM calls and latency |
WITH_SANDBOX=1 |
Docker sandbox | Isolated container for running untrusted code from agents |
WITH_TAILSCALE=1 |
Tailscale | Expose gateway over Tailscale private network |
WITH_REDIS=1 |
Redis | Redis-backed caching layer |
Flags can be combined and work with all commands:
# Start with browser automation and tracing
make up WITH_BROWSER=1 WITH_OTEL=1
# Stop everything including optional services
make down WITH_BROWSER=1 WITH_OTEL=1
When GOCLAW_*_API_KEY environment variables are set, the gateway auto-onboards without interactive prompts — detects provider, runs migrations, and seeds default data.
Docker image variants:
Image Description latestBackend + embedded web UI + Python (recommended) latest-baseBackend API-only, no web UI, no runtimes latest-fullAll runtimes + skill dependencies pre-installed latest-otelLatest + OpenTelemetry tracing goclaw-webStandalone nginx + React SPA (for custom reverse proxy) For custom builds (Tailscale, Redis):
docker build --build-arg ENABLE_TSNET=true ...See the Deployment Guide for details.
Updating
Docker
docker compose pull && docker compose up -d
Binary (with embedded web UI)
goclaw update --apply # Downloads, verifies SHA256, swaps binary, restarts
Web Dashboard
Open About dialog → click Update Now (admin only). The update includes both backend and web dashboard when using the default latest image.
Multi-Agent Orchestration
Each agent runs with its own identity, tools, LLM provider, and context files. Three delegation modes — sync (wait), async (fire-and-forget), bidirectional — connected through explicit permission links with concurrency limits.
Details: Agent Teams docs
Knowledge Vault
Document registry with [[wikilinks]] for bidirectional linking. Hybrid search combines full-text (BM25) and semantic (pgvector) for precise retrieval. Filesystem sync keeps vault in sync with on-disk files.
Self-Evolution
Agents improve themselves through a 3-stage guardrailed pipeline: metrics collection → suggestion analysis → auto-adaptation. Can refine communication style and domain expertise (CAPABILITIES.md) — but never change identity, name, or core purpose.
Provider Adapters
20+ LLM providers unified through a single adapter interface. Capability-based routing, encrypted API keys (AES-256-GCM), extended thinking support per-provider, and prompt caching for Anthropic + OpenAI.
Event-Driven Architecture
Typed domain events power the consolidation pipeline — session summaries, knowledge graph extraction, and dreaming promotion all run asynchronously via worker pools with dedup and retry.
Built-in Tools
30+ tools across 8 categories:
| Category | Tools | Description |
|---|---|---|
| Filesystem | read_file, write_file, edit_file, list_files, search, glob |
File operations with virtual FS routing |
| Runtime | exec, browser |
Shell commands (approval workflow) + browser automation |
| Web | web_search, web_fetch |
Search (Brave, DuckDuckGo) + content extraction |
| Memory | memory_search, memory_get, knowledge_graph_search |
3-tier memory + KG traversal |
| Media | create_image, create_audio, create_video, read_*, tts |
Generation + analysis (multi-provider) |
| Skills | skill_search, use_skill, skill_manage |
BM25 + semantic hybrid search |
| Teams | team_tasks, spawn, delegate, message |
Task board + orchestration + messaging |
| Automation | cron, heartbeat, sessions_* |
Scheduling + session management |
Full tool reference at docs.goclaw.sh
Documentation
Full documentation at docs.goclaw.sh — or browse the source in goclaw-docs/
| Section | Topics |
|---|---|
| Getting Started | Installation, Quick Start, Configuration, Web Dashboard Tour |
| Core Concepts | Agent Loop, Sessions, Tools, Memory, Multi-Tenancy |
| Agents | Creating Agents, Context Files, Personality, Sharing & Access |
| Providers | Anthropic, OpenAI, OpenRouter, Gemini, DeepSeek, +15 more |
| Channels | Telegram, Discord, Slack, Feishu, Zalo, WhatsApp, WebSocket |
| Agent Teams | Teams, Task Board, Messaging, Delegation & Handoff |
| Advanced | Custom Tools, MCP, Skills, Cron, Sandbox, Hooks, RBAC |
| Deployment | Docker Compose, Database, Security, Observability, Tailscale |
| Reference | CLI Commands, REST API, WebSocket Protocol, Environment Variables |
Testing
go test ./... # Unit tests
go test -v ./tests/integration/ -timeout 120s # Integration tests (requires running gateway)
Project Status
See CHANGELOG.md for detailed feature status including what's been tested in production and what's still in progress.
Acknowledgments
GoClaw was originally inspired by the OpenClaw project architecture.
License
CC BY-NC 4.0 — Creative Commons Attribution-NonCommercial 4.0 International