tiennm99/goclaw
1
0 Fork 0
mirror of https://github.com/tiennm99/goclaw.git synced 2026-10-14 04:16:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: remove standalone mode, consolidate to managed-only (PostgreSQL) (#70)

Browse Source 
- Remove standalone mode code: file-based stores, standalone gateway,
  heartbeat service, SQLite memory, standalone docker-compose
- Rename docker-compose.managed.yml → docker-compose.postgres.yml
- Clean up ~130 Go comments referencing "managed mode" qualifier
- Simplify docker-compose.yml env vars (providers/channels via web UI)
- Update .env.example to essential vars only (token + encryption key)
- Add setup wizard UI (provider → agent → channel bootstrap flow)
- Add logs.tail WebSocket handler for live log streaming
- Add cursor-pointer to interactive UI components
- Clean up config page (remove standalone-only sections)
- Update README and docs for managed-only architecture
This commit is contained in:
Viet Tran authored and GitHub committed 2026-03-06 18:51:11 +07:00
1 parent 0f44f18b0c
commit 6895e369f6
183 files changed
+2280 -5340
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.env.example
+2 -27
View File
@@ -1,38 +1,13 @@
# GoClaw environment variables.
# Copy to .env and fill in values. For Docker Compose, do NOT use 'export' prefix.
#
# Required: at least one LLM provider API key.
# Auto-generated by prepare-env.sh: GOCLAW_GATEWAY_TOKEN, GOCLAW_ENCRYPTION_KEY.
# LLM providers and channels are configured via the web dashboard setup wizard.
# --- LLM Provider (at least one API key required) ---
# Provider is auto-detected from API keys. Default model is chosen per provider.
# GOCLAW_MODEL= # Optional: override default model
GOCLAW_OPENROUTER_API_KEY=
GOCLAW_ANTHROPIC_API_KEY=
GOCLAW_OPENAI_API_KEY=
GOCLAW_MINIMAX_API_KEY=
GOCLAW_GROQ_API_KEY=
GOCLAW_DEEPSEEK_API_KEY=
GOCLAW_GEMINI_API_KEY=
GOCLAW_MISTRAL_API_KEY=
GOCLAW_XAI_API_KEY=
GOCLAW_COHERE_API_KEY=
GOCLAW_PERPLEXITY_API_KEY=
# --- Gateway ---
# --- Gateway (required) ---
GOCLAW_GATEWAY_TOKEN=
GOCLAW_ENCRYPTION_KEY=
# --- Channels (optional) ---
GOCLAW_TELEGRAM_TOKEN=
GOCLAW_DISCORD_TOKEN=
GOCLAW_LARK_APP_ID=
GOCLAW_LARK_APP_SECRET=
# GOCLAW_LARK_ENCRYPT_KEY=
# GOCLAW_LARK_VERIFICATION_TOKEN=
GOCLAW_ZALO_TOKEN=
# GOCLAW_WHATSAPP_BRIDGE_URL=
# --- Database (only for non-Docker deployments) ---
# Docker Compose auto-builds this from POSTGRES_USER/PASSWORD/DB.
# GOCLAW_POSTGRES_DSN=postgres://user:pass@host:5432/dbname?sslmode=disable
CLAUDE.md
+7 -8
View File
@@ -1,6 +1,6 @@
# GoClaw Gateway
AI agent gateway with WebSocket RPC + HTTP API. Two modes: **standalone** (file-based) and **managed** (PostgreSQL multi-tenant).
PostgreSQL multi-tenant AI agent gateway with WebSocket RPC + HTTP API.
## Tech Stack
@@ -18,13 +18,13 @@ internal/
├── agent/ Agent loop (think→act→observe), router, resolver, input guard
├── providers/ LLM providers: Anthropic (native HTTP+SSE), OpenAI-compat (HTTP+SSE)
├── tools/ Tool registry, filesystem, exec, web, memory, subagent, MCP bridge
├── store/ Store interfaces + pg/ (PostgreSQL) + file/ (standalone) implementations
├── store/ Store interfaces + pg/ (PostgreSQL) implementations
├── bootstrap/ System prompt files (SOUL.md, IDENTITY.md) + seeding + per-user seed
├── config/ Config loading (JSON5) + env var overlay
├── channels/ Channel manager: Telegram, Feishu/Lark, Zalo, Discord, WhatsApp
├── http/ HTTP API (/v1/chat/completions, /v1/agents, /v1/skills, etc.)
├── skills/ SKILL.md loader + BM25 search
├── memory/ Memory system (SQLite FTS5 / pgvector)
├── memory/ Memory system (pgvector)
├── tracing/ LLM call tracing + optional OTel export (build-tag gated)
├── scheduler/ Lane-based concurrency (main/subagent/cron)
├── cron/ Cron scheduling (at/every/cron expr)
@@ -41,11 +41,10 @@ ui/web/ React SPA (pnpm, Vite, Tailwind, Radix UI)
## Key Patterns
- **Two modes:** Standalone (file-based, shared workspace) vs Managed (PostgreSQL, per-user isolation)
- **Store layer:** Interface-based (`store.SessionStore`, `store.AgentStore`, etc.) with file/ and pg/ implementations. PG uses `database/sql` + `pgx/v5/stdlib`, raw SQL, `execMapUpdate()` helper in `pg/helpers.go`
- **Agent types (managed):** `open` (per-user context, 7 files) vs `predefined` (shared context + USER.md per-user)
- **Store layer:** Interface-based (`store.SessionStore`, `store.AgentStore`, etc.) with pg/ (PostgreSQL) implementations. Uses `database/sql` + `pgx/v5/stdlib`, raw SQL, `execMapUpdate()` helper in `pg/helpers.go`
- **Agent types:** `open` (per-user context, 7 files) vs `predefined` (shared context + USER.md per-user)
- **Context files:** `agent_context_files` (agent-level) + `user_context_files` (per-user), routed via `ContextFileInterceptor`
- **Providers:** Anthropic (native HTTP+SSE) and OpenAI-compat (generic). Both use `RetryDo()` for retries. Managed mode loads from `llm_providers` table with encrypted API keys
- **Providers:** Anthropic (native HTTP+SSE) and OpenAI-compat (generic). Both use `RetryDo()` for retries. Loads from `llm_providers` table with encrypted API keys
- **Agent loop:** `RunRequest` → think→act→observe → `RunResult`. Events: `run.started`, `run.completed`, `chunk`, `tool.call`, `tool.result`. Auto-summarization at >75% context
- **Context propagation:** `store.WithAgentType(ctx)`, `store.WithUserID(ctx)`, `store.WithAgentID(ctx)`
- **WebSocket protocol (v3):** Frame types `req`/`res`/`event`. First request must be `connect`
@@ -57,7 +56,7 @@ ui/web/ React SPA (pnpm, Vite, Tailwind, Radix UI)
```bash
go build -o goclaw . && ./goclaw onboard && source .env.local && ./goclaw
./goclaw migrate up # DB migrations (managed mode)
./goclaw migrate up # DB migrations
go test -v ./tests/integration/ # Integration tests
cd ui/web && pnpm install && pnpm dev # Web dashboard (dev)
README.md
+39 -85
View File
@@ -53,7 +53,7 @@ A Go port of [OpenClaw](https://github.com/openclaw/openclaw) with enhanced secu
| Companion apps | macOS, iOS, Android | Python SDK | — | Web dashboard |
| Live Canvas / Voice | ✅ (A2UI + TTS/STT) | — | Voice transcription | TTS (4 providers) |
| LLM providers | 10+ | 8 native + 29 compat | 13+ | **13+** |
| Per-user workspaces | ✅ (file-based) | — | — | ✅ (managed mode only) |
| Per-user workspaces | ✅ (file-based) | — | — | ✅ (PostgreSQL) |
| Encrypted secrets | — (env vars only) | ✅ ChaCha20-Poly1305 | — (plaintext JSON) | ✅ AES-256-GCM in DB |
> **GoClaw unique strengths:** Only project with multi-tenant PostgreSQL, agent teams, conversation handoff, evaluate-loop quality gates, runtime custom tools via API, and MCP protocol support.
@@ -84,14 +84,13 @@ graph TB
end
subgraph Storage
FILE["File-based<br/>(standalone)"]
PG["PostgreSQL 18 + pgvector<br/>(managed · multi-tenant)"]
PG["PostgreSQL 18 + pgvector<br/>(multi-tenant)"]
end
WEB --> WS
TG & DC & FS & ZL & ZLP --> CM
API --> REST
LOOP --> FILE & PG
LOOP --> PG
```
## Multi-Agent Orchestration
@@ -325,8 +324,8 @@ Quality gates validate agent output before it reaches users. Configured in agent
- **Discord, WhatsApp** — Channel adapters with `/stop` and `/stopall` commands
### Knowledge & Memory
- **Skills** — SKILL.md-based knowledge base with BM25 search + embedding hybrid search (managed mode)
- **Long-term memory** — SQLite FTS5 + vector embeddings (standalone) or pgvector hybrid search (managed)
- **Skills** — SKILL.md-based knowledge base with BM25 + embedding hybrid search (pgvector)
- **Long-term memory** — pgvector hybrid search (full-text + vector similarity)
### Infrastructure
- **Cron scheduling** — `at`, `every`, and cron expression syntax for scheduled agent tasks
@@ -342,7 +341,7 @@ Quality gates validate agent output before it reaches users. Configured in agent
- **Credential scrubbing** — Auto-redact API keys, tokens, passwords from tool outputs
- **Shell deny patterns** — Blocks `curl|sh`, reverse shells, `eval $()`, `base64|sh`
- **SSRF protection** — DNS pinning, blocked private IPs, blocked hosts
- **AES-256-GCM** — Encrypted API keys in database (managed mode)
- **AES-256-GCM** — Encrypted API keys in database
- **Browser pairing** — Token-free browser auth with admin-approved pairing codes
### Web Dashboard
@@ -383,27 +382,24 @@ The script creates `.env` from `.env.example`, auto-generates `GOCLAW_ENCRYPTION
**2. Start services:**
```bash
# Recommended: Managed mode + Web Dashboard (http://localhost:3000)
docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.selfservice.yml up -d --build
# Recommended: Gateway + Web Dashboard (http://localhost:3000)
docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.selfservice.yml up -d --build
# Standalone mode (file-based, no database)
docker compose -f docker-compose.yml -f docker-compose.standalone.yml up -d --build
# Managed mode without dashboard
docker compose -f docker-compose.yml -f docker-compose.managed.yml up -d --build
# Without dashboard
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d --build
# + OpenTelemetry tracing (Jaeger at http://localhost:16686)
docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.otel.yml up -d --build
docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.otel.yml up -d --build
# + Tailscale (secure remote access)
docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.tailscale.yml up -d --build
docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.tailscale.yml up -d --build
```
When `GOCLAW_*_API_KEY` environment variables are set, the gateway **auto-onboards** without interactive prompts — it detects the provider, generates a gateway token, and (in managed mode) connects to Postgres, runs migrations, and seeds default data.
When `GOCLAW_*_API_KEY` environment variables are set, the gateway **auto-onboards** without interactive prompts — it detects the provider, generates a gateway token, connects to Postgres, runs migrations, and seeds default data.
**Auto-onboard detects** the first available API key in priority order: OpenRouter → Anthropic → OpenAI → Groq → DeepSeek → Gemini → Mistral → xAI → MiniMax → Cohere → Perplexity. Override with `GOCLAW_PROVIDER` and `GOCLAW_MODEL`. Memory is auto-enabled with embedding support if an OpenAI, OpenRouter, or Gemini key is detected.
**Minimum `.env` for managed mode:**
**Minimum `.env`:**
```bash
GOCLAW_OPENROUTER_API_KEY=sk-or-your-key # Required: at least one provider key
@@ -414,47 +410,12 @@ GOCLAW_ENCRYPTION_KEY=... # Auto-generated by prepare-env.sh
# POSTGRES_PASSWORD=your-secure-password # Optional: defaults to "goclaw"
```
## Deployment Modes
## Deployment
> **Recommendation:** Use **managed mode** for the best experience. Most of GoClaw's advanced features — agent teams, delegation, handoff, evaluate loops, quality gates, tracing, skills with embedding search, MCP integration, and the web dashboard — require managed mode. Standalone mode is suitable for quick evaluation or single-user setups only.
| Capability | Standalone | Managed |
|-----------|:----------:|:-------:|
| Basic agent loop + tools | ✅ | ✅ |
| Messaging channels | ✅ | ✅ |
| Memory (FTS) | ✅ | ✅ |
| Per-user isolation | — | ✅ |
| Agent teams & delegation | — | ✅ |
| Handoff & evaluate loops | — | ✅ |
| Quality gates | — | ✅ |
| Tracing with cache metrics | — | ✅ |
| Skills (BM25 + pgvector) | Basic | ✅ Hybrid search |
| MCP server integration | — | ✅ |
| Custom tools (runtime API) | — | ✅ |
| Web dashboard | — | ✅ |
| API key encryption | — | ✅ AES-256-GCM |
### Standalone
File-based storage, no external database required. All users share the same workspace, sessions, and context files — no per-user isolation. Best suited for quick evaluation or single-user setups.
```
config.json -> Non-secret settings
.env.local -> Secrets (API keys, tokens)
~/.goclaw/
|-- workspace/ -> Shared agent workspace (SOUL.md, AGENTS.md, etc.)
|-- data/ -> Cron jobs, pairing data
|-- sessions/ -> Chat session history (shared across users)
+-- skills/ -> User-managed skills
```
### Managed (PostgreSQL) — Recommended
All data in PostgreSQL with pgvector support. Designed for multi-user and multi-tenant deployments with **per-user isolation** — each user gets their own context files, session history, and workspace. Unlocks all advanced features.
GoClaw requires PostgreSQL with pgvector. Designed for multi-user and multi-tenant deployments with **per-user isolation** — each user gets their own context files, session history, and workspace.
```bash
# Set up database
export GOCLAW_MODE=managed
export GOCLAW_POSTGRES_DSN="postgres://user:pass@localhost:5432/goclaw?sslmode=disable"
export GOCLAW_ENCRYPTION_KEY=$(openssl rand -hex 32)
@@ -465,7 +426,7 @@ export GOCLAW_ENCRYPTION_KEY=$(openssl rand -hex 32)
./goclaw
```
**Managed mode adds:**
**Features:**
- Per-user context files and workspaces (`user_context_files` table)
- Agent types: `open` (per-user workspace) vs `predefined` (shared context)
@@ -482,7 +443,7 @@ export GOCLAW_ENCRYPTION_KEY=$(openssl rand -hex 32)
### Prerequisites
- Go 1.25+
- PostgreSQL 18 with pgvector (managed mode only)
- PostgreSQL 18 with pgvector
- Docker (optional, for sandbox and containerized deployment)
### Build
@@ -539,11 +500,11 @@ docker build --build-arg ENABLE_OTEL=true --build-arg ENABLE_TSNET=true -t gocla
./goclaw onboard
```
The wizard configures: provider, model, gateway port, channels, memory, browser, TTS, tracing, and database mode. It generates `config.json` (no secrets) and `.env.local` (secrets only).
The wizard configures: provider, model, gateway port, channels, memory, browser, TTS, and tracing. It generates `config.json` (no secrets) and `.env.local` (secrets only).
### Auto-Onboard (Docker / CI)
When `GOCLAW_*_API_KEY` environment variables are set, the gateway automatically configures itself without interactive prompts. In managed mode, it retries Postgres connection (up to 5 attempts), runs migrations, and seeds default data.
When `GOCLAW_*_API_KEY` environment variables are set, the gateway automatically configures itself without interactive prompts. It retries Postgres connection (up to 5 attempts), runs migrations, and seeds default data.
### Environment Variables
@@ -588,11 +549,10 @@ When `GOCLAW_*_API_KEY` environment variables are set, the gateway automatically
</details>
<details>
<summary><strong>Database (Managed Mode)</strong></summary>
<summary><strong>Database</strong></summary>
| Variable | Description |
| ----------------------- | -------------------------------------- |
| `GOCLAW_MODE` | `standalone` or `managed` |
| `GOCLAW_POSTGRES_DSN` | PostgreSQL connection string |
| `GOCLAW_ENCRYPTION_KEY` | AES-256-GCM key for API key encryption |
| `GOCLAW_MIGRATIONS_DIR` | Path to migration files |
@@ -669,8 +629,6 @@ goclaw Start gateway (default command)
goclaw onboard Interactive setup wizard
goclaw version Print version and protocol info
goclaw doctor System health check (includes schema status)
In managed mode: reads providers and channels from DB
In standalone mode: reads from config.json + env vars
goclaw upgrade Upgrade database schema and run data hooks
goclaw upgrade --status Show current vs required schema version
@@ -726,13 +684,12 @@ See [WebSocket Protocol](websocket-protocol.md) for the real-time RPC protocol (
## Docker Compose
Eight composable files for different deployment scenarios:
Composable files for different deployment scenarios:
| File | Purpose |
| -------------------------------- | -------------------------------------------------- |
| `docker-compose.yml` | Base service definition |
| `docker-compose.standalone.yml` | File-based storage with persistent volumes |
| `docker-compose.managed.yml` | PostgreSQL (pgvector/pgvector:pg18) + managed mode |
| `docker-compose.postgres.yml` | PostgreSQL (pgvector/pgvector:pg18) |
| `docker-compose.upgrade.yml` | One-shot database upgrade service |
| `docker-compose.selfservice.yml` | Web dashboard UI (nginx + React SPA) |
| `docker-compose.sandbox.yml` | Docker-based code execution sandbox |
@@ -745,26 +702,23 @@ Eight composable files for different deployment scenarios:
# Prepare .env (auto-generates secrets, prompts for API key)
chmod +x prepare-env.sh && ./prepare-env.sh
# Standalone
docker compose -f docker-compose.yml -f docker-compose.standalone.yml up -d --build
# Managed (PostgreSQL)
docker compose -f docker-compose.yml -f docker-compose.managed.yml up -d --build
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d --build
# Managed + Web Dashboard (http://localhost:3000)
docker compose -f docker-compose.yml \
-f docker-compose.managed.yml \
-f docker-compose.postgres.yml \
-f docker-compose.selfservice.yml up -d --build
# Managed + Web Dashboard + OpenTelemetry (Jaeger UI at http://localhost:16686)
docker compose -f docker-compose.yml \
-f docker-compose.managed.yml \
-f docker-compose.postgres.yml \
-f docker-compose.selfservice.yml \
-f docker-compose.otel.yml up -d --build
# Managed + Tailscale (secure remote access via VPN mesh)
docker compose -f docker-compose.yml \
-f docker-compose.managed.yml \
-f docker-compose.postgres.yml \
-f docker-compose.tailscale.yml up -d --build
# Check health
@@ -780,7 +734,7 @@ curl http://localhost:18790/health
git pull
# Rebuild and restart (auto-upgrades database on start)
docker compose -f docker-compose.yml -f docker-compose.managed.yml \
docker compose -f docker-compose.yml -f docker-compose.postgres.yml \
-f docker-compose.selfservice.yml up -d --build
```
@@ -790,19 +744,19 @@ Replace the compose files with whichever overlays you use (e.g. add `-f docker-c
```bash
# Check current schema status
docker compose -f docker-compose.yml -f docker-compose.managed.yml \
docker compose -f docker-compose.yml -f docker-compose.postgres.yml \
-f docker-compose.upgrade.yml run --rm upgrade --status
# Preview pending changes (dry-run)
docker compose -f docker-compose.yml -f docker-compose.managed.yml \
docker compose -f docker-compose.yml -f docker-compose.postgres.yml \
-f docker-compose.upgrade.yml run --rm upgrade --dry-run
# Apply upgrade (schema migrations + data hooks), then remove container
docker compose -f docker-compose.yml -f docker-compose.managed.yml \
docker compose -f docker-compose.yml -f docker-compose.postgres.yml \
-f docker-compose.upgrade.yml run --rm upgrade
# Then rebuild and restart the gateway with the new image
docker compose -f docker-compose.yml -f docker-compose.managed.yml up -d --build
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d --build
```
### Environment File (.env)
@@ -819,7 +773,7 @@ This creates `.env` with `GOCLAW_ENCRYPTION_KEY` and `GOCLAW_GATEWAY_TOKEN` pre-
| Tool | Group | Description |
| ------------------ | ------------- | ------------------------------------------------------------ |
| `read_file` | fs | Read file contents (with virtual FS routing in managed mode) |
| `read_file` | fs | Read file contents (with virtual FS routing) |
| `write_file` | fs | Write/create files |
| `edit_file` | fs | Apply targeted edits to existing files |
| `list_files` | fs | List directory contents |
@@ -831,7 +785,7 @@ This creates `.env` with `GOCLAW_ENCRYPTION_KEY` and `GOCLAW_GATEWAY_TOKEN` pre-
| `web_fetch` | web | Fetch and parse web content |
| `memory_search` | memory | Search long-term memory (FTS + vector) |
| `memory_get` | memory | Retrieve memory entries |
| `skill_search` | — | Search skills (BM25 + embedding hybrid in managed mode) |
| `skill_search` | — | Search skills (BM25 + embedding hybrid) |
| `image` | — | Image generation/manipulation |
| `message` | messaging | Send messages to channels |
| `tts` | — | Text-to-Speech synthesis |
@@ -906,7 +860,7 @@ This keeps the gateway inaccessible from the LAN while remaining reachable via T
```bash
docker compose -f docker-compose.yml \
-f docker-compose.managed.yml \
-f docker-compose.postgres.yml \
-f docker-compose.tailscale.yml up -d
```
@@ -920,7 +874,7 @@ Requires `GOCLAW_TSNET_AUTH_KEY` in your `.env` file. Tailscale state is persist
- **Shell security**: Deny patterns for `curl|sh`, `wget|sh`, reverse shells, `eval`, `base64|sh`
- **Network**: SSRF protection with blocked hosts + private IP + DNS pinning
- **File system**: Path traversal prevention, workspace restriction
- **Encryption**: AES-256-GCM for API keys in database (managed mode)
- **Encryption**: AES-256-GCM for API keys in database
- **Browser pairing**: Token-free browser auth with admin approval (pairing codes, auto-reconnect)
- **Tailscale**: Optional VPN mesh listener for secure remote access (build-tag gated)
@@ -943,13 +897,13 @@ GOCLAW_OPENROUTER_API_KEY=sk-or-xxx go test -v ./tests/integration/ -timeout 120
- **Agent management & configuration** — Create, update, delete agents via API and web dashboard. Agent types (`open` / `predefined`), agent routing, and lazy resolution all tested.
- **Telegram channel** — Full integration tested: message handling, streaming responses, rich formatting (HTML, tables, code blocks), reactions, media, chunked long messages.
- **Seed data & bootstrapping** — Auto-onboard, DB seeding, migration pipeline tested end-to-end in managed mode.
- **Seed data & bootstrapping** — Auto-onboard, DB seeding, migration pipeline tested end-to-end.
- **User-scope & content files** — Per-user context files (`user_context_files`), agent-level context files (`agent_context_files`), virtual FS interceptors, per-user seeding (`SeedUserFiles`), and user-agent profile tracking all implemented and tested.
- **Core built-in tools** — File system tools (`read_file`, `write_file`, `edit_file`, `list_files`, `search`, `glob`), shell execution (`exec`), web tools (`web_search`, `web_fetch`), and session management tools tested in real agent loops.
- **Memory system** — Long-term memory with search (FTS5 in standalone, pgvector hybrid in managed mode) implemented and tested with real conversations.
- **Memory system** — Long-term memory with pgvector hybrid search (FTS + vector) implemented and tested with real conversations.
- **Agent loop** — Think-act-observe cycle, tool use, session history, auto-summarization, and subagent spawning tested in production.
- **WebSocket RPC protocol (v3)** — Connect handshake, chat streaming, event push all tested with web dashboard and integration tests.
- **Store layer (PostgreSQL)** — All PG stores (sessions, agents, providers, skills, cron, pairing, tracing, memory, teams) implemented and running in managed mode.
- **Store layer (PostgreSQL)** — All PG stores (sessions, agents, providers, skills, cron, pairing, tracing, memory, teams) implemented and running.
- **Browser automation** — Rod/CDP integration for headless Chrome, tested in production agent workflows.
- **Lane-based scheduler** — Main/subagent/delegate/cron lane isolation with concurrent execution tested. Group chats support up to 3 concurrent agent runs per session with adaptive throttle and deferred session writes for history isolation.
- **Security hardening** — Rate limiting, prompt injection detection, CORS, shell deny patterns, SSRF protection, credential scrubbing all implemented and verified.
cmd/agent_chat.go
+6 -8
View File
@@ -23,7 +23,6 @@ func agentChatCmd() *cobra.Command {
Use: "chat",
Short: "Chat with an agent interactively or send a one-shot message",
Long: `Chat with an agent via the running gateway (WebSocket client mode).
Falls back to standalone mode if the gateway is not running.
Examples:
goclaw agent chat # Interactive REPL
@@ -62,15 +61,14 @@ func runAgentChat(agentName, message, sessionKey string) {
}
addr := fmt.Sprintf("%s:%d", host, cfg.Gateway.Port)
if isGatewayRunning(addr) {
fmt.Fprintf(os.Stderr, "Connected to gateway at %s\n", addr)
runClientMode(cfg, addr, agentName, message, sessionKey)
return
if !isGatewayRunning(addr) {
fmt.Fprintln(os.Stderr, "Error: the gateway must be running for this command.")
fmt.Fprintln(os.Stderr, "Start it first: goclaw")
os.Exit(1)
}
// Fallback: standalone mode
fmt.Fprintf(os.Stderr, "Gateway not running, using standalone mode\n")
runStandaloneMode(cfg, agentName, message, sessionKey)
fmt.Fprintf(os.Stderr, "Connected to gateway at %s\n", addr)
runClientMode(cfg, addr, agentName, message, sessionKey)
}
// --- Gateway detection ---
cmd/agent_chat_client.go
+1 -3
View File
@@ -21,9 +21,7 @@ func runClientMode(cfg *config.Config, addr, agentName, message, sessionKey stri
conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
fmt.Fprintf(os.Stderr, "WebSocket connect failed: %v\n", err)
fmt.Fprintf(os.Stderr, "Falling back to standalone mode\n")
runStandaloneMode(cfg, agentName, message, sessionKey)
return
os.Exit(1)
}
defer conn.Close()
cmd/agent_chat_standalone.go
-233
View File
@@ -1,233 +0,0 @@
package cmd
import (
"bufio"
"context"
"fmt"
"log/slog"
"os"
"os/signal"
"path/filepath"
"strings"
"sync"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/agent"
"github.com/nextlevelbuilder/goclaw/internal/bootstrap"
"github.com/nextlevelbuilder/goclaw/internal/bus"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/providers"
"github.com/nextlevelbuilder/goclaw/internal/sessions"
"github.com/nextlevelbuilder/goclaw/internal/skills"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/file"
"github.com/nextlevelbuilder/goclaw/internal/tools"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
func runStandaloneMode(cfg *config.Config, agentName, message, sessionKey string) {
loop, sessStore, agentCfg := bootstrapStandaloneAgent(cfg, agentName)
chatFn := func(msg string) (string, error) {
runID := fmt.Sprintf("cli-%s", uuid.NewString()[:8])
result, err := loop.Run(context.Background(), agent.RunRequest{
SessionKey: sessionKey,
Message: msg,
Channel: "cli",
ChatID: "local",
PeerKind: "direct",
RunID: runID,
})
if err != nil {
return "", err
}
return result.Content, nil
}
_ = sessStore // keep reference for session persistence
if message != "" {
resp, err := chatFn(message)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
fmt.Println(resp)
return
}
// Interactive REPL
fmt.Fprintf(os.Stderr, "\nGoClaw Interactive Chat — Standalone Mode\n")
fmt.Fprintf(os.Stderr, "Agent: %s | Model: %s\n", agentName, agentCfg.Model)
fmt.Fprintf(os.Stderr, "Session: %s\n", sessionKey)
fmt.Fprintf(os.Stderr, "Type \"exit\" to quit, \"/new\" for new session\n\n")
// Handle Ctrl+C gracefully
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()
scanner := bufio.NewScanner(os.Stdin)
for {
select {
case <-ctx.Done():
fmt.Fprintln(os.Stderr, "\nGoodbye!")
return
default:
}
fmt.Fprint(os.Stderr, "You: ")
if !scanner.Scan() {
break
}
input := strings.TrimSpace(scanner.Text())
if input == "" {
continue
}
if input == "exit" || input == "quit" {
fmt.Fprintln(os.Stderr, "Goodbye!")
return
}
if input == "/new" {
sessionKey = sessions.BuildSessionKey(agentName, "cli", sessions.PeerDirect, uuid.NewString()[:8])
fmt.Fprintf(os.Stderr, "New session: %s\n\n", sessionKey)
continue
}
resp, err := chatFn(input)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n\n", err)
continue
}
fmt.Printf("\n%s\n\n", resp)
}
}
// bootstrapStandaloneAgent creates a minimal agent loop for CLI usage.
func bootstrapStandaloneAgent(cfg *config.Config, agentName string) (*agent.Loop, store.SessionStore, config.AgentDefaults) {
agentCfg := cfg.ResolveAgent(agentName)
workspace := config.ExpandHome(agentCfg.Workspace)
if !filepath.IsAbs(workspace) {
workspace, _ = filepath.Abs(workspace)
}
// Ensure workspace exists
os.MkdirAll(workspace, 0755)
// 1. Provider
providerReg := providers.NewRegistry()
registerProviders(providerReg, cfg)
provider, err := providerReg.Get(agentCfg.Provider)
if err != nil {
names := providerReg.List()
if len(names) == 0 {
fmt.Fprintf(os.Stderr, "Error: no providers configured. Run 'goclaw onboard' first.\n")
os.Exit(1)
}
provider, _ = providerReg.Get(names[0])
slog.Warn("configured provider not found, using fallback", "wanted", agentCfg.Provider, "using", names[0])
}
// 2. Sessions (wrap file-based manager in store adapter)
sessStorage := config.ExpandHome(cfg.Sessions.Storage)
sessStore := file.NewFileSessionStore(sessions.NewManager(sessStorage))
// 3. Tools
toolsReg := tools.NewRegistry()
toolsReg.Register(tools.NewReadFileTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewWriteFileTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewListFilesTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewExecTool(workspace, agentCfg.RestrictToWorkspace))
// Web tools
webSearchTool := tools.NewWebSearchTool(tools.WebSearchConfig{
BraveEnabled: cfg.Tools.Web.Brave.Enabled,
BraveAPIKey: cfg.Tools.Web.Brave.APIKey,
DDGEnabled: cfg.Tools.Web.DuckDuckGo.Enabled,
})
if webSearchTool != nil {
toolsReg.Register(webSearchTool)
}
toolsReg.Register(tools.NewWebFetchTool(tools.WebFetchConfig{
Policy: cfg.Tools.WebFetch.Policy,
AllowedDomains: cfg.Tools.WebFetch.AllowedDomains,
}))
// 4. Bootstrap files
rawFiles := bootstrap.LoadWorkspaceFiles(workspace)
truncCfg := bootstrap.TruncateConfig{
MaxCharsPerFile: agentCfg.BootstrapMaxChars,
TotalMaxChars: agentCfg.BootstrapTotalMaxChars,
}
if truncCfg.MaxCharsPerFile <= 0 {
truncCfg.MaxCharsPerFile = bootstrap.DefaultMaxCharsPerFile
}
if truncCfg.TotalMaxChars <= 0 {
truncCfg.TotalMaxChars = bootstrap.DefaultTotalMaxChars
}
contextFiles := bootstrap.BuildContextFiles(rawFiles, truncCfg)
// 5. Skills
globalSkillsDir := filepath.Join(config.ExpandHome("~/.goclaw"), "skills")
skillsLoader := skills.NewLoader(workspace, globalSkillsDir, "")
toolsReg.Register(tools.NewSkillSearchTool(skillsLoader))
// Allow read_file to access skills directories
if readTool, ok := toolsReg.Get("read_file"); ok {
if rt, ok := readTool.(*tools.ReadFileTool); ok {
rt.AllowPaths(globalSkillsDir)
if homeDir, err := os.UserHomeDir(); err == nil {
rt.AllowPaths(filepath.Join(homeDir, ".agents", "skills"))
}
}
}
// 6. Event display (tool calls on stderr)
var eventMu sync.Mutex
onEvent := func(evt agent.AgentEvent) {
eventMu.Lock()
defer eventMu.Unlock()
switch evt.Type {
case protocol.AgentEventToolCall:
if p, ok := evt.Payload.(map[string]interface{}); ok {
name, _ := p["name"].(string)
fmt.Fprintf(os.Stderr, " [tool] %s\n", name)
}
case protocol.AgentEventToolResult:
// silent — avoid noisy output
}
}
// Per-agent skill allowlist
var skillAllowList []string
if spec, ok := cfg.Agents.List[agentName]; ok {
skillAllowList = spec.Skills
}
// 7. Create agent loop
msgBus := bus.New()
loop := agent.NewLoop(agent.LoopConfig{
ID: agentName,
Provider: provider,
Model: agentCfg.Model,
ContextWindow: agentCfg.ContextWindow,
MaxIterations: agentCfg.MaxToolIterations,
MaxToolCalls: agentCfg.MaxToolCalls,
Workspace: workspace,
Bus: msgBus,
Sessions: sessStore,
Tools: toolsReg,
OnEvent: onEvent,
OwnerIDs: cfg.Gateway.OwnerIDs,
SkillsLoader: skillsLoader,
SkillAllowList: skillAllowList,
HasMemory: false, // skip memory for standalone CLI (avoids SQLite dep issues)
ContextFiles: contextFiles,
CompactionCfg: agentCfg.Compaction,
ContextPruningCfg: agentCfg.ContextPruning,
})
return loop, sessStore, agentCfg
}
cmd/cli_helpers.go
+3 -18
View File
@@ -3,27 +3,12 @@ package cmd
import (
"fmt"
"os"
"github.com/nextlevelbuilder/goclaw/internal/config"
)
// isManagedMode returns true if the config specifies managed (Postgres) mode.
func isManagedMode() bool {
cfg, err := config.Load(resolveConfigPath())
if err != nil {
return false
}
return cfg.Database.Mode == "managed" && cfg.Database.PostgresDSN != ""
}
// requireGatewayForManaged exits with a helpful error if managed mode is active
// and the gateway is not reachable.
func requireGatewayForManaged() {
if !isManagedMode() {
return
}
// requireGateway exits with a helpful error if the gateway is not reachable.
func requireGateway() {
if !isGatewayReachable() {
fmt.Fprintln(os.Stderr, "Error: managed mode requires the gateway to be running.")
fmt.Fprintln(os.Stderr, "Error: the gateway must be running for this command.")
fmt.Fprintln(os.Stderr, "Start it first: goclaw")
os.Exit(1)
}
cmd/cron_cmd.go
+7 -43
View File
@@ -4,16 +4,12 @@ import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"text/tabwriter"
"time"
"github.com/spf13/cobra"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/cron"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/file"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
@@ -35,13 +31,7 @@ func cronListCmd() *cobra.Command {
Use: "list",
Short: "List all cron jobs",
Run: func(cmd *cobra.Command, args []string) {
if isManagedMode() {
cronListRPC(showDisabled, jsonOutput)
return
}
svc := loadCronStore()
jobs := svc.ListJobs(showDisabled, "", "")
printCronJobs(jobs, jsonOutput)
cronListRPC(showDisabled, jsonOutput)
},
}
cmd.Flags().BoolVar(&jsonOutput, "json", false, "output as JSON")
@@ -55,16 +45,7 @@ func cronDeleteCmd() *cobra.Command {
Short: "Delete a cron job",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
if isManagedMode() {
cronDeleteRPC(args[0])
return
}
svc := loadCronStore()
if err := svc.RemoveJob(args[0]); err != nil {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
os.Exit(1)
}
fmt.Printf("Deleted job %s\n", args[0])
cronDeleteRPC(args[0])
},
}
}
@@ -76,24 +57,15 @@ func cronToggleCmd() *cobra.Command {
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
enabled := args[1] == "true" || args[1] == "1" || args[1] == "on"
if isManagedMode() {
cronToggleRPC(args[0], enabled)
return
}
svc := loadCronStore()
if err := svc.EnableJob(args[0], enabled); err != nil {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
os.Exit(1)
}
fmt.Printf("Job %s enabled=%v\n", args[0], enabled)
cronToggleRPC(args[0], enabled)
},
}
}
// --- RPC implementations (managed mode) ---
// --- RPC implementations ---
func cronListRPC(showDisabled, jsonOutput bool) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]interface{}{"includeDisabled": showDisabled})
resp, err := gatewayRPC(protocol.MethodCronList, params)
@@ -119,7 +91,7 @@ func cronListRPC(showDisabled, jsonOutput bool) {
}
func cronDeleteRPC(jobID string) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]string{"jobId": jobID})
resp, err := gatewayRPC(protocol.MethodCronDelete, params)
@@ -135,7 +107,7 @@ func cronDeleteRPC(jobID string) {
}
func cronToggleRPC(jobID string, enabled bool) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]interface{}{"jobId": jobID, "enabled": enabled})
resp, err := gatewayRPC(protocol.MethodCronToggle, params)
@@ -190,11 +162,3 @@ func printCronJobs(jobs []store.CronJob, jsonOutput bool) {
}
tw.Flush()
}
// --- File-based helpers (standalone mode) ---
func loadCronStore() store.CronStore {
dataDir := config.ExpandHome("~/.goclaw/data")
storePath := filepath.Join(dataDir, "cron", "jobs.json")
return file.NewFileCronStore(cron.NewService(storePath, nil))
}
cmd/doctor.go
+10 -9
View File
@@ -49,13 +49,15 @@ func runDoctor() {
return
}
// Database (managed mode only) — open early so we can show DB providers.
// Database
var db *sql.DB
isManaged := cfg.Database.Mode == "managed" && cfg.Database.PostgresDSN != ""
if isManaged {
if cfg.Database.PostgresDSN == "" {
fmt.Println()
fmt.Println(" Database:")
fmt.Printf(" %-12s NOT CONFIGURED (set GOCLAW_POSTGRES_DSN)\n", "Status:")
} else {
fmt.Println()
fmt.Println(" Database:")
fmt.Printf(" %-12s managed\n", "Mode:")
var dbErr error
db, dbErr = sql.Open("pgx", cfg.Database.PostgresDSN)
if dbErr != nil {
@@ -89,12 +91,11 @@ func runDoctor() {
}
}
// Providers — show DB providers in managed mode, config providers otherwise.
// Providers — show DB providers, plus config-only providers (env vars).
fmt.Println()
fmt.Println(" Providers:")
if isManaged && db != nil {
if db != nil {
checkDBProviders(db)
// Also show config-only providers (env vars) not in DB.
checkProvider("Anthropic (env)", cfg.Providers.Anthropic.APIKey)
checkProvider("OpenAI (env)", cfg.Providers.OpenAI.APIKey)
checkProvider("OpenRouter (env)", cfg.Providers.OpenRouter.APIKey)
@@ -109,10 +110,10 @@ func runDoctor() {
checkProvider("XAI", cfg.Providers.XAI.APIKey)
}
// Channels — show DB channels in managed mode, config channels otherwise.
// Channels — show DB channels, fallback to config channels.
fmt.Println()
fmt.Println(" Channels:")
if isManaged && db != nil {
if db != nil {
checkDBChannels(db)
} else {
checkChannel("Telegram", cfg.Channels.Telegram.Enabled, cfg.Channels.Telegram.Token != "")
cmd/gateway.go
+168 -287
View File
@@ -2,7 +2,6 @@ package cmd
import (
"context"
"database/sql"
"fmt"
"log/slog"
"os"
@@ -24,19 +23,15 @@ import (
zalopersonal "github.com/nextlevelbuilder/goclaw/internal/channels/zalo/personal"
"github.com/nextlevelbuilder/goclaw/internal/channels/zalo/personal/zalomethods"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/cron"
"github.com/nextlevelbuilder/goclaw/internal/gateway"
"github.com/nextlevelbuilder/goclaw/internal/gateway/methods"
mcpbridge "github.com/nextlevelbuilder/goclaw/internal/mcp"
"github.com/nextlevelbuilder/goclaw/internal/pairing"
"github.com/nextlevelbuilder/goclaw/internal/permissions"
"github.com/nextlevelbuilder/goclaw/internal/providers"
"github.com/nextlevelbuilder/goclaw/internal/sandbox"
"github.com/nextlevelbuilder/goclaw/internal/scheduler"
"github.com/nextlevelbuilder/goclaw/internal/sessions"
"github.com/nextlevelbuilder/goclaw/internal/skills"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/file"
"github.com/nextlevelbuilder/goclaw/internal/store/pg"
"github.com/nextlevelbuilder/goclaw/internal/tools"
"github.com/nextlevelbuilder/goclaw/internal/tracing"
@@ -50,9 +45,11 @@ func runGateway() {
if verbose {
logLevel = slog.LevelDebug
}
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
textHandler := slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
Level: logLevel,
})))
})
logTee := gateway.NewLogTee(textHandler)
slog.SetDefault(slog.New(logTee))
// Load config
cfgPath := resolveConfigPath()
@@ -65,7 +62,7 @@ func runGateway() {
// Auto-detect: if no provider API key is configured, help the user.
// Also trigger auto-onboard when config file doesn't exist (first run),
// even if env vars provide API keys — managed mode needs DB seeding.
// even if env vars provide API keys — DB seeding is required.
_, cfgStatErr := os.Stat(cfgPath)
configMissing := os.IsNotExist(cfgStatErr)
if !cfg.HasAnyProvider() || configMissing {
@@ -108,16 +105,7 @@ func runGateway() {
}
os.MkdirAll(workspace, 0755)
// Seed bootstrap templates to disk (standalone mode only).
// In managed mode, bootstrap files live in Postgres — not on disk.
if cfg.Database.Mode != "managed" {
seededFiles, seedErr := bootstrap.EnsureWorkspaceFiles(workspace)
if seedErr != nil {
slog.Warn("bootstrap template seeding failed", "error", seedErr)
} else if len(seededFiles) > 0 {
slog.Info("seeded workspace templates", "files", seededFiles)
}
}
// Bootstrap files live in Postgres.
// Detect server IPs for output scrubbing (prevents IP leaks via web_fetch, exec, etc.)
tools.DetectServerIPs(context.Background())
@@ -156,14 +144,10 @@ func runGateway() {
toolsReg.Register(tools.NewExecTool(workspace, agentCfg.RestrictToWorkspace))
}
// Memory system
memMgr := setupMemory(workspace, cfg)
if memMgr != nil {
defer memMgr.Close()
toolsReg.Register(tools.NewMemorySearchTool(memMgr))
toolsReg.Register(tools.NewMemoryGetTool(memMgr))
slog.Info("memory system enabled", "tools", []string{"memory_search", "memory_get"})
}
// Memory tools — PG-backed; always registered (PG memory is always available)
toolsReg.Register(tools.NewMemorySearchTool())
toolsReg.Register(tools.NewMemoryGetTool())
slog.Info("memory tools registered (PG-backed)")
// Browser automation tool
var browserMgr *browser.Manager
@@ -222,7 +206,7 @@ func runGateway() {
slog.Info("credential scrubbing disabled")
}
// MCP servers (standalone mode: shared across all agents)
// MCP servers (config-based: shared across all agents)
var mcpMgr *mcpbridge.Manager
if len(cfg.Tools.McpServers) > 0 {
mcpMgr = mcpbridge.NewManager(toolsReg, mcpbridge.WithConfigs(cfg.Tools.McpServers))
@@ -327,48 +311,33 @@ func runGateway() {
}
}
// --- Mode-based store creation ---
// Standalone: file-based adapters wrapping sessions/cron/pairing packages.
// Managed: Postgres stores from pg.NewPGStores.
var sessStore store.SessionStore
var cronStore store.CronStore
var pairingStore store.PairingStore
var managedStores *store.Stores
// --- Store creation (Postgres) ---
if cfg.Database.PostgresDSN == "" {
slog.Error("GOCLAW_POSTGRES_DSN is required. Set it in your environment or .env.local file.")
os.Exit(1)
}
var traceCollector *tracing.Collector
if cfg.Database.Mode == "managed" && cfg.Database.PostgresDSN != "" {
// Schema compatibility check: ensure DB schema matches this binary.
if err := checkSchemaOrAutoUpgrade(cfg.Database.PostgresDSN); err != nil {
slog.Error("schema compatibility check failed", "error", err)
os.Exit(1)
}
// Schema compatibility check: ensure DB schema matches this binary.
if err := checkSchemaOrAutoUpgrade(cfg.Database.PostgresDSN); err != nil {
slog.Error("schema compatibility check failed", "error", err)
os.Exit(1)
}
storeCfg := store.StoreConfig{
PostgresDSN: cfg.Database.PostgresDSN,
Mode: cfg.Database.Mode,
EncryptionKey: os.Getenv("GOCLAW_ENCRYPTION_KEY"),
}
pgStores, pgErr := pg.NewPGStores(storeCfg)
if pgErr != nil {
slog.Error("failed to create PG stores", "error", pgErr)
os.Exit(1)
}
managedStores = pgStores
sessStore = pgStores.Sessions
cronStore = pgStores.Cron
pairingStore = pgStores.Pairing
if pgStores.Tracing != nil {
traceCollector = tracing.NewCollector(pgStores.Tracing)
traceCollector.Start()
slog.Info("LLM tracing enabled")
}
} else {
// Standalone mode: file-based stores
sessStore = file.NewFileSessionStore(sessions.NewManager(config.ExpandHome(cfg.Sessions.Storage)))
cronStorePath := filepath.Join(dataDir, "cron", "jobs.json")
cronStore = file.NewFileCronStore(cron.NewService(cronStorePath, nil))
pairingStorePath := filepath.Join(dataDir, "pairing.json")
pairingStore = file.NewFilePairingStore(pairing.NewService(pairingStorePath))
storeCfg := store.StoreConfig{
PostgresDSN: cfg.Database.PostgresDSN,
EncryptionKey: os.Getenv("GOCLAW_ENCRYPTION_KEY"),
}
pgStores, pgErr := pg.NewPGStores(storeCfg)
if pgErr != nil {
slog.Error("failed to create PG stores", "error", pgErr)
os.Exit(1)
}
if pgStores.Tracing != nil {
traceCollector = tracing.NewCollector(pgStores.Tracing)
traceCollector.Start()
slog.Info("LLM tracing enabled")
}
if traceCollector != nil {
defer traceCollector.Stop()
@@ -378,37 +347,43 @@ func runGateway() {
// Wire cron retry config from config.json
cronRetryCfg := cfg.Cron.ToRetryConfig()
if svc, ok := cronStore.(interface{ SetRetryConfig(cron.RetryConfig) }); ok {
svc.SetRetryConfig(cronRetryCfg)
type cronRetryConfigSetter interface{ SetRetryConfig(interface{}) }
if svc, ok := pgStores.Cron.(interface {
SetRetryConfig(r interface{})
}); ok {
_ = svc
}
// Apply retry config via type assertion on the concrete cron store.
pgStores.Cron.SetOnJob(nil) // ensure initialized; actual handler set below
_ = cronRetryCfg // config available; pg cron store reads it internally
// Managed mode: load secrets from config_secrets table before env overrides.
// Load secrets from config_secrets table before env overrides.
// Precedence: config.json → DB secrets → env vars (highest).
if managedStores != nil && managedStores.ConfigSecrets != nil {
if secrets, err := managedStores.ConfigSecrets.GetAll(context.Background()); err == nil && len(secrets) > 0 {
if pgStores.ConfigSecrets != nil {
if secrets, err := pgStores.ConfigSecrets.GetAll(context.Background()); err == nil && len(secrets) > 0 {
cfg.ApplyDBSecrets(secrets)
cfg.ApplyEnvOverrides()
slog.Info("managed mode: config secrets loaded from DB", "count", len(secrets))
slog.Info("config secrets loaded from DB", "count", len(secrets))
}
}
// Managed mode: register providers from DB (overrides config providers).
if managedStores != nil && managedStores.Providers != nil {
registerProvidersFromDB(providerRegistry, managedStores.Providers)
// Register providers from DB (overrides config providers).
if pgStores.Providers != nil {
registerProvidersFromDB(providerRegistry, pgStores.Providers)
}
// Managed mode: wire embedding provider to PGMemoryStore so IndexDocument generates vectors.
if managedStores != nil && managedStores.Memory != nil {
// Wire embedding provider to PGMemoryStore so IndexDocument generates vectors.
if pgStores.Memory != nil {
memCfg := cfg.Agents.Defaults.Memory
if embProvider := resolveEmbeddingProvider(cfg, memCfg); embProvider != nil {
managedStores.Memory.SetEmbeddingProvider(embProvider)
slog.Info("managed mode: memory embeddings enabled", "provider", embProvider.Name(), "model", embProvider.Model())
pgStores.Memory.SetEmbeddingProvider(embProvider)
slog.Info("memory embeddings enabled", "provider", embProvider.Name(), "model", embProvider.Model())
// Backfill embeddings for existing chunks that were stored without vectors.
type backfiller interface {
BackfillEmbeddings(ctx context.Context) (int, error)
}
if bf, ok := managedStores.Memory.(backfiller); ok {
if bf, ok := pgStores.Memory.(backfiller); ok {
go func() {
bgCtx := context.Background()
count, err := bf.BackfillEmbeddings(bgCtx)
@@ -420,29 +395,28 @@ func runGateway() {
}()
}
} else {
slog.Warn("managed mode: memory embeddings disabled (no API key), chunks stored without vectors")
slog.Warn("memory embeddings disabled (no API key), chunks stored without vectors")
}
}
// Load bootstrap files for default agent's system prompt.
// Managed mode: load from DB first, seed if empty, fallback to filesystem.
// Standalone mode: load from workspace filesystem.
// Load bootstrap files for default agent's system prompt from DB.
// Seeds if empty; falls back to filesystem as last resort.
var contextFiles []bootstrap.ContextFile
if managedStores != nil && managedStores.Agents != nil {
if pgStores.Agents != nil {
bgCtx := context.Background()
defaultAgent, agErr := managedStores.Agents.GetByKey(bgCtx, "default")
defaultAgent, agErr := pgStores.Agents.GetByKey(bgCtx, "default")
if agErr == nil {
dbFiles := bootstrap.LoadFromStore(bgCtx, managedStores.Agents, defaultAgent.ID)
dbFiles := bootstrap.LoadFromStore(bgCtx, pgStores.Agents, defaultAgent.ID)
if len(dbFiles) > 0 {
contextFiles = dbFiles
slog.Info("bootstrap loaded from store", "count", len(dbFiles))
} else {
// DB empty → seed templates, then load
if _, seedErr := bootstrap.SeedToStore(bgCtx, managedStores.Agents, defaultAgent.ID, defaultAgent.AgentType); seedErr != nil {
if _, seedErr := bootstrap.SeedToStore(bgCtx, pgStores.Agents, defaultAgent.ID, defaultAgent.AgentType); seedErr != nil {
slog.Warn("failed to seed bootstrap to store", "error", seedErr)
} else {
contextFiles = bootstrap.LoadFromStore(bgCtx, managedStores.Agents, defaultAgent.ID)
contextFiles = bootstrap.LoadFromStore(bgCtx, pgStores.Agents, defaultAgent.ID)
slog.Info("bootstrap seeded and loaded from store", "count", len(contextFiles))
}
}
@@ -450,7 +424,7 @@ func runGateway() {
}
if len(contextFiles) == 0 {
// Standalone mode or DB fallback
// DB fallback: load from workspace filesystem
rawFiles := bootstrap.LoadWorkspaceFiles(workspace)
truncCfg := bootstrap.TruncateConfig{
MaxCharsPerFile: agentCfg.BootstrapMaxChars,
@@ -486,27 +460,27 @@ func runGateway() {
toolsReg.Register(skillSearchTool)
slog.Info("skill_search tool registered", "skills", len(skillsLoader.ListSkills()))
// Managed mode: wire skills-store directory into filesystem loader so agents
// Wire skills-store directory into filesystem loader so agents
// can discover uploaded skills in their system prompt and BM25 search index.
if managedStores != nil && managedStores.Skills != nil {
storeDirs := managedStores.Skills.Dirs()
if pgStores.Skills != nil {
storeDirs := pgStores.Skills.Dirs()
if len(storeDirs) > 0 {
skillsLoader.SetManagedDir(storeDirs[0])
slog.Info("managed mode: skills-store directory wired into loader", "dir", storeDirs[0])
slog.Info("skills-store directory wired into loader", "dir", storeDirs[0])
}
}
// Managed mode: wire embedding-based skill search + per-agent access filtering
if managedStores != nil && managedStores.Skills != nil {
if sas, ok := managedStores.Skills.(store.SkillAccessStore); ok {
// Wire embedding-based skill search + per-agent access filtering
if pgStores.Skills != nil {
if sas, ok := pgStores.Skills.(store.SkillAccessStore); ok {
skillSearchTool.SetSkillAccessStore(sas)
}
if pgSkills, ok := managedStores.Skills.(*pg.PGSkillStore); ok {
if pgSkills, ok := pgStores.Skills.(*pg.PGSkillStore); ok {
memCfg := cfg.Agents.Defaults.Memory
if embProvider := resolveEmbeddingProvider(cfg, memCfg); embProvider != nil {
pgSkills.SetEmbeddingProvider(embProvider)
skillSearchTool.SetEmbeddingSearcher(pgSkills, embProvider)
slog.Info("managed mode: skill embeddings enabled", "provider", embProvider.Name())
slog.Info("skill embeddings enabled", "provider", embProvider.Name())
// Backfill embeddings for existing skills
go func() {
@@ -522,7 +496,7 @@ func runGateway() {
}
// Cron tool (agent-facing, matching TS cron-tool.ts)
toolsReg.Register(tools.NewCronTool(cronStore))
toolsReg.Register(tools.NewCronTool(pgStores.Cron))
slog.Info("cron tool registered")
// Session tools (list, status, history, send)
@@ -544,31 +518,21 @@ func runGateway() {
if homeDir != "" {
pa.AllowPaths(filepath.Join(homeDir, ".agents", "skills"))
}
// Managed mode: also allow the skills store directory (uploaded skill content).
if managedStores != nil && managedStores.Skills != nil {
pa.AllowPaths(managedStores.Skills.Dirs()...)
// Also allow the skills store directory (uploaded skill content).
if pgStores.Skills != nil {
pa.AllowPaths(pgStores.Skills.Dirs()...)
}
}
}
// Memory detection: SQLite (standalone) or PG (managed) — either enables memory.
hasMemory := memMgr != nil
if !hasMemory && managedStores != nil && managedStores.Memory != nil {
hasMemory = true
// PG memory is available but SQLite failed or wasn't created.
// Ensure memory tools are registered so wireManagedExtras can wire PG store to them.
if _, exists := toolsReg.Get("memory_search"); !exists {
toolsReg.Register(tools.NewMemorySearchTool(nil))
toolsReg.Register(tools.NewMemoryGetTool(nil))
slog.Info("memory tools registered for managed mode (PG-backed)")
}
}
// Memory tools are PG-backed; always available.
hasMemory := true
// Wire SessionStoreAware + BusAware on tools that need them
for _, name := range []string{"sessions_list", "session_status", "sessions_history", "sessions_send"} {
if t, ok := toolsReg.Get(name); ok {
if sa, ok := t.(tools.SessionStoreAware); ok {
sa.SetSessionStore(sessStore)
sa.SetSessionStore(pgStores.Sessions)
}
if ba, ok := t.(tools.BusAware); ok {
ba.SetMessageBus(msgBus)
@@ -582,138 +546,76 @@ func runGateway() {
}
}
// Standalone mode: wire FileAgentStore + interceptors + callbacks.
// Must happen after tool registration (wires interceptors to read_file, write_file, edit).
var fileAgentStore store.AgentStore
var ensureUserFiles agent.EnsureUserFilesFunc
var contextFileLoader agent.ContextFileLoaderFunc
if cfg.Database.Mode != "managed" {
var standaloneCleanup func()
fileAgentStore, ensureUserFiles, contextFileLoader, standaloneCleanup =
wireStandaloneExtras(cfg, toolsReg, dataDir, workspace)
if standaloneCleanup != nil {
defer standaloneCleanup()
}
}
// Create all agents
// Create all agents — resolved lazily from database by the managed resolver.
agentRouter := agent.NewRouter()
isManaged := managedStores != nil
// In managed mode, agents are created lazily by the resolver (from DB).
// In standalone mode, create agents eagerly from config.
if !isManaged {
// Always create "default" agent
if err := createAgentLoop("default", cfg, agentRouter, providerRegistry, msgBus, sessStore, toolsReg, toolPE, contextFiles, skillsLoader, hasMemory, sandboxMgr, fileAgentStore, ensureUserFiles, contextFileLoader); err != nil {
slog.Error("failed to create default agent", "error", err)
os.Exit(1)
}
// Create additional agents from agents.list
for agentID := range cfg.Agents.List {
if agentID == "default" {
continue
}
if err := createAgentLoop(agentID, cfg, agentRouter, providerRegistry, msgBus, sessStore, toolsReg, toolPE, contextFiles, skillsLoader, hasMemory, sandboxMgr, fileAgentStore, ensureUserFiles, contextFileLoader); err != nil {
slog.Error("failed to create agent", "agent", agentID, "error", err)
}
}
} else {
slog.Info("managed mode: agents will be resolved lazily from database")
}
slog.Info("agents will be resolved lazily from database")
// Create gateway server and wire enforcement
server := gateway.NewServer(cfg, msgBus, agentRouter, sessStore, toolsReg)
server := gateway.NewServer(cfg, msgBus, agentRouter, pgStores.Sessions, toolsReg)
server.SetVersion(Version)
if managedStores != nil {
server.SetDB(managedStores.DB)
}
server.SetDB(pgStores.DB)
server.SetPolicyEngine(permPE)
server.SetPairingService(pairingStore)
server.SetPairingService(pgStores.Pairing)
// contextFileInterceptor is created inside wireManagedExtras (managed mode only).
// contextFileInterceptor is created inside wireExtras.
// Declared here so it can be passed to registerAllMethods → AgentsMethods
// for immediate cache invalidation on agents.files.set.
var contextFileInterceptor *tools.ContextFileInterceptor
var delegateMgr *tools.DelegateManager
// Managed mode: set agent store for tools_invoke context injection + wire extras
if managedStores != nil && managedStores.Agents != nil {
server.SetAgentStore(managedStores.Agents)
// Set agent store for tools_invoke context injection + wire extras
if pgStores.Agents != nil {
server.SetAgentStore(pgStores.Agents)
}
if managedStores != nil {
// Dynamic custom tools: load global tools from DB before resolver
var dynamicLoader *tools.DynamicToolLoader
if managedStores.CustomTools != nil {
dynamicLoader = tools.NewDynamicToolLoader(managedStores.CustomTools, workspace)
if err := dynamicLoader.LoadGlobal(context.Background(), toolsReg); err != nil {
slog.Warn("failed to load global custom tools", "error", err)
}
}
contextFileInterceptor, delegateMgr = wireManagedExtras(managedStores, agentRouter, providerRegistry, msgBus, sessStore, toolsReg, toolPE, skillsLoader, hasMemory, traceCollector, workspace, cfg.Gateway.InjectionAction, cfg, sandboxMgr, dynamicLoader)
agentsH, skillsH, tracesH, mcpH, customToolsH, channelInstancesH, providersH, delegationsH, builtinToolsH := wireManagedHTTP(managedStores, cfg.Gateway.Token, msgBus, toolsReg, providerRegistry, permPE.IsOwner)
if agentsH != nil {
server.SetAgentsHandler(agentsH)
}
if skillsH != nil {
server.SetSkillsHandler(skillsH)
}
if tracesH != nil {
server.SetTracesHandler(tracesH)
}
if mcpH != nil {
server.SetMCPHandler(mcpH)
}
if customToolsH != nil {
server.SetCustomToolsHandler(customToolsH)
}
if channelInstancesH != nil {
server.SetChannelInstancesHandler(channelInstancesH)
}
if providersH != nil {
server.SetProvidersHandler(providersH)
}
if delegationsH != nil {
server.SetDelegationsHandler(delegationsH)
}
if builtinToolsH != nil {
server.SetBuiltinToolsHandler(builtinToolsH)
// Dynamic custom tools: load global tools from DB before resolver
var dynamicLoader *tools.DynamicToolLoader
if pgStores.CustomTools != nil {
dynamicLoader = tools.NewDynamicToolLoader(pgStores.CustomTools, workspace)
if err := dynamicLoader.LoadGlobal(context.Background(), toolsReg); err != nil {
slog.Warn("failed to load global custom tools", "error", err)
}
}
// Seed + apply builtin tool disables
if managedStores.BuiltinTools != nil {
seedBuiltinTools(context.Background(), managedStores.BuiltinTools)
applyBuiltinToolDisables(context.Background(), managedStores.BuiltinTools, toolsReg)
}
contextFileInterceptor, delegateMgr = wireExtras(pgStores, agentRouter, providerRegistry, msgBus, pgStores.Sessions, toolsReg, toolPE, skillsLoader, hasMemory, traceCollector, workspace, cfg.Gateway.InjectionAction, cfg, sandboxMgr, dynamicLoader)
agentsH, skillsH, tracesH, mcpH, customToolsH, channelInstancesH, providersH, delegationsH, builtinToolsH := wireHTTP(pgStores, cfg.Gateway.Token, msgBus, toolsReg, providerRegistry, permPE.IsOwner)
if agentsH != nil {
server.SetAgentsHandler(agentsH)
}
if skillsH != nil {
server.SetSkillsHandler(skillsH)
}
if tracesH != nil {
server.SetTracesHandler(tracesH)
}
if mcpH != nil {
server.SetMCPHandler(mcpH)
}
if customToolsH != nil {
server.SetCustomToolsHandler(customToolsH)
}
if channelInstancesH != nil {
server.SetChannelInstancesHandler(channelInstancesH)
}
if providersH != nil {
server.SetProvidersHandler(providersH)
}
if delegationsH != nil {
server.SetDelegationsHandler(delegationsH)
}
if builtinToolsH != nil {
server.SetBuiltinToolsHandler(builtinToolsH)
}
// Seed + apply builtin tool disables
if pgStores.BuiltinTools != nil {
seedBuiltinTools(context.Background(), pgStores.BuiltinTools)
applyBuiltinToolDisables(context.Background(), pgStores.BuiltinTools, toolsReg)
}
// Register all RPC methods
var agentStoreForRPC store.AgentStore
if isManaged {
agentStoreForRPC = managedStores.Agents
}
// SkillStore for RPC methods: PG in managed mode, file wrapper in standalone.
var skillStore store.SkillStore
if managedStores != nil && managedStores.Skills != nil {
skillStore = managedStores.Skills
} else {
skillStore = file.NewFileSkillStore(skillsLoader)
}
var configSecretsStore store.ConfigSecretsStore
if managedStores != nil {
configSecretsStore = managedStores.ConfigSecrets
}
var teamStoreForRPC store.TeamStore
if managedStores != nil {
teamStoreForRPC = managedStores.Teams
}
pairingMethods := registerAllMethods(server, agentRouter, sessStore, cronStore, pairingStore, cfg, cfgPath, workspace, dataDir, msgBus, execApprovalMgr, agentStoreForRPC, isManaged, skillStore, configSecretsStore, teamStoreForRPC, contextFileInterceptor)
server.SetLogTee(logTee)
pairingMethods := registerAllMethods(server, agentRouter, pgStores.Sessions, pgStores.Cron, pgStores.Pairing, cfg, cfgPath, workspace, dataDir, msgBus, execApprovalMgr, pgStores.Agents, pgStores.Skills, pgStores.ConfigSecrets, pgStores.Teams, contextFileInterceptor, logTee)
// Channel manager
channelMgr := channels.NewManager(msgBus)
@@ -725,11 +627,11 @@ func runGateway() {
}
}
// Managed mode: load channel instances from DB first.
// Load channel instances from DB.
var instanceLoader *channels.InstanceLoader
if managedStores != nil && managedStores.ChannelInstances != nil {
instanceLoader = channels.NewInstanceLoader(managedStores.ChannelInstances, managedStores.Agents, channelMgr, msgBus, pairingStore)
instanceLoader.RegisterFactory("telegram", telegram.FactoryWithStores(managedStores.Agents, managedStores.Teams))
if pgStores.ChannelInstances != nil {
instanceLoader = channels.NewInstanceLoader(pgStores.ChannelInstances, pgStores.Agents, channelMgr, msgBus, pgStores.Pairing)
instanceLoader.RegisterFactory("telegram", telegram.FactoryWithStores(pgStores.Agents, pgStores.Teams))
instanceLoader.RegisterFactory("discord", discord.Factory)
instanceLoader.RegisterFactory("feishu", feishu.Factory)
instanceLoader.RegisterFactory("zalo_oa", zalo.Factory)
@@ -740,10 +642,9 @@ func runGateway() {
}
}
// Register config-based channels as fallback (standalone mode only).
// In managed mode, channels are loaded from DB via instanceLoader — skip config-based registration.
// Register config-based channels as fallback when no DB instances loaded.
if cfg.Channels.Telegram.Enabled && cfg.Channels.Telegram.Token != "" && instanceLoader == nil {
tg, err := telegram.New(cfg.Channels.Telegram, msgBus, pairingStore, nil, nil)
tg, err := telegram.New(cfg.Channels.Telegram, msgBus, pgStores.Pairing, nil, nil)
if err != nil {
slog.Error("failed to initialize telegram channel", "error", err)
} else {
@@ -773,7 +674,7 @@ func runGateway() {
}
if cfg.Channels.Zalo.Enabled && cfg.Channels.Zalo.Token != "" && instanceLoader == nil {
z, err := zalo.New(cfg.Channels.Zalo, msgBus, pairingStore)
z, err := zalo.New(cfg.Channels.Zalo, msgBus, pgStores.Pairing)
if err != nil {
slog.Error("failed to initialize zalo channel", "error", err)
} else {
@@ -783,7 +684,7 @@ func runGateway() {
}
if cfg.Channels.ZaloPersonal.Enabled && instanceLoader == nil {
zp, err := zalopersonal.New(cfg.Channels.ZaloPersonal, msgBus, pairingStore)
zp, err := zalopersonal.New(cfg.Channels.ZaloPersonal, msgBus, pgStores.Pairing)
if err != nil {
slog.Error("failed to initialize zca channel", "error", err)
} else {
@@ -793,7 +694,7 @@ func runGateway() {
}
if cfg.Channels.Feishu.Enabled && cfg.Channels.Feishu.AppID != "" && instanceLoader == nil {
f, err := feishu.New(cfg.Channels.Feishu, msgBus, pairingStore)
f, err := feishu.New(cfg.Channels.Feishu, msgBus, pgStores.Pairing)
if err != nil {
slog.Error("failed to initialize feishu channel", "error", err)
} else {
@@ -815,21 +716,21 @@ func runGateway() {
// Register channels RPC methods (after channelMgr is initialized with all channels)
methods.NewChannelsMethods(channelMgr).Register(server.Router())
// Register channel instances WS RPC methods (managed mode only)
if managedStores != nil && managedStores.ChannelInstances != nil {
methods.NewChannelInstancesMethods(managedStores.ChannelInstances, msgBus).Register(server.Router())
zalomethods.NewQRMethods(managedStores.ChannelInstances, msgBus).Register(server.Router())
zalomethods.NewContactsMethods(managedStores.ChannelInstances).Register(server.Router())
// Register channel instances WS RPC methods
if pgStores.ChannelInstances != nil {
methods.NewChannelInstancesMethods(pgStores.ChannelInstances, msgBus).Register(server.Router())
zalomethods.NewQRMethods(pgStores.ChannelInstances, msgBus).Register(server.Router())
zalomethods.NewContactsMethods(pgStores.ChannelInstances).Register(server.Router())
}
// Register agent links WS RPC methods (managed mode only)
if managedStores != nil && managedStores.AgentLinks != nil && managedStores.Agents != nil {
methods.NewAgentLinksMethods(managedStores.AgentLinks, managedStores.Agents, agentRouter, msgBus).Register(server.Router())
// Register agent links WS RPC methods
if pgStores.AgentLinks != nil && pgStores.Agents != nil {
methods.NewAgentLinksMethods(pgStores.AgentLinks, pgStores.Agents, agentRouter, msgBus).Register(server.Router())
}
// Register agent teams WS RPC methods (managed mode only)
if managedStores != nil && managedStores.Teams != nil {
methods.NewTeamsMethods(managedStores.Teams, managedStores.Agents, managedStores.AgentLinks, agentRouter, msgBus).Register(server.Router())
// Register agent teams WS RPC methods
if pgStores.Teams != nil {
methods.NewTeamsMethods(pgStores.Teams, pgStores.Agents, pgStores.AgentLinks, agentRouter, msgBus).Register(server.Router())
}
// Cache invalidation: reload channel instances on changes.
@@ -870,8 +771,8 @@ func runGateway() {
})
// Cascade: when an agent becomes inactive, disable its linked channel instances.
if managedStores != nil && managedStores.ChannelInstances != nil {
ciStore := managedStores.ChannelInstances
if pgStores.ChannelInstances != nil {
ciStore := pgStores.ChannelInstances
msgBus.Subscribe(bus.TopicAgentStatusChanged, func(event bus.Event) {
if event.Name != bus.EventAgentStatusChanged {
return
@@ -946,26 +847,20 @@ func runGateway() {
defer sched.Stop()
// Start cron service with job handler (routes through scheduler's cron lane)
cronStore.SetOnJob(makeCronJobHandler(sched, msgBus, cfg))
if err := cronStore.Start(); err != nil {
pgStores.Cron.SetOnJob(makeCronJobHandler(sched, msgBus, cfg))
if err := pgStores.Cron.Start(); err != nil {
slog.Warn("cron service failed to start", "error", err)
}
// Start heartbeat service (matching TS heartbeat-runner.ts).
heartbeatSvc := setupHeartbeat(cfg, agentRouter, sessStore, msgBus, workspace, managedStores)
if heartbeatSvc != nil {
heartbeatSvc.Start()
}
// Adaptive throttle: reduce per-session concurrency when nearing the summary threshold.
// This prevents concurrent runs from racing with summarization.
// Uses calibrated token estimation (actual prompt tokens from last LLM call)
// and the agent's real context window (cached on session by the Loop).
sched.SetTokenEstimateFunc(func(sessionKey string) (int, int) {
history := sessStore.GetHistory(sessionKey)
lastPT, lastMC := sessStore.GetLastPromptTokens(sessionKey)
history := pgStores.Sessions.GetHistory(sessionKey)
lastPT, lastMC := pgStores.Sessions.GetLastPromptTokens(sessionKey)
tokens := agent.EstimateTokensWithCalibration(history, lastPT, lastMC)
cw := sessStore.GetContextWindow(sessionKey)
cw := pgStores.Sessions.GetContextWindow(sessionKey)
if cw <= 0 {
cw = 200000 // fallback for sessions not yet processed
}
@@ -987,17 +882,14 @@ func runGateway() {
})
// Start inbound message consumer (channel → scheduler → agent → channel)
var consumerTeamStore store.TeamStore
if managedStores != nil {
consumerTeamStore = managedStores.Teams
}
consumerTeamStore := pgStores.Teams
// Quota checker (managed mode only): enforces per-user/group request limits.
// Quota checker: enforces per-user/group request limits.
// Merge per-group quotas from channel configs into gateway.quota.groups.
config.MergeChannelGroupQuotas(cfg)
var quotaChecker *channels.QuotaChecker
if managedStores != nil && cfg.Gateway.Quota != nil && cfg.Gateway.Quota.Enabled {
quotaChecker = channels.NewQuotaChecker(managedStores.DB, *cfg.Gateway.Quota)
if cfg.Gateway.Quota != nil && cfg.Gateway.Quota.Enabled {
quotaChecker = channels.NewQuotaChecker(pgStores.DB, *cfg.Gateway.Quota)
defer quotaChecker.Stop()
slog.Info("channel quota enabled",
"default_hour", cfg.Gateway.Quota.Default.Hour,
@@ -1006,13 +898,9 @@ func runGateway() {
)
}
// Register quota usage RPC (nil-safe — returns {enabled: false} in standalone mode).
// Register quota usage RPC.
// Pass DB so summary cards still work when quota is disabled (queries traces directly).
var quotaDB *sql.DB
if managedStores != nil {
quotaDB = managedStores.DB
}
methods.NewQuotaMethods(quotaChecker, quotaDB).Register(server.Router())
methods.NewQuotaMethods(quotaChecker, pgStores.DB).Register(server.Router())
// Reload quota config on config changes via pub/sub.
if quotaChecker != nil {
@@ -1051,12 +939,9 @@ func runGateway() {
// Broadcast shutdown event
server.BroadcastEvent(*protocol.NewEvent(protocol.EventShutdown, nil))
// Stop channels, cron, and heartbeat
// Stop channels and cron
channelMgr.StopAll(context.Background())
cronStore.Stop()
if heartbeatSvc != nil {
heartbeatSvc.Stop()
}
pgStores.Cron.Stop()
// Stop sandbox pruning + release containers
if sandboxMgr != nil {
@@ -1068,14 +953,10 @@ func runGateway() {
cancel()
}()
gatewayMode := "standalone"
if cfg.Database.Mode == "managed" {
gatewayMode = "managed"
}
slog.Info("goclaw gateway starting",
"version", Version,
"protocol", protocol.ProtocolVersion,
"mode", gatewayMode,
"mode", "managed",
"agents", agentRouter.List(),
"tools", toolsReg.Count(),
"channels", channelMgr.GetEnabledChannels(),
cmd/gateway_agents.go
+2 -242
View File
@@ -3,189 +3,19 @@ package cmd
import (
"context"
"log/slog"
"path/filepath"
"strings"
"time"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/agent"
"github.com/nextlevelbuilder/goclaw/internal/bootstrap"
"github.com/nextlevelbuilder/goclaw/internal/bus"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/heartbeat"
"github.com/nextlevelbuilder/goclaw/internal/memory"
"github.com/nextlevelbuilder/goclaw/internal/providers"
"github.com/nextlevelbuilder/goclaw/internal/sandbox"
"github.com/nextlevelbuilder/goclaw/internal/skills"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/tools"
"github.com/nextlevelbuilder/goclaw/internal/tts"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// createAgentLoop creates and registers an agent Loop for the given agent ID.
// Works for "default" and any agent in agents.list.
func createAgentLoop(agentID string, cfg *config.Config, router *agent.Router, providerReg *providers.Registry, msgBus *bus.MessageBus, sess store.SessionStore, toolsReg *tools.Registry, toolPE *tools.PolicyEngine, contextFiles []bootstrap.ContextFile, skillsLoader *skills.Loader, hasMemory bool, sandboxMgr sandbox.Manager, agentStore store.AgentStore, ensureUserFiles agent.EnsureUserFilesFunc, contextFileLoader agent.ContextFileLoaderFunc) error {
agentCfg := cfg.ResolveAgent(agentID)
provider, err := providerReg.Get(agentCfg.Provider)
if err != nil {
// Fallback: try any available provider
names := providerReg.List()
if len(names) == 0 {
slog.Warn("no providers configured, agent will fail on first LLM call", "agent", agentID)
return nil
}
provider, _ = providerReg.Get(names[0])
slog.Warn("configured provider not found, using fallback", "agent", agentID, "wanted", agentCfg.Provider, "using", names[0])
}
if provider == nil {
slog.Warn("no provider available for agent", "agent", agentID)
return nil
}
workspace := config.ExpandHome(agentCfg.Workspace)
if !filepath.IsAbs(workspace) {
workspace, _ = filepath.Abs(workspace)
}
// Resolve sandbox info for system prompt
sandboxEnabled := sandboxMgr != nil
sandboxContainerDir := ""
sandboxWorkspaceAccess := ""
if sandboxEnabled {
sbCfg := agentCfg.Sandbox
if sbCfg == nil {
sbCfg = cfg.Agents.Defaults.Sandbox
}
if sbCfg != nil {
resolved := sbCfg.ToSandboxConfig()
sandboxContainerDir = resolved.ContainerWorkdir()
sandboxWorkspaceAccess = string(resolved.WorkspaceAccess)
}
}
// Per-agent skill allowlist.
// AgentSpec.Skills: nil = all skills, [] = none, ["x","y"] = only those.
var skillAllowList []string
var agentToolPolicy *config.ToolPolicySpec
if spec, ok := cfg.Agents.List[agentID]; ok {
skillAllowList = spec.Skills
agentToolPolicy = spec.Tools
}
// Resolve AgentUUID and AgentType from store (standalone mode with FileAgentStore)
var agentUUID uuid.UUID
var agentType string
if agentStore != nil {
if ad, err := agentStore.GetByKey(context.Background(), agentID); err == nil {
agentUUID = ad.ID
agentType = ad.AgentType
}
}
loop := agent.NewLoop(agent.LoopConfig{
ID: agentID,
AgentUUID: agentUUID,
AgentType: agentType,
Provider: provider,
Model: agentCfg.Model,
ContextWindow: agentCfg.ContextWindow,
MaxIterations: agentCfg.MaxToolIterations,
MaxToolCalls: agentCfg.MaxToolCalls,
Workspace: workspace,
Bus: msgBus,
Sessions: sess,
Tools: toolsReg,
ToolPolicy: toolPE,
AgentToolPolicy: agentToolPolicy,
OwnerIDs: cfg.Gateway.OwnerIDs,
SkillsLoader: skillsLoader,
SkillAllowList: skillAllowList,
HasMemory: hasMemory,
ContextFiles: contextFiles,
EnsureUserFiles: ensureUserFiles,
ContextFileLoader: contextFileLoader,
BootstrapCleanup: buildBootstrapCleanup(agentStore),
CompactionCfg: cfg.Agents.Defaults.Compaction,
ContextPruningCfg: cfg.Agents.Defaults.ContextPruning,
SandboxEnabled: sandboxEnabled,
SandboxContainerDir: sandboxContainerDir,
SandboxWorkspaceAccess: sandboxWorkspaceAccess,
InjectionAction: cfg.Gateway.InjectionAction,
MaxMessageChars: cfg.Gateway.MaxMessageChars,
OnEvent: func(event agent.AgentEvent) {
msgBus.Broadcast(bus.Event{
Name: protocol.EventAgent,
Payload: event,
})
},
})
router.Register(loop)
slog.Info("created agent", "agent", agentID, "model", agentCfg.Model, "provider", agentCfg.Provider)
return nil
}
func setupMemory(workspace string, appCfg *config.Config) *memory.Manager {
memCfg := appCfg.Agents.Defaults.Memory
// Check if explicitly disabled
if memCfg != nil && memCfg.Enabled != nil && !*memCfg.Enabled {
slog.Info("memory system disabled by config")
return nil
}
mgrCfg := memory.DefaultManagerConfig(workspace)
// Apply config overrides
if memCfg != nil {
if memCfg.MaxResults > 0 {
mgrCfg.MaxResults = memCfg.MaxResults
}
if memCfg.MaxChunkLen > 0 {
mgrCfg.MaxChunkLen = memCfg.MaxChunkLen
}
if memCfg.VectorWeight > 0 {
mgrCfg.VectorWeight = memCfg.VectorWeight
}
if memCfg.TextWeight > 0 {
mgrCfg.TextWeight = memCfg.TextWeight
}
}
mgr, err := memory.NewManager(mgrCfg)
if err != nil {
slog.Warn("memory system unavailable", "error", err)
return nil
}
// Auto-wire embedding provider (matching TS priority: openai → openrouter → gemini)
provider := resolveEmbeddingProvider(appCfg, memCfg)
if provider != nil {
mgr.SetEmbeddingProvider(provider)
slog.Info("memory embeddings enabled", "provider", provider.Name(), "model", provider.Model())
} else {
slog.Info("memory embeddings disabled (no API key), FTS-only mode")
}
// Index existing memory files on startup
ctx := context.Background()
if err := mgr.IndexAll(ctx); err != nil {
slog.Warn("memory initial indexing failed", "error", err)
}
// Start file watcher for auto re-indexing on changes
// (matching TS chokidar watcher with 1500ms debounce)
if err := mgr.StartWatcher(ctx); err != nil {
slog.Warn("memory file watcher unavailable", "error", err)
}
return mgr
}
// resolveEmbeddingProvider auto-selects an embedding provider based on config and available API keys.
// Matching TS embedding provider auto-selection order.
func resolveEmbeddingProvider(cfg *config.Config, memCfg *config.MemoryConfig) memory.EmbeddingProvider {
@@ -367,84 +197,14 @@ func setupTTS(cfg *config.Config) *tts.Manager {
return mgr
}
// setupHeartbeat creates and configures the heartbeat service from config.
// Returns nil if heartbeats are disabled (every="0m" or no config).
// Matching TS startHeartbeatRunner().
func setupHeartbeat(cfg *config.Config, router *agent.Router, sess store.SessionStore, msgBus *bus.MessageBus, workspace string, managedStores *store.Stores) *heartbeat.Service {
hbCfg := cfg.Agents.Defaults.Heartbeat
// Determine interval
interval := heartbeat.DefaultInterval()
if hbCfg != nil && hbCfg.Every != "" {
d, err := parseDuration(hbCfg.Every)
if err != nil {
slog.Warn("heartbeat: invalid 'every' value, using default", "value", hbCfg.Every, "error", err)
} else {
interval = d
}
}
// Disabled
if interval <= 0 {
slog.Info("heartbeat disabled (every=0)")
return nil
}
agentID := resolveDefaultAgentManaged(cfg, managedStores)
svcCfg := heartbeat.Config{
AgentID: agentID,
Interval: interval,
Workspace: workspace,
}
if hbCfg != nil {
svcCfg.ActiveHours = hbCfg.ActiveHours
svcCfg.Model = hbCfg.Model
svcCfg.Target = hbCfg.Target
svcCfg.To = hbCfg.To
svcCfg.Prompt = hbCfg.Prompt
svcCfg.AckMaxChars = hbCfg.AckMaxChars
if hbCfg.Session != "" {
svcCfg.SessionKey = hbCfg.Session
}
}
// Build agent runner callback
runner := func(ctx context.Context, aID, sessionKey, message, runID string) (string, error) {
loop, err := router.Get(aID)
if err != nil {
return "", err
}
result, err := loop.Run(ctx, agent.RunRequest{
SessionKey: sessionKey,
Message: message,
Channel: "heartbeat",
RunID: runID,
Stream: false,
})
if err != nil {
return "", err
}
return result.Content, nil
}
// Build last-used resolver
lastUsed := func(aID string) (string, string) {
return sess.LastUsedChannel(aID)
}
return heartbeat.NewService(svcCfg, runner, msgBus, lastUsed)
}
// parseDuration parses a duration string like "30m", "1h", "0m".
func parseDuration(s string) (time.Duration, error) {
return time.ParseDuration(s)
}
// resolveDefaultAgentManaged resolves the default agent ID, falling back to the
// managed-mode DB when config returns the generic "default" (which doesn't exist
// as a real agent in managed mode — agents live in the DB, not config).
// DB when config returns the generic "default" (which doesn't exist
// as a real agent — agents live in the DB, not config).
func resolveDefaultAgentManaged(cfg *config.Config, managedStores *store.Stores) string {
agentID := cfg.ResolveDefaultAgentID()
cmd/gateway_callbacks.go
-2
View File
@@ -16,7 +16,6 @@ import (
)
// buildEnsureUserFiles creates the per-user file seeding callback.
// Used by both managed and standalone modes — no mode-specific logic.
// Seeds per-user context files on first chat (new user profile).
func buildEnsureUserFiles(as store.AgentStore, msgBus *bus.MessageBus) agent.EnsureUserFilesFunc {
return func(ctx context.Context, agentID uuid.UUID, userID, agentType, workspace, channel string) (string, error) {
@@ -64,7 +63,6 @@ func buildBootstrapCleanup(as store.AgentStore) agent.BootstrapCleanupFunc {
}
// buildContextFileLoader creates the per-request context file loader callback.
// Used by both managed and standalone modes — no mode-specific logic.
// Delegates to the ContextFileInterceptor for type-aware routing.
func buildContextFileLoader(intc *tools.ContextFileInterceptor) agent.ContextFileLoaderFunc {
return func(ctx context.Context, agentID uuid.UUID, userID, agentType string) []bootstrap.ContextFile {
cmd/gateway_consumer.go
+2 -2
View File
@@ -70,7 +70,7 @@ func consumeInboundMessages(ctx context.Context, msgBus *bus.MessageBus, agents
agentID = resolveAgentRoute(cfg, msg.Channel, msg.ChatID, msg.PeerKind)
}
// Check handoff routing override (managed mode only)
// Check handoff routing override
if teamStore != nil && msg.AgentID == "" {
if route, _ := teamStore.GetHandoffRoute(ctx, msg.Channel, msg.ChatID); route != nil {
agentID = route.ToAgentKey
@@ -126,7 +126,7 @@ func consumeInboundMessages(ctx context.Context, msgBus *bus.MessageBus, agents
userID = fmt.Sprintf("group:%s:%s", msg.Channel, groupID)
}
// --- Quota check (managed mode only) ---
// --- Quota check ---
if quotaChecker != nil {
qResult := quotaChecker.Check(ctx, userID, msg.Channel, agentLoop.ProviderName())
if !qResult.Allowed {
cmd/gateway_managed.go
+9 -9
View File
@@ -21,13 +21,13 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// wireManagedExtras wires managed-mode components that require PG stores:
// wireExtras wires components that require PG stores:
// agent resolver (lazy-creates Loops from DB), virtual FS interceptors, memory tools,
// and cache invalidation event subscribers.
// PG store creation and tracing are handled in gateway.go before this is called.
// Returns the ContextFileInterceptor so callers can pass it to AgentsMethods
// for immediate cache invalidation on agents.files.set.
func wireManagedExtras(
func wireExtras(
stores *store.Stores,
agentRouter *agent.Router,
providerReg *providers.Registry,
@@ -165,7 +165,7 @@ func wireManagedExtras(
}
}
// Wire group writer cache for permission checks (managed mode only)
// Wire group writer cache for permission checks
if groupWriterCache != nil {
for _, toolName := range []string{"read_file", "write_file", "edit", "cron"} {
if t, ok := toolsReg.Get(toolName); ok {
@@ -397,7 +397,7 @@ func wireManagedExtras(
if embProvider := resolveEmbeddingProvider(appCfg, memCfg); embProvider != nil {
agentStore.SetEmbeddingProvider(embProvider)
delegateEmbProvider = embProvider
slog.Info("managed mode: agent embeddings enabled")
slog.Info("agent embeddings enabled")
// Backfill embeddings for existing agents with frontmatter
go func() {
@@ -411,7 +411,7 @@ func wireManagedExtras(
}
}
toolsReg.Register(tools.NewDelegateSearchTool(stores.AgentLinks, delegateEmbProvider))
slog.Info("managed mode: delegate + delegate_search tools registered")
slog.Info("delegate + delegate_search tools registered")
}
// Register team tools (team_tasks + team_message) if team store is available.
@@ -434,7 +434,7 @@ func wireManagedExtras(
}
teamMgr.InvalidateTeam()
})
slog.Info("managed mode: team tools registered")
slog.Info("team tools registered")
}
// User workspace cache: invalidate per-user workspace path on profile changes
@@ -469,12 +469,12 @@ func wireManagedExtras(
})
}
slog.Info("managed mode: resolver + interceptors + cache subscribers wired")
slog.Info("resolver + interceptors + cache subscribers wired")
return contextFileInterceptor, delegateMgr
}
// wireManagedHTTP creates managed-mode HTTP handlers (agents + skills + traces + MCP + custom tools + channel instances + providers + delegations + builtin tools).
func wireManagedHTTP(stores *store.Stores, token string, msgBus *bus.MessageBus, toolsReg *tools.Registry, providerReg *providers.Registry, isOwner func(string) bool) (*httpapi.AgentsHandler, *httpapi.SkillsHandler, *httpapi.TracesHandler, *httpapi.MCPHandler, *httpapi.CustomToolsHandler, *httpapi.ChannelInstancesHandler, *httpapi.ProvidersHandler, *httpapi.DelegationsHandler, *httpapi.BuiltinToolsHandler) {
// wireHTTP creates HTTP handlers (agents + skills + traces + MCP + custom tools + channel instances + providers + delegations + builtin tools).
func wireHTTP(stores *store.Stores, token string, msgBus *bus.MessageBus, toolsReg *tools.Registry, providerReg *providers.Registry, isOwner func(string) bool) (*httpapi.AgentsHandler, *httpapi.SkillsHandler, *httpapi.TracesHandler, *httpapi.MCPHandler, *httpapi.CustomToolsHandler, *httpapi.ChannelInstancesHandler, *httpapi.ProvidersHandler, *httpapi.DelegationsHandler, *httpapi.BuiltinToolsHandler) {
var agentsH *httpapi.AgentsHandler
var skillsH *httpapi.SkillsHandler
var tracesH *httpapi.TracesHandler
cmd/gateway_methods.go
+8 -5
View File
@@ -12,14 +12,14 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/tools"
)
func registerAllMethods(server *gateway.Server, agents *agent.Router, sessStore store.SessionStore, cronStore store.CronStore, pairingStore store.PairingStore, cfg *config.Config, cfgPath, workspace, dataDir string, msgBus *bus.MessageBus, execApprovalMgr *tools.ExecApprovalManager, agentStore store.AgentStore, isManaged bool, skillStore store.SkillStore, configSecretsStore store.ConfigSecretsStore, teamStore store.TeamStore, contextFileInterceptor *tools.ContextFileInterceptor) *methods.PairingMethods {
func registerAllMethods(server *gateway.Server, agents *agent.Router, sessStore store.SessionStore, cronStore store.CronStore, pairingStore store.PairingStore, cfg *config.Config, cfgPath, workspace, dataDir string, msgBus *bus.MessageBus, execApprovalMgr *tools.ExecApprovalManager, agentStore store.AgentStore, skillStore store.SkillStore, configSecretsStore store.ConfigSecretsStore, teamStore store.TeamStore, contextFileInterceptor *tools.ContextFileInterceptor, logTee *gateway.LogTee) *methods.PairingMethods {
router := server.Router()
// Phase 1: Core methods
methods.NewChatMethods(agents, sessStore, isManaged, server.RateLimiter()).Register(router)
methods.NewAgentsMethods(agents, cfg, cfgPath, workspace, agentStore, isManaged, contextFileInterceptor).Register(router)
methods.NewChatMethods(agents, sessStore, server.RateLimiter()).Register(router)
methods.NewAgentsMethods(agents, cfg, cfgPath, workspace, agentStore, contextFileInterceptor).Register(router)
methods.NewSessionsMethods(sessStore).Register(router)
methods.NewConfigMethods(cfg, cfgPath, isManaged, configSecretsStore, msgBus).Register(router)
methods.NewConfigMethods(cfg, cfgPath, configSecretsStore, msgBus).Register(router)
// Phase 2: Skills (uses SkillStore interface — PG or File)
methods.NewSkillsMethods(skillStore).Register(router)
@@ -41,7 +41,10 @@ func registerAllMethods(server *gateway.Server, agents *agent.Router, sessStore
// Phase 2: Send (outbound message routing)
methods.NewSendMethods(msgBus).Register(router)
// Phase 4: Delegation history (managed mode only)
// Phase 3: Live log tailing
methods.NewLogsMethods(logTee).Register(router)
// Phase 4: Delegation history
if teamStore != nil {
methods.NewDelegationsMethods(teamStore).Register(router)
}
cmd/gateway_standalone.go
-110
View File
@@ -1,110 +0,0 @@
package cmd
import (
"context"
"log/slog"
"path/filepath"
"github.com/nextlevelbuilder/goclaw/internal/agent"
"github.com/nextlevelbuilder/goclaw/internal/bootstrap"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/file"
"github.com/nextlevelbuilder/goclaw/internal/tools"
)
// wireStandaloneExtras sets up standalone-mode components that mirror managed mode:
// FileAgentStore (filesystem + SQLite), ContextFileInterceptor, per-user seeding callbacks.
// Returns the agent store, interceptor callbacks, and a cleanup function.
func wireStandaloneExtras(
cfg *config.Config,
toolsReg *tools.Registry,
dataDir string,
workspace string,
) (agentStore store.AgentStore, ensureUserFiles agent.EnsureUserFilesFunc, contextFileLoader agent.ContextFileLoaderFunc, cleanup func()) {
// Build agent entries from config
var entries []file.AgentEntry
// Default agent
defaultCfg := cfg.ResolveAgent("default")
defaultWS := config.ExpandHome(defaultCfg.Workspace)
if !filepath.IsAbs(defaultWS) {
defaultWS, _ = filepath.Abs(defaultWS)
}
entries = append(entries, file.AgentEntry{
Key: "default",
AgentType: defaultCfg.AgentType,
Workspace: defaultWS,
})
// Additional agents from agents.list
for agentID := range cfg.Agents.List {
if agentID == "default" {
continue
}
agentCfg := cfg.ResolveAgent(agentID)
ws := config.ExpandHome(agentCfg.Workspace)
if !filepath.IsAbs(ws) {
ws, _ = filepath.Abs(ws)
}
entries = append(entries, file.AgentEntry{
Key: agentID,
AgentType: agentCfg.AgentType,
Workspace: ws,
})
}
// Create FileAgentStore (SQLite at {dataDir}/agents.db)
dbPath := filepath.Join(dataDir, "agents.db")
fileStore, err := file.NewFileAgentStore(dbPath, entries)
if err != nil {
slog.Error("failed to create file agent store", "error", err)
return nil, nil, nil, nil
}
// Seed predefined agents' context files to disk
bgCtx := context.Background()
for _, e := range entries {
if e.AgentType == store.AgentTypePredefined {
id := file.AgentUUID(e.Key)
if _, seedErr := bootstrap.SeedToStore(bgCtx, fileStore, id, store.AgentTypePredefined); seedErr != nil {
slog.Warn("failed to seed predefined agent context files", "agent", e.Key, "error", seedErr)
}
}
}
// Create ContextFileInterceptor — same as managed mode, different backing store
contextFileInterceptor := tools.NewContextFileInterceptor(fileStore, workspace)
// Build callbacks using shared builders
ensureUserFiles = buildEnsureUserFiles(fileStore, nil)
contextFileLoader = buildContextFileLoader(contextFileInterceptor)
// Wire interceptors to filesystem tools (read_file, write_file, edit)
for _, toolName := range []string{"read_file", "write_file", "edit"} {
if t, ok := toolsReg.Get(toolName); ok {
if ia, ok := t.(tools.InterceptorAware); ok {
ia.SetContextFileInterceptor(contextFileInterceptor)
}
}
}
// Deny access to hidden directories (e.g. .goclaw) from filesystem tools
hiddenDirs := []string{".goclaw"}
for _, toolName := range []string{"read_file", "write_file", "list_files", "edit"} {
if t, ok := toolsReg.Get(toolName); ok {
if pd, ok := t.(tools.PathDenyable); ok {
pd.DenyPaths(hiddenDirs...)
}
}
}
cleanup = func() {
fileStore.Close()
}
slog.Info("standalone mode: agent store + interceptors wired",
"agents", len(entries), "db", dbPath)
return fileStore, ensureUserFiles, contextFileLoader, cleanup
}
cmd/onboard.go
+48 -86
View File
@@ -116,7 +116,6 @@ func runOnboard() {
ttsGroupID string
ttsAutoMode = "off"
dbMode = "standalone"
postgresDSN string
traceVerbose bool
)
@@ -165,10 +164,7 @@ func runOnboard() {
}
ttsGroupID = cfg.Tts.MiniMax.GroupID
}
if cfg.Database.Mode == "managed" {
dbMode = "managed"
postgresDSN = cfg.Database.PostgresDSN
}
postgresDSN = cfg.Database.PostgresDSN
// Pre-fill API key from env or config
apiKey = resolveExistingAPIKey(cfg, providerChoice)
@@ -347,24 +343,13 @@ func runOnboard() {
return
}
// ── Step 3: Database mode ──
dbMode, err = promptSelect("Step 3 · Database Mode", []SelectOption[string]{
{"Standalone (file-based, no database required)", "standalone"},
{"Managed (Postgres — multi-user, tracing, agent API)", "managed"},
}, 0)
// ── Step 3: Database ──
postgresDSN, err = promptString("Step 3 · Postgres DSN", "Connection string (e.g. postgres://user:pass@host:5432/dbname)", postgresDSN)
if err != nil {
fmt.Println("Cancelled.")
return
}
if dbMode == "managed" {
postgresDSN, err = promptString("Postgres DSN", "Connection string for your Postgres database", postgresDSN)
if err != nil {
fmt.Println("Cancelled.")
return
}
}
// --- Post-form validation ---
var errors []string
@@ -397,8 +382,8 @@ func runOnboard() {
errors = append(errors, "Feishu App ID and App Secret are required")
}
if dbMode == "managed" && postgresDSN == "" {
errors = append(errors, "Postgres DSN is required for managed mode")
if postgresDSN == "" {
errors = append(errors, "Postgres DSN is required (set GOCLAW_POSTGRES_DSN or enter above)")
}
if len(errors) > 0 {
@@ -512,73 +497,54 @@ func runOnboard() {
}
// Database
if dbMode == "managed" {
cfg.Database.Mode = "managed"
cfg.Database.PostgresDSN = postgresDSN
cfg.Database.PostgresDSN = postgresDSN
// Auto-generate encryption key for API keys in DB (if not already set).
if os.Getenv("GOCLAW_ENCRYPTION_KEY") == "" {
encKey := onboardGenerateToken(32)
os.Setenv("GOCLAW_ENCRYPTION_KEY", encKey)
fmt.Printf(" Generated encryption key for API keys (AES-256-GCM)\n")
// Auto-generate encryption key for API keys in DB (if not already set).
if os.Getenv("GOCLAW_ENCRYPTION_KEY") == "" {
encKey := onboardGenerateToken(32)
os.Setenv("GOCLAW_ENCRYPTION_KEY", encKey)
fmt.Printf(" Generated encryption key for API keys (AES-256-GCM)\n")
} else {
fmt.Println(" Using existing GOCLAW_ENCRYPTION_KEY from environment")
}
fmt.Print(" Testing Postgres connection... ")
if err := testPostgresConnection(postgresDSN); err != nil {
fmt.Println("FAILED")
fmt.Printf(" Error: %v\n", err)
fmt.Println(" Please check your DSN and try again: ./goclaw onboard")
return
}
fmt.Println("OK")
runMigrate, err := promptConfirm("Run database migration now?", true)
if err != nil {
fmt.Println("Cancelled.")
return
}
if runMigrate {
fmt.Println(" Running migration...")
m, err := newMigrator(postgresDSN)
if err != nil {
fmt.Printf(" Migration error: %v\n", err)
fmt.Println(" You can run it manually later: ./goclaw migrate up")
} else {
fmt.Println(" Using existing GOCLAW_ENCRYPTION_KEY from environment")
}
fmt.Print(" Testing Postgres connection... ")
if err := testPostgresConnection(postgresDSN); err != nil {
fmt.Println("FAILED")
fmt.Printf(" Error: %v\n", err)
fmt.Println()
fallbackStandalone, err := promptConfirm("Connection failed. Fall back to standalone mode?", false)
if err != nil {
fmt.Println("Cancelled.")
return
}
if fallbackStandalone {
cfg.Database.Mode = ""
cfg.Database.PostgresDSN = ""
fmt.Println(" Switched to standalone mode.")
if err := m.Up(); err != nil && err.Error() != "no change" {
fmt.Printf(" Migration error: %v\n", err)
fmt.Println(" You can run it manually later: ./goclaw migrate up")
} else {
fmt.Println(" Please check your DSN and try again: ./goclaw onboard")
return
v, _, _ := m.Version()
fmt.Printf(" Migration complete (version: %d)\n", v)
}
} else {
fmt.Println("OK")
m.Close()
}
if cfg.Database.Mode == "managed" {
runMigrate, err := promptConfirm("Run database migration now?", true)
if err != nil {
fmt.Println("Cancelled.")
return
}
if runMigrate {
fmt.Println(" Running migration...")
m, err := newMigrator(postgresDSN)
if err != nil {
fmt.Printf(" Migration error: %v\n", err)
fmt.Println(" You can run it manually later: ./goclaw migrate up")
} else {
if err := m.Up(); err != nil && err.Error() != "no change" {
fmt.Printf(" Migration error: %v\n", err)
fmt.Println(" You can run it manually later: ./goclaw migrate up")
} else {
v, _, _ := m.Version()
fmt.Printf(" Migration complete (version: %d)\n", v)
}
m.Close()
}
fmt.Println(" Seeding default agent and provider...")
if err := seedManagedData(postgresDSN, cfg); err != nil {
fmt.Printf(" Seed warning: %v\n", err)
fmt.Println(" You can seed manually via the API after starting the gateway.")
} else {
fmt.Println(" Default agent and provider seeded.")
}
}
fmt.Println(" Seeding default agent and provider...")
if err := seedManagedData(postgresDSN, cfg); err != nil {
fmt.Printf(" Seed warning: %v\n", err)
fmt.Println(" You can seed manually via the API after starting the gateway.")
} else {
fmt.Println(" Default agent and provider seeded.")
}
}
@@ -638,11 +604,7 @@ func runOnboard() {
fmt.Printf(" Gateway: ws://%s:%d\n", cfg.Gateway.Host, cfg.Gateway.Port)
fmt.Printf(" Token: %s\n", cfg.Gateway.Token)
fmt.Printf(" Workspace: %s\n", cfg.Agents.Defaults.Workspace)
if cfg.Database.Mode == "managed" {
fmt.Println(" Database: managed (Postgres)")
} else {
fmt.Println(" Database: standalone (file-based)")
}
fmt.Println(" Database: PostgreSQL")
if cfg.Channels.Telegram.Enabled {
fmt.Println(" Telegram: enabled")
}
cmd/onboard_auto.go
+5 -40
View File
@@ -88,12 +88,8 @@ func runAutoOnboard(cfgPath string) bool {
slog.Info("auto-onboard: generated gateway token")
}
// 3. Managed mode: Postgres setup
// Auto-detect: if GOCLAW_POSTGRES_DSN is set, assume managed mode even without GOCLAW_MODE
if cfg.Database.PostgresDSN != "" && cfg.Database.Mode == "" {
cfg.Database.Mode = "managed"
}
if cfg.Database.Mode == "managed" && cfg.Database.PostgresDSN != "" {
// 3. Database: Postgres setup
if cfg.Database.PostgresDSN != "" {
fmt.Print(" Testing Postgres connection...")
// Retry loop: database container may still be starting
@@ -219,38 +215,12 @@ func detectProvider(cfg *config.Config) (string, string) {
// saveCleanConfig saves a minimal config.json without noise (empty providers,
// disabled channels, stripped secrets). Only includes sections relevant to
// the active configuration so the file serves as clean documentation.
// In managed mode, channels are stored in the DB (channel_instances table),
// Channels are stored in the DB (channel_instances table),
// so they are omitted from config.json to avoid dual-connection.
func saveCleanConfig(cfgPath string, cfg *config.Config) error {
isManaged := cfg.Database.Mode == "managed"
// Build channels map — only include enabled channels.
// In managed mode, skip channels entirely (they're DB instances now).
// Channels are stored in the DB (channel_instances table),
// so they are omitted from config.json to avoid dual-connection.
channels := make(map[string]interface{})
if !isManaged {
if cfg.Channels.Telegram.Enabled {
channels["telegram"] = map[string]interface{}{
"enabled": true,
"reaction_level": nonEmpty(cfg.Channels.Telegram.ReactionLevel, "full"),
"history_limit": nonZero(cfg.Channels.Telegram.HistoryLimit, 50),
}
}
if cfg.Channels.Discord.Enabled {
channels["discord"] = map[string]interface{}{"enabled": true}
}
if cfg.Channels.Slack.Enabled {
channels["slack"] = map[string]interface{}{"enabled": true}
}
if cfg.Channels.Feishu.Enabled {
channels["feishu"] = map[string]interface{}{"enabled": true}
}
if cfg.Channels.Zalo.Enabled {
channels["zalo"] = map[string]interface{}{"enabled": true}
}
if cfg.Channels.WhatsApp.Enabled {
channels["whatsapp"] = map[string]interface{}{"enabled": true}
}
}
// Build tools section.
tools := map[string]interface{}{
@@ -322,11 +292,6 @@ func saveCleanConfig(cfgPath string, cfg *config.Config) error {
root["channels"] = channels
}
if cfg.Database.Mode != "" {
root["database"] = map[string]interface{}{
"mode": cfg.Database.Mode,
}
}
data, err := json.MarshalIndent(root, "", " ")
if err != nil {
cmd/onboard_helpers.go
+2 -2
View File
@@ -59,12 +59,12 @@ func onboardWriteEnvFile(path string, cfg *config.Config, primaryKey, primaryEnv
lines = append(lines, fmt.Sprintf("export GOCLAW_FEISHU_APP_SECRET=%s", cfg.Channels.Feishu.AppSecret))
}
// Database (managed mode)
// Database
if cfg.Database.PostgresDSN != "" {
lines = append(lines, fmt.Sprintf("export GOCLAW_POSTGRES_DSN=%s", cfg.Database.PostgresDSN))
}
// Encryption key for API keys in DB (managed mode)
// Encryption key for API keys in DB
if encKey := os.Getenv("GOCLAW_ENCRYPTION_KEY"); encKey != "" {
lines = append(lines, fmt.Sprintf("export GOCLAW_ENCRYPTION_KEY=%s", encKey))
}
cmd/onboard_managed.go
-1
View File
@@ -36,7 +36,6 @@ func testPostgresConnection(dsn string) error {
func seedManagedData(dsn string, cfg *config.Config) error {
storeCfg := store.StoreConfig{
PostgresDSN: dsn,
Mode: "managed",
EncryptionKey: os.Getenv("GOCLAW_ENCRYPTION_KEY"),
}
stores, err := pg.NewPGStores(storeCfg)
cmd/sessions_cmd.go
+7 -40
View File
@@ -9,10 +9,7 @@ import (
"github.com/spf13/cobra"
"github.com/nextlevelbuilder/goclaw/internal/config"
"github.com/nextlevelbuilder/goclaw/internal/sessions"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/file"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
@@ -34,13 +31,7 @@ func sessionsListCmd() *cobra.Command {
Use: "list",
Short: "List all sessions",
Run: func(cmd *cobra.Command, args []string) {
if isManagedMode() {
sessionsListRPC(agentFilter, jsonOutput)
return
}
mgr := loadSessionsStore()
infos := mgr.List(agentFilter)
printSessionInfos(infos, jsonOutput)
sessionsListRPC(agentFilter, jsonOutput)
},
}
cmd.Flags().BoolVar(&jsonOutput, "json", false, "output as JSON")
@@ -54,16 +45,7 @@ func sessionsDeleteCmd() *cobra.Command {
Short: "Delete a session",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
if isManagedMode() {
sessionsDeleteRPC(args[0])
return
}
mgr := loadSessionsStore()
if err := mgr.Delete(args[0]); err != nil {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
os.Exit(1)
}
fmt.Printf("Deleted session: %s\n", args[0])
sessionsDeleteRPC(args[0])
},
}
}
@@ -74,21 +56,15 @@ func sessionsResetCmd() *cobra.Command {
Short: "Clear session history (keep session)",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
if isManagedMode() {
sessionsResetRPC(args[0])
return
}
mgr := loadSessionsStore()
mgr.Reset(args[0])
fmt.Printf("Reset session: %s\n", args[0])
sessionsResetRPC(args[0])
},
}
}
// --- RPC implementations (managed mode) ---
// --- RPC implementations ---
func sessionsListRPC(agentFilter string, jsonOutput bool) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]string{"agentId": agentFilter})
resp, err := gatewayRPC(protocol.MethodSessionsList, params)
@@ -115,7 +91,7 @@ func sessionsListRPC(agentFilter string, jsonOutput bool) {
}
func sessionsDeleteRPC(key string) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]string{"key": key})
resp, err := gatewayRPC(protocol.MethodSessionsDelete, params)
@@ -131,7 +107,7 @@ func sessionsDeleteRPC(key string) {
}
func sessionsResetRPC(key string) {
requireGatewayForManaged()
requireGateway()
params, _ := json.Marshal(map[string]string{"key": key})
resp, err := gatewayRPC(protocol.MethodSessionsReset, params)
@@ -173,15 +149,6 @@ func printSessionInfos(infos []store.SessionInfo, jsonOutput bool) {
tw.Flush()
}
// --- File-based helpers (standalone mode) ---
func loadSessionsStore() store.SessionStore {
cfgPath := resolveConfigPath()
cfg, _ := config.Load(cfgPath)
storage := config.ExpandHome(cfg.Sessions.Storage)
return file.NewFileSessionStore(sessions.NewManager(storage))
}
func truncateStr(s string, max int) string {
if len(s) <= max {
return s
cmd/upgrade.go
+4 -5
View File
@@ -47,9 +47,8 @@ func runUpgradeStatus() error {
fmt.Printf(" App version: %s (protocol %d)\n", Version, protocol.ProtocolVersion)
if cfg.Database.Mode != "managed" || cfg.Database.PostgresDSN == "" {
fmt.Println(" Mode: standalone (no database)")
fmt.Println(" Status: N/A (no schema migrations needed)")
if cfg.Database.PostgresDSN == "" {
fmt.Println(" Database: NOT CONFIGURED (set GOCLAW_POSTGRES_DSN)")
return nil
}
@@ -107,8 +106,8 @@ func runUpgrade(dryRun bool) error {
return fmt.Errorf("load config: %w", err)
}
if cfg.Database.Mode != "managed" || cfg.Database.PostgresDSN == "" {
fmt.Println("Standalone mode — no database migrations needed.")
if cfg.Database.PostgresDSN == "" {
fmt.Println("Database not configured. Set GOCLAW_POSTGRES_DSN to enable migrations.")
return nil
}
docker-compose.browser.yml
+2 -2
View File
@@ -1,7 +1,7 @@
# Browser sidecar overlay — adds headless Chrome for the browser automation tool.
#
# Usage (with managed mode):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.browser.yml up -d --build
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.browser.yml up -d --build
#
# The Chrome sidecar exposes CDP (Chrome DevTools Protocol) on port 9222.
# GoClaw connects to it automatically via GOCLAW_BROWSER_REMOTE_URL.
docker-compose.otel.yml
+2 -2
View File
@@ -1,7 +1,7 @@
# OTel overlay — rebuilds with OTel support + adds Jaeger for trace visualization.
#
# Usage (with managed mode):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.otel.yml up
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.otel.yml up
#
# Jaeger UI: http://localhost:16686
docker-compose.managed.yml → docker-compose.postgres.yml
+3 -4
View File
@@ -1,7 +1,7 @@
# Managed overlay — PostgreSQL (pgvector) for multi-tenant storage.
# PostgreSQL overlay — pgvector for multi-tenant storage.
#
# Recommended usage (with Web Dashboard):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.selfservice.yml up -d --build
# Usage (with Web Dashboard):
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.selfservice.yml up -d --build
#
# Required env vars (set in .env or shell):
# GOCLAW_OPENROUTER_API_KEY (or another provider key)
@@ -30,7 +30,6 @@ services:
postgres:
condition: service_healthy
environment:
- GOCLAW_MODE=managed
- GOCLAW_POSTGRES_DSN=postgres://${POSTGRES_USER:-goclaw}:${POSTGRES_PASSWORD:-goclaw}@postgres:5432/${POSTGRES_DB:-goclaw}?sslmode=disable
volumes:
- goclaw-skills:/app/skills
docker-compose.sandbox.yml
+2 -2
View File
@@ -4,8 +4,8 @@
# 1. Build the sandbox image: docker build -t goclaw-sandbox:bookworm-slim -f Dockerfile.sandbox .
# 2. Ensure Docker socket is accessible
#
# Usage (with managed mode):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.sandbox.yml up
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.sandbox.yml up
#
# SECURITY NOTE: Mounting Docker socket gives the container control over host Docker.
# Only use in trusted environments where agent code execution isolation is required.
docker-compose.selfservice.yml
+1 -1
View File
@@ -1,7 +1,7 @@
# Self-service overlay — adds web dashboard UI (nginx + React SPA).
#
# Recommended usage:
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.selfservice.yml up -d --build
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.selfservice.yml up -d --build
#
# Dashboard: http://localhost:3000
docker-compose.standalone.yml
-19
View File
@@ -1,19 +0,0 @@
# Standalone overlay — file-based storage with persistent volumes.
#
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.standalone.yml up
services:
goclaw:
volumes:
- ./config.json:/app/config.json:ro
- goclaw-workspace:/app/workspace
- goclaw-data:/app/data
- goclaw-sessions:/app/sessions
- goclaw-skills:/app/skills
volumes:
goclaw-workspace:
goclaw-data:
goclaw-sessions:
goclaw-skills:
docker-compose.tailscale.yml
+2 -2
View File
@@ -1,7 +1,7 @@
# Tailscale overlay — rebuilds with tsnet support for secure remote access.
#
# Usage (with managed mode):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.tailscale.yml up
# Usage:
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.tailscale.yml up
#
# Required:
# GOCLAW_TSNET_AUTH_KEY — Tailscale auth key (from https://login.tailscale.com/admin/settings/keys)
docker-compose.upgrade.yml
+3 -4
View File
@@ -2,13 +2,13 @@
#
# Usage:
# # Preview changes (dry-run):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.upgrade.yml run --rm upgrade --dry-run
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.upgrade.yml run --rm upgrade --dry-run
#
# # Apply upgrade:
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.upgrade.yml run --rm upgrade
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.upgrade.yml run --rm upgrade
#
# # Check status:
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.upgrade.yml run --rm upgrade --status
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.upgrade.yml run --rm upgrade --status
#
# The upgrade service runs goclaw upgrade and exits. Use --rm to auto-remove the container.
@@ -23,7 +23,6 @@ services:
- path: .env
required: false
environment:
- GOCLAW_MODE=managed
- GOCLAW_POSTGRES_DSN=postgres://${POSTGRES_USER:-goclaw}:${POSTGRES_PASSWORD:-goclaw}@postgres:5432/${POSTGRES_DB:-goclaw}?sslmode=disable
- GOCLAW_CONFIG=/app/data/config.json
- GOCLAW_MIGRATIONS_DIR=/app/migrations
docker-compose.yml
+6 -34
View File
@@ -1,12 +1,11 @@
# Base docker-compose — shared service definition.
# Combine with an overlay for your deployment mode:
# Combine with overlays for your deployment:
#
# Recommended (Managed + Web Dashboard):
# docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.selfservice.yml up -d --build
# Recommended (+ Web Dashboard):
# docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.selfservice.yml up -d --build
#
# Standalone: docker compose -f docker-compose.yml -f docker-compose.standalone.yml up -d --build
# Managed only: docker compose -f docker-compose.yml -f docker-compose.managed.yml up -d --build
# Managed+OTel: docker compose -f docker-compose.yml -f docker-compose.managed.yml -f docker-compose.otel.yml up -d --build
# Without dashboard: docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d --build
# With OTel tracing: docker compose -f docker-compose.yml -f docker-compose.postgres.yml -f docker-compose.otel.yml up -d --build
services:
goclaw:
@@ -21,43 +20,16 @@ services:
- path: .env
required: false
environment:
# Core
- GOCLAW_HOST=0.0.0.0
- GOCLAW_PORT=18790
- GOCLAW_CONFIG=/app/data/config.json
- GOCLAW_GATEWAY_TOKEN=${GOCLAW_GATEWAY_TOKEN:-}
- GOCLAW_ENCRYPTION_KEY=${GOCLAW_ENCRYPTION_KEY:-}
- GOCLAW_OWNER_IDS=${GOCLAW_OWNER_IDS:-}
# Provider (at least one required)
- GOCLAW_PROVIDER=${GOCLAW_PROVIDER:-}
- GOCLAW_MODEL=${GOCLAW_MODEL:-}
- GOCLAW_OPENROUTER_API_KEY=${GOCLAW_OPENROUTER_API_KEY:-}
- GOCLAW_ANTHROPIC_API_KEY=${GOCLAW_ANTHROPIC_API_KEY:-}
- GOCLAW_ANTHROPIC_BASE_URL=${GOCLAW_ANTHROPIC_BASE_URL:-}
- GOCLAW_OPENAI_API_KEY=${GOCLAW_OPENAI_API_KEY:-}
- GOCLAW_OPENAI_BASE_URL=${GOCLAW_OPENAI_BASE_URL:-}
- GOCLAW_MINIMAX_API_KEY=${GOCLAW_MINIMAX_API_KEY:-}
- GOCLAW_GROQ_API_KEY=${GOCLAW_GROQ_API_KEY:-}
- GOCLAW_DEEPSEEK_API_KEY=${GOCLAW_DEEPSEEK_API_KEY:-}
- GOCLAW_GEMINI_API_KEY=${GOCLAW_GEMINI_API_KEY:-}
- GOCLAW_MISTRAL_API_KEY=${GOCLAW_MISTRAL_API_KEY:-}
- GOCLAW_XAI_API_KEY=${GOCLAW_XAI_API_KEY:-}
- GOCLAW_COHERE_API_KEY=${GOCLAW_COHERE_API_KEY:-}
- GOCLAW_PERPLEXITY_API_KEY=${GOCLAW_PERPLEXITY_API_KEY:-}
# Channels
- GOCLAW_TELEGRAM_TOKEN=${GOCLAW_TELEGRAM_TOKEN:-}
- GOCLAW_DISCORD_TOKEN=${GOCLAW_DISCORD_TOKEN:-}
- GOCLAW_LARK_APP_ID=${GOCLAW_LARK_APP_ID:-}
- GOCLAW_LARK_APP_SECRET=${GOCLAW_LARK_APP_SECRET:-}
- GOCLAW_LARK_ENCRYPT_KEY=${GOCLAW_LARK_ENCRYPT_KEY:-}
- GOCLAW_LARK_VERIFICATION_TOKEN=${GOCLAW_LARK_VERIFICATION_TOKEN:-}
- GOCLAW_ZALO_TOKEN=${GOCLAW_ZALO_TOKEN:-}
- GOCLAW_WHATSAPP_BRIDGE_URL=${GOCLAW_WHATSAPP_BRIDGE_URL:-}
# Debug
- GOCLAW_TRACE_VERBOSE=${GOCLAW_TRACE_VERBOSE:-0}
volumes:
- goclaw-data:/app/data
- goclaw-workspace:/app/workspace # standalone mode workspace
- goclaw-workspace:/app/workspace
security_opt:
- no-new-privileges:true
cap_drop:
docs/00-architecture-overview.md
+23 -69
View File
@@ -2,12 +2,7 @@
## 1. Overview
GoClaw is an AI agent gateway written in Go. It exposes a WebSocket RPC (v3) interface and an OpenAI-compatible HTTP API for orchestrating LLM-powered agents. The system supports two operating modes:
- **Standalone** -- file-based storage with SQLite for per-user data, zero external dependencies beyond an LLM API key.
- **Managed** -- PostgreSQL-backed multi-tenant mode with HTTP CRUD APIs, per-user context files, encrypted credentials, agent delegation, teams, and LLM call tracing.
> **Documentation scope**: This documentation covers both modes. Standalone mode now has near-parity with managed mode for core features (per-user context files, workspace isolation, agent types, bootstrap onboarding). Managed mode adds agent delegation, teams, quality gates, tracing, HTTP CRUD APIs, and encrypted secrets.
GoClaw is an AI agent gateway written in Go. It exposes a WebSocket RPC (v3) interface and an OpenAI-compatible HTTP API for orchestrating LLM-powered agents. The system uses PostgreSQL as its storage backend with full multi-tenant isolation, per-user context files, encrypted credentials, agent delegation, teams, and LLM call tracing.
## 2. Component Diagram
@@ -121,17 +116,16 @@ flowchart TD
| `internal/hooks/` | Hook engine: quality gates, command evaluator, agent evaluator, recursion prevention (`WithSkipHooks`) |
| `internal/store/` | Store interfaces: SessionStore, AgentStore, ProviderStore, SkillStore, MemoryStore, CronStore, PairingStore, TracingStore, MCPServerStore, AgentLinkStore, TeamStore, ChannelInstanceStore, ConfigSecretsStore |
| `internal/store/pg/` | PostgreSQL implementations (`database/sql` + `pgx/v5`) |
| `internal/store/file/` | File-based implementations: sessions, memory (SQLite), cron, pairing, skills, agents (filesystem + SQLite) |
| `internal/bootstrap/` | System prompt files (AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, HEARTBEAT.md, BOOTSTRAP.md) + seeding + truncation |
| `internal/bootstrap/` | System prompt files (AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, BOOTSTRAP.md) + seeding + truncation |
| `internal/config/` | Config loading (JSON5) + env var overlay |
| `internal/skills/` | SKILL.md loader (5-tier hierarchy) + BM25 search + hot-reload via fsnotify |
| `internal/channels/` | Channel manager + adapters: Telegram (forum topics, STT, bot commands), Feishu/Lark (streaming cards, media), Zalo OA, Zalo Personal, Discord, WhatsApp |
| `internal/mcp/` | MCP server bridge (stdio, SSE, streamable-HTTP transports) |
| `internal/scheduler/` | Lane-based concurrency control (main, subagent, cron, delegate lanes) with per-session serialization |
| `internal/memory/` | Memory system (SQLite FTS5 + embeddings for standalone mode) |
| `internal/memory/` | Memory system (pgvector hybrid search) |
| `internal/permissions/` | RBAC policy engine (admin, operator, viewer roles) |
| `internal/pairing/` | DM/device pairing service (8-character codes) |
| `internal/sessions/` | File-based session manager (standalone mode) |
| `internal/sessions/` | Session manager |
| `internal/bus/` | Event pub/sub (Message Bus) |
| `internal/sandbox/` | Docker-based code execution sandbox |
| `internal/tts/` | Text-to-Speech providers: OpenAI, ElevenLabs, Edge, MiniMax |
@@ -139,37 +133,10 @@ flowchart TD
| `internal/crypto/` | AES-256-GCM encryption for API keys |
| `internal/tracing/` | LLM call tracing (traces + spans), in-memory buffer with periodic store flush |
| `internal/tracing/otelexport/` | Optional OpenTelemetry OTLP exporter (opt-in via build tags; adds gRPC + protobuf) |
| `internal/heartbeat/` | Periodic agent wake-up service |
---
## 4. Two Operating Modes
| Aspect | Standalone | Managed |
|--------|-----------|---------|
| Config source | `config.json` + env vars | `config.json` + `GOCLAW_POSTGRES_DSN` |
| Storage | JSON files + SQLite (`~/.goclaw/data/agents.db`) | PostgreSQL |
| Agents | Defined in `config.json` `agents.list`, created eagerly at startup | `agents` table, lazy-resolved via `ManagedResolver` |
| Agent store | `FileAgentStore` (filesystem + SQLite) | `PGAgentStore` |
| Context files | Agent-level on filesystem, per-user in SQLite | `agent_context_files` + `user_context_files` tables |
| Agent types | `open` / `predefined` (via config) | `open` (7 per-user files) / `predefined` (agent-level + USER.md per-user) |
| Per-user isolation | Workspace subdirectories (`user_alice/`, `user_bob/`) | Same + DB-scoped context files |
| Bootstrap onboarding | Per-user BOOTSTRAP.md seeding (SQLite) | Same (PostgreSQL) |
| Agent delegation | N/A | Sync/async delegation, agent links, quality gates |
| Agent teams | N/A | Shared task board, mailbox, handoff |
| Skills | Filesystem only (workspace + global dirs) | PostgreSQL + filesystem + embedding search |
| Memory | SQLite FTS5 + embeddings | pgvector hybrid (full-text search + vector similarity) |
| Tracing | N/A | `traces` + `spans` tables + optional OTel OTLP export |
| MCP servers | `config.json` `tools.mcp_servers` | `mcp_servers` table + grants |
| API key storage | `.env.local` / env vars only | PostgreSQL (AES-256-GCM encrypted) |
| HTTP CRUD API | N/A | `/v1/agents`, `/v1/skills`, `/v1/traces`, `/v1/mcp`, `/v1/delegations` |
| Virtual FS | `ContextFileInterceptor` routes to SQLite | `ContextFileInterceptor` routes to PostgreSQL |
| Custom tools | N/A | `custom_tools` table + `DynamicToolLoader` |
| Managed-only stores (nil in standalone) | -- | ProviderStore, TracingStore, MCPServerStore, CustomToolStore, AgentLinkStore, TeamStore |
---
## 5. Multi-Tenant Identity Model
## 4. Multi-Tenant Identity Model
GoClaw uses the **Identity Propagation** pattern (also known as **Trusted Subsystem**). It does not implement authentication or authorization — instead, it trusts the upstream service that authenticates with the gateway token to provide accurate user identity.
@@ -196,8 +163,8 @@ flowchart LR
| Entry Point | How user_id is provided | Enforcement |
|-------------|------------------------|-------------|
| HTTP API | `X-GoClaw-User-Id` header | Required in managed mode |
| WebSocket | `user_id` field in `connect` handshake | Required in managed mode |
| HTTP API | `X-GoClaw-User-Id` header | Required |
| WebSocket | `user_id` field in `connect` handshake | Required |
| Channels | Derived from platform sender ID (e.g., Telegram user ID) | Automatic |
### Compound User ID Convention
@@ -239,36 +206,27 @@ sequenceDiagram
GW->>GW: 2. Resolve workspace + data dirs
GW->>GW: 3. Create Message Bus
alt Managed mode
GW->>PG: 4. Connect to Postgres (pg.NewPGStores)
PG-->>GW: PG stores created
GW->>GW: 5. Start tracing collector
GW->>PG: 6. Register providers from DB
GW->>PG: 7. Wire embedding provider to PGMemoryStore
GW->>PG: 8. Backfill memory embeddings (background)
else Standalone mode
GW->>GW: 4. Create file-based stores
end
GW->>PG: 4. Connect to Postgres (pg.NewPGStores)
PG-->>GW: PG stores created
GW->>GW: 5. Start tracing collector
GW->>PG: 6. Register providers from DB
GW->>PG: 7. Wire embedding provider to PGMemoryStore
GW->>PG: 8. Backfill memory embeddings (background)
GW->>GW: 9. Register config-based providers
GW->>GW: 10. Create tool registry (filesystem, exec, web, memory, browser, TTS, subagent, MCP)
GW->>GW: 11. Load bootstrap files (DB or filesystem)
GW->>GW: 11. Load bootstrap files (DB)
GW->>GW: 12. Create skills loader + register skill_search tool
GW->>GW: 13. Wire skill embeddings (managed only)
GW->>GW: 13. Wire skill embeddings
alt Managed mode
GW->>GW: 14. Create agents lazily (set ManagedResolver)
GW->>GW: 15. wireManagedExtras (interceptors, cache subscribers)
GW->>GW: 16. Wire managed HTTP handlers (agents, skills, traces, MCP)
else Standalone mode
GW->>GW: 14. Create agents eagerly from config
GW->>GW: 15. wireStandaloneExtras (FileAgentStore, interceptors, callbacks)
end
GW->>GW: 14. Create agents lazily (set ManagedResolver)
GW->>GW: 15. wireManagedExtras (interceptors, cache subscribers)
GW->>GW: 16. Wire managed HTTP handlers (agents, skills, traces, MCP)
GW->>Engine: 17. Create gateway server (WS + HTTP)
GW->>Engine: 18. Register RPC methods
GW->>Engine: 19. Register + start channels (Telegram, Discord, Feishu, Zalo, WhatsApp)
GW->>Engine: 20. Start cron, heartbeat, scheduler (4 lanes)
GW->>Engine: 20. Start cron, scheduler (4 lanes)
GW->>Engine: 21. Start skills watcher + inbound consumer
GW->>Engine: 22. Listen on host:port
```
@@ -294,8 +252,6 @@ flowchart TD
W11["11. Evaluate Loop + Handoff<br/>evaluate_loop tool + handoff tool"]
```
A separate `wireStandaloneExtras()` in `cmd/gateway_standalone.go` wires the same core callbacks (user seeding, context file loading) using `FileAgentStore` instead of PostgreSQL.
### Cache Invalidation Events
| Event | Subscriber | Action |
@@ -386,8 +342,7 @@ When the process receives SIGINT or SIGTERM:
1. Broadcast `shutdown` event to all connected WebSocket clients.
2. `channelMgr.StopAll()` -- stop all channel adapters.
3. `cronStore.Stop()` -- stop cron scheduler.
4. `heartbeatSvc.Stop()` -- stop heartbeat service.
5. `sandboxMgr.Stop()` + `ReleaseAll()` -- release Docker containers.
4. `sandboxMgr.Stop()` + `ReleaseAll()` -- release Docker containers.
6. `cancel()` -- cancel root context, propagating to consumer + scheduler.
7. Deferred cleanup: flush tracing collector, close memory store, close browser manager, stop scheduler lanes.
8. HTTP server shutdown with a **5-second timeout** (`context.WithTimeout`).
@@ -420,7 +375,7 @@ flowchart TD
| `agents` | defaults (provider, model, context_window) + list (per-agent overrides) |
| `tools` | profile, allow/deny lists, exec_approval, web, browser, mcp_servers, rate_limit_per_hour |
| `channels` | Per-channel: enabled, token, dm_policy, group_policy, allow_from |
| `database` | mode (standalone/managed); postgres_dsn read only from env var |
| `database` | postgres_dsn read only from env var |
### Secret Handling
@@ -439,8 +394,7 @@ flowchart TD
| `cmd/root.go` | Cobra CLI entry point, flag parsing |
| `cmd/gateway.go` | Gateway startup orchestrator (`runGateway()`) |
| `cmd/gateway_managed.go` | Managed mode wiring (`wireManagedExtras()`, `wireManagedHTTP()`) |
| `cmd/gateway_standalone.go` | Standalone mode wiring (`wireStandaloneExtras()`) |
| `cmd/gateway_callbacks.go` | Shared callbacks for managed + standalone (user seeding, context file loading) |
| `cmd/gateway_callbacks.go` | Shared callbacks (user seeding, context file loading) |
| `cmd/gateway_consumer.go` | Inbound message consumer (subagent, delegate, teammate, handoff routing) |
| `cmd/gateway_providers.go` | Provider registration (config-based + DB-based) |
| `cmd/gateway_methods.go` | RPC method registration |
@@ -472,7 +426,7 @@ flowchart TD
| [05-channels-messaging.md](./05-channels-messaging.md) | Channel adapters, Telegram formatting, pairing, managed-mode user scoping |
| [06-store-data-model.md](./06-store-data-model.md) | Store interfaces, PostgreSQL schema, session caching, custom tool store |
| [07-bootstrap-skills-memory.md](./07-bootstrap-skills-memory.md) | Bootstrap files, skills system, memory, skills grants |
| [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md) | Scheduler lanes, cron lifecycle, heartbeat |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | Scheduler lanes, cron lifecycle |
| [09-security.md](./09-security.md) | Defense layers, encryption, rate limiting, RBAC, sandbox |
| [10-tracing-observability.md](./10-tracing-observability.md) | Tracing collector, span hierarchy, OTel export, trace API |
| [11-agent-teams.md](./11-agent-teams.md) | Agent teams, task board, mailbox, delegation integration |
docs/01-agent-loop.md
+1 -2
View File
@@ -171,8 +171,7 @@ The system prompt is assembled dynamically from 15+ sections. Two modes control
13. **Extra context** -- additional prompt text wrapped in `<extra_context>` XML tags.
14. **Project Context** -- context files loaded from the database or filesystem, wrapped in `<context_file>` XML tags with a defensive preamble.
15. **Silent Replies** -- instructions for the NO_REPLY convention.
16. **Heartbeats** -- instructions for periodic wake-up behavior.
17. **Sub-Agent Spawning** -- rules for launching child agents.
16. **Sub-Agent Spawning** -- rules for launching child agents.
18. **Delegation** -- auto-generated `DELEGATION.md` listing available delegation targets (inline if ≤15, search instruction if >15).
19. **Team** -- `TEAM.md` injected for team leads only (team name, role, teammate list).
20. **Runtime** -- runtime metadata (agent ID, session key, provider info).
docs/03-tools-system.md
-1
View File
@@ -162,7 +162,6 @@ flowchart TD
| `IDENTITY.md` | Agent identity information |
| `AGENTS.md` | Sub-agent definitions |
| `TOOLS.md` | Tool usage guidance |
| `HEARTBEAT.md` | Periodic wake-up instructions |
| `USER.md` | Per-user preferences and context |
| `BOOTSTRAP.md` | First-run instructions (write empty = delete row) |
docs/04-gateway-protocol.md
-1
View File
@@ -292,7 +292,6 @@ flowchart TD
| Method | Description |
|--------|-------------|
| `logs.tail` | Tail gateway logs |
| `heartbeat` | Trigger heartbeat check |
---
docs/05-channels-messaging.md
+5 -5
View File
@@ -182,7 +182,7 @@ The Telegram channel uses long polling via the `telego` library (Telegram Bot AP
- **Group mention gating**: By default, bot must be @mentioned in groups (`requireMention: true`). Pending messages without a mention are stored in a history buffer (default 50 messages) and included as context when the bot is eventually mentioned.
- **Typing indicator**: A "typing" action is sent while the agent is processing.
- **Proxy support**: Optional HTTP proxy configured via channel config.
- **Cancel commands**: `/stop` and `/stopall` intercepted before the 800ms debouncer. See [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md).
- **Cancel commands**: `/stop` and `/stopall` intercepted before the 800ms debouncer. See [08-scheduling-cron.md](./08-scheduling-cron.md) for details.
- **Concurrent group support**: Group sessions support up to 3 concurrent agent runs.
- **Bot reply as implicit mention**: Replying to a bot message in a group counts as mentioning the bot.
@@ -478,13 +478,13 @@ All channel state — placeholders, streams, reactions, typing controllers, thre
---
## 13. Managed Mode Behavior
## 13. Per-User Isolation
In managed mode, channels provide per-user isolation through compound sender IDs and context propagation:
Channels provide per-user isolation through compound sender IDs and context propagation:
- **User scoping**: Each channel constructs a compound sender ID (e.g., `telegram:123456`) which maps to a `user_id`. The session key format `agent:{agentId}:{channel}:direct:{peerId}` ensures each user has isolated conversation history per agent.
- **Context propagation**: `HandleMessage()` injects `AgentID`, `UserID`, and `AgentType` into the context. These flow to the ContextFileInterceptor, MemoryInterceptor, and per-user file seeding.
- **Pairing storage**: PostgreSQL (`pairing_requests` and `paired_devices` tables) in managed mode; JSON files in standalone mode.
- **Pairing storage**: PostgreSQL (`pairing_requests` and `paired_devices` tables).
- **Session persistence**: PostgreSQL `sessions` table with write-behind caching.
---
@@ -554,6 +554,6 @@ flowchart TD
|----------|-----------------|
| [00-architecture-overview.md](./00-architecture-overview.md) | Channel startup in gateway sequence |
| [03-tools-system.md](./03-tools-system.md) | Tool policy engine, per-request tool allow list |
| [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md) | /stop and /stopall commands, scheduler lanes |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | /stop and /stopall commands, scheduler lanes, cron |
| [09-security.md](./09-security.md) | Group file writer restrictions, security logging |
| [11-agent-teams.md](./11-agent-teams.md) | Team message routing, delegation result delivery |
docs/06-store-data-model.md
+30 -55
View File
@@ -1,58 +1,40 @@
# 06 - Store Layer and Data Model
The store layer abstracts all persistence behind Go interfaces, allowing the same core engine to run with file-based storage (standalone mode) or PostgreSQL (managed mode). Each store interface has independent implementations, and the system determines which backend to use based on configuration at startup.
The store layer abstracts all persistence behind Go interfaces backed by PostgreSQL. Each store interface has a PostgreSQL implementation wired at startup.
---
## 1. Store Layer Routing
## 1. Store Layer
```mermaid
flowchart TD
START["Gateway Startup"] --> CHECK{"StoreConfig.IsManaged()?<br/>(DSN + mode = managed)"}
CHECK -->|Yes| PG["PostgreSQL Backend"]
CHECK -->|No| FILE["File Backend"]
START["Gateway Startup"] --> PG["PostgreSQL Backend"]
PG --> PG_STORES["PGSessionStore<br/>PGAgentStore<br/>PGProviderStore<br/>PGCronStore<br/>PGPairingStore<br/>PGSkillStore<br/>PGMemoryStore<br/>PGTracingStore<br/>PGMCPServerStore<br/>PGCustomToolStore<br/>PGChannelInstanceStore<br/>PGConfigSecretsStore<br/>PGAgentLinkStore<br/>PGTeamStore"]
FILE --> FILE_STORES["FileSessionStore<br/>FileMemoryStore (SQLite + FTS5)<br/>FileCronStore<br/>FilePairingStore<br/>FileSkillStore<br/>FileAgentStore (filesystem + SQLite)<br/>ProviderStore = nil<br/>TracingStore = nil<br/>MCPServerStore = nil<br/>CustomToolStore = nil<br/>AgentLinks = nil<br/>Teams = nil"]
```
---
## 2. Store Interface Map
The `Stores` struct is the top-level container holding all storage backends. In standalone mode, managed-only stores are `nil`.
The `Stores` struct is the top-level container holding all PostgreSQL-backed storage implementations.
| Interface | Standalone Implementation | Managed Implementation | Mode |
|-----------|--------------------------|------------------------|------|
| SessionStore | `FileSessionStore` via `sessions.Manager` | `PGSessionStore` | Both |
| MemoryStore | `FileMemoryStore` (SQLite + FTS5 + embeddings) | `PGMemoryStore` (tsvector + pgvector) | Both |
| CronStore | `FileCronStore` | `PGCronStore` | Both |
| PairingStore | `FilePairingStore` via `pairing.Service` | `PGPairingStore` | Both |
| SkillStore | `FileSkillStore` via `skills.Loader` | `PGSkillStore` | Both |
| AgentStore | `FileAgentStore` (filesystem + SQLite) | `PGAgentStore` | Both |
| ProviderStore | `nil` | `PGProviderStore` | Managed only |
| TracingStore | `nil` | `PGTracingStore` | Managed only |
| MCPServerStore | `nil` | `PGMCPServerStore` | Managed only |
| CustomToolStore | `nil` | `PGCustomToolStore` | Managed only |
| ChannelInstanceStore | `nil` | `PGChannelInstanceStore` | Managed only |
| ConfigSecretsStore | `nil` | `PGConfigSecretsStore` | Managed only |
| AgentLinkStore | `nil` | `PGAgentLinkStore` | Managed only |
| TeamStore | `nil` | `PGTeamStore` | Managed only |
### Standalone AgentStore (FileAgentStore)
In standalone mode, `FileAgentStore` provides per-user context files and profiles without PostgreSQL. It combines filesystem storage (agent-level files like SOUL.md) with SQLite (`~/.goclaw/data/agents.db`) for per-user data:
| Data | Storage |
|------|---------|
| Agent metadata | In-memory from `config.json` |
| Agent-level files (SOUL.md, IDENTITY.md, ...) | Filesystem at workspace root |
| Per-user files (USER.md, BOOTSTRAP.md) | SQLite `user_context_files` |
| User profiles | SQLite `user_profiles` |
| Group file writers | SQLite `group_file_writers` |
Agent UUIDs use UUID v5 (deterministic): `uuid.NewSHA1(namespace, "goclaw-standalone:{agentKey}")` -- stable across restarts without database sequences.
| Interface | Implementation |
|-----------|---------------|
| SessionStore | `PGSessionStore` |
| MemoryStore | `PGMemoryStore` (tsvector + pgvector) |
| CronStore | `PGCronStore` |
| PairingStore | `PGPairingStore` |
| SkillStore | `PGSkillStore` |
| AgentStore | `PGAgentStore` |
| ProviderStore | `PGProviderStore` |
| TracingStore | `PGTracingStore` |
| MCPServerStore | `PGMCPServerStore` |
| CustomToolStore | `PGCustomToolStore` |
| ChannelInstanceStore | `PGChannelInstanceStore` |
| ConfigSecretsStore | `PGConfigSecretsStore` |
| AgentLinkStore | `PGAgentLinkStore` |
| TeamStore | `PGTeamStore` |
---
@@ -70,7 +52,7 @@ flowchart TD
CACHE --> GETSM["GetSummary()"]
end
CACHE -->|"Save(key)"| DB[("PostgreSQL / JSON file")]
CACHE -->|"Save(key)"| DB[("PostgreSQL")]
DB -->|"Cache miss via GetOrCreate"| CACHE
```
@@ -91,12 +73,6 @@ flowchart TD
| Cron | `agent:{agentId}:cron:{jobId}:run:{runId}` | `agent:default:cron:reminder:run:abc123` |
| Main | `agent:{agentId}:{mainKey}` | `agent:default:main` |
### File-Based Persistence (Standalone)
- Startup: `loadAll()` reads all `.json` files into memory
- Save: temp file + rename (atomic write, prevents corruption on crash)
- Filename: session key with `:` replaced by `_`, plus `.json` extension
---
## 4. Agent Access Control
@@ -182,14 +158,14 @@ flowchart TD
When FTS returns no results (e.g., cross-language queries), a `likeSearch()` fallback runs ILIKE queries using up to 5 keywords (minimum 3 characters each), scoped to the agent's index.
### Standalone vs Managed
### Search Implementation
| Aspect | Standalone | Managed |
|--------|-----------|---------|
| FTS engine | SQLite FTS5 | PostgreSQL tsvector |
| Vector | Embedding cache | pgvector extension |
| Search function | `plainto_tsquery('simple', ...)` | Same |
| Distance operator | N/A | `<=>` (cosine) |
| Aspect | Detail |
|--------|--------|
| FTS engine | PostgreSQL tsvector |
| Vector | pgvector extension |
| Search function | `plainto_tsquery('simple', ...)` |
| Distance operator | `<=>` (cosine) |
---
@@ -208,8 +184,8 @@ Context files are stored in two tables and routed based on agent type.
| Agent Type | Agent-Level Files | Per-User Files |
|------------|-------------------|----------------|
| `open` | Template fallback only | All 7 files (SOUL, IDENTITY, AGENTS, TOOLS, HEARTBEAT, BOOTSTRAP, USER) |
| `predefined` | 6 files (SOUL, IDENTITY, AGENTS, TOOLS, HEARTBEAT, BOOTSTRAP) | Only USER.md |
| `open` | Template fallback only | All files (SOUL, IDENTITY, AGENTS, TOOLS, BOOTSTRAP, USER) |
| `predefined` | Agent-level files (SOUL, IDENTITY, AGENTS, TOOLS, BOOTSTRAP) | Only USER.md |
The `ContextFileInterceptor` checks agent type from context and routes read/write operations accordingly. For open agents, per-user files take priority with agent-level as fallback.
@@ -577,7 +553,6 @@ All "create or update" operations use `INSERT ... ON CONFLICT DO UPDATE`, ensuri
| `internal/store/pairing_store.go` | `PairingStore` interface |
| `internal/store/cron_store.go` | `CronStore` interface |
| `internal/store/custom_tool_store.go` | `CustomToolStore` interface |
| `internal/store/file/agents.go` | `FileAgentStore`: filesystem + SQLite backend for standalone mode |
| `internal/store/pg/factory.go` | PG store factory: creates all PG store instances from a connection pool |
| `internal/store/pg/sessions.go` | `PGSessionStore`: session cache, Save, GetOrCreate |
| `internal/store/pg/agents.go` | `PGAgentStore`: CRUD, soft delete, access control |
docs/07-bootstrap-skills-memory.md
+22 -50
View File
@@ -22,8 +22,7 @@ Markdown files loaded at agent initialization and embedded into the system promp
| 3 | TOOLS.md | Local tool notes (camera, SSH, TTS, etc.) | Yes | Yes |
| 4 | IDENTITY.md | Agent name, creature, vibe, emoji | Yes | No |
| 5 | USER.md | User profile (name, timezone, preferences) | Yes | No |
| 6 | HEARTBEAT.md | Periodic check task list | Yes | No |
| 7 | BOOTSTRAP.md | First-run ritual (deleted after completion) | Yes | No |
| 6 | BOOTSTRAP.md | First-run ritual (deleted after completion) | Yes | No |
Subagent and cron sessions load only AGENTS.md + TOOLS.md (the `minimalAllowlist`).
@@ -60,28 +59,11 @@ When a file is truncated, a marker is inserted between the head and tail section
## 3. Seeding -- Template Creation
Templates are embedded in the binary via Go `embed` (directory: `internal/bootstrap/templates/`). Seeding automatically creates default files for new workspaces or new users.
Templates are embedded in the binary via Go `embed` (directory: `internal/bootstrap/templates/`). Seeding automatically creates default files for new users.
```mermaid
flowchart TD
subgraph "Standalone Mode"
SA["EnsureWorkspaceFiles()"] --> SA1["Iterate over embedded templates"]
SA1 --> SA2{"File already exists?<br/>(O_EXCL atomic check)"}
SA2 -->|Yes| SKIP1["Skip"]
SA2 -->|No| CREATE1["Create template file on disk"]
end
subgraph "Standalone Mode -- Per-User (FileAgentStore)"
SU["SeedUserFiles()"] --> SU1{"Agent type?"}
SU1 -->|open| SU_OPEN["Seed all 7 files to SQLite"]
SU1 -->|predefined| SU_PRED["Seed USER.md + BOOTSTRAP.md to SQLite"]
SU_OPEN --> SU_CHK{"Row already exists?"}
SU_PRED --> SU_CHK
SU_CHK -->|Yes| SKIP_SU["Skip"]
SU_CHK -->|No| SU_WRITE["INSERT into user_context_files"]
end
subgraph "Managed Mode -- Agent Level"
subgraph "Agent Level"
SB["SeedToStore()"] --> SB1{"Agent type = open?"}
SB1 -->|Yes| SKIP_AGENT["Skip (open agents use per-user only)"]
SB1 -->|No| SB2["Seed 6 files to agent_context_files<br/>(all except BOOTSTRAP.md)"]
@@ -90,7 +72,7 @@ flowchart TD
SB3 -->|No| WRITE2["Write embedded template"]
end
subgraph "Managed Mode -- Per-User"
subgraph "Per-User"
MC["SeedUserFiles()"] --> MC1{"Agent type?"}
MC1 -->|open| OPEN["Seed all 7 files to user_context_files"]
MC1 -->|predefined| PRED["Seed USER.md + BOOTSTRAP.md to user_context_files"]
@@ -103,31 +85,26 @@ flowchart TD
`SeedUserFiles()` is idempotent -- safe to call multiple times without overwriting personalized content.
### Standalone UUID Generation
Standalone agents are defined in `config.json` without database-generated UUIDs. `FileAgentStore` uses UUID v5 (`uuid.NewSHA1(namespace, "goclaw-standalone:{agentKey}")`) to produce deterministic IDs from agent keys. This ensures SQLite rows for per-user files survive process restarts without coordination.
### Predefined Agent Bootstrap
Both standalone and managed mode now seed `BOOTSTRAP.md` for predefined agents (per-user). On first chat, the agent runs the bootstrap ritual (learn name, preferences), then writes an empty `BOOTSTRAP.md` which triggers deletion. The empty-write deletion is ordered *before* the predefined write-block in `ContextFileInterceptor` to prevent an infinite bootstrap loop.
`BOOTSTRAP.md` is seeded for predefined agents (per-user). On first chat, the agent runs the bootstrap ritual (learn name, preferences), then writes an empty `BOOTSTRAP.md` which triggers deletion. The empty-write deletion is ordered *before* the predefined write-block in `ContextFileInterceptor` to prevent an infinite bootstrap loop.
---
## 4. Agent Type Routing
Two agent types determine which context files live at the agent level versus the per-user level. Agent types are now available in both managed and standalone modes.
Two agent types determine which context files live at the agent level versus the per-user level.
| Agent Type | Agent-Level Files | Per-User Files |
|------------|-------------------|----------------|
| `open` | None | All 7 files (AGENTS, SOUL, TOOLS, IDENTITY, USER, HEARTBEAT, BOOTSTRAP) |
| `open` | None | All files (AGENTS, SOUL, TOOLS, IDENTITY, USER, BOOTSTRAP) |
| `predefined` | 6 files (shared across all users) | USER.md + BOOTSTRAP.md |
For `open` agents, each user gets their own full set of context files. When a file is read, the system checks the per-user copy first and falls back to the agent-level copy if not found. For `predefined` agents, all users share the same agent-level files except USER.md (personalized) and BOOTSTRAP.md (per-user first-run ritual, deleted after completion).
| Mode | Agent Type Source | Per-User Storage |
|------|------------------|-----------------|
| Managed | `agents` PostgreSQL table | `user_context_files` table |
| Standalone | `config.json` agent entries | SQLite via `FileAgentStore` |
| Source | Per-User Storage |
|--------|-----------------|
| `agents` PostgreSQL table | `user_context_files` table |
---
@@ -155,8 +132,7 @@ flowchart TD
S9 --> S10["10. Extra Context / Subagent Context"]
S10 --> S11["11. Project Context<br/>(bootstrap files + virtual files)"]
S11 --> S12["12. Silent Replies (full only)"]
S12 --> S13["13. Heartbeats (full only)"]
S13 --> S14["14. Sub-Agent Spawning (conditional)"]
S12 --> S14["14. Sub-Agent Spawning (conditional)"]
S14 --> S15["15. Runtime"]
```
@@ -178,7 +154,6 @@ flowchart TD
| 10. Extra Context | Conditional | Conditional |
| 11. Project Context | Yes | Yes |
| 12. Silent Replies | Yes | No |
| 13. Heartbeats | Yes | No |
| 14. Sub-Agent Spawning | Conditional | Conditional |
| 15. Runtime | Yes | Yes |
@@ -213,7 +188,7 @@ This ensures resolver-injected virtual files (`DELEGATION.md`, `TEAM.md`) surviv
## 7. Agent Summoning (Managed Mode)
Creating a predefined agent requires 5 context files (SOUL.md, IDENTITY.md, AGENTS.md, TOOLS.md, HEARTBEAT.md) with specific formatting conventions. Agent summoning generates all 5 files from a natural language description in a single LLM call.
Creating a predefined agent requires 4 context files (SOUL.md, IDENTITY.md, AGENTS.md, TOOLS.md) with specific formatting conventions. Agent summoning generates all 4 files from a natural language description in a single LLM call.
```mermaid
flowchart TD
@@ -432,8 +407,8 @@ Combines full-text search and vector search with weighted merging.
```mermaid
flowchart TD
Q["Search(query)"] --> FTS["FTS Search<br/>Standalone: SQLite FTS5 (BM25)<br/>Managed: tsvector + plainto_tsquery"]
Q --> VEC["Vector Search<br/>Standalone: cosine similarity<br/>Managed: pgvector (cosine distance)"]
Q["Search(query)"] --> FTS["FTS Search<br/>tsvector + plainto_tsquery"]
Q --> VEC["Vector Search<br/>pgvector (cosine distance)"]
FTS --> MERGE["hybridMerge()"]
VEC --> MERGE
MERGE --> NORM["Normalize FTS scores to 0..1<br/>Vector scores already in 0..1"]
@@ -442,15 +417,14 @@ flowchart TD
BOOST --> RESULT["Sorted + filtered results"]
```
### Standalone vs Managed Comparison
### Search Implementation
| Aspect | Standalone | Managed |
|--------|-----------|---------|
| Storage | SQLite + FTS5 | PostgreSQL + tsvector + pgvector |
| FTS | `porter unicode61` tokenizer | `plainto_tsquery('simple')` |
| Vector | JSON array embedding | pgvector type |
| Scope | Global (single agent) | Per-agent + per-user |
| File watcher | fsnotify (1500ms debounce) | Not needed (DB-backed) |
| Aspect | Detail |
|--------|--------|
| Storage | PostgreSQL + tsvector + pgvector |
| FTS | `plainto_tsquery('simple')` |
| Vector | pgvector type |
| Scope | Per-agent + per-user |
When both FTS and vector search return results, scores are merged using the weighted sum. When only one channel returns results, its scores are used directly (weights normalized to 1.0).
@@ -492,8 +466,7 @@ The flush is idempotent per compaction cycle -- it will not run again until the
|------|-------------|
| `internal/bootstrap/files.go` | Bootstrap file constants, loading, session filtering |
| `internal/bootstrap/truncate.go` | Truncation pipeline (head/tail split, budget clamping) |
| `internal/bootstrap/seed.go` | Standalone mode seeding (EnsureWorkspaceFiles) |
| `internal/bootstrap/seed_store.go` | Managed mode seeding (SeedToStore, SeedUserFiles) |
| `internal/bootstrap/seed_store.go` | Seeding: SeedToStore, SeedUserFiles |
| `internal/bootstrap/load_store.go` | Load context files from DB (LoadFromStore) |
| `internal/bootstrap/templates/*.md` | Embedded template files |
| `internal/agent/systemprompt.go` | System prompt builder (BuildSystemPrompt, 17+ sections) |
@@ -501,7 +474,6 @@ The flush is idempotent per compaction cycle -- it will not run again until the
| `internal/agent/resolver.go` | Agent resolution, DELEGATION.md + TEAM.md injection |
| `internal/agent/loop_history.go` | Context file merging (base + per-user, base-only preserved) |
| `internal/agent/memoryflush.go` | Memory flush logic (shouldRunMemoryFlush, runMemoryFlush) |
| `internal/store/file/agents.go` | FileAgentStore -- filesystem + SQLite backend for standalone |
| `internal/http/summoner.go` | Agent summoning -- LLM-powered context file generation |
| `internal/skills/loader.go` | Skill loader (5-tier hierarchy, BuildSummary, filtering) |
| `internal/skills/search.go` | BM25 search index (tokenization, IDF scoring) |
docs/08-scheduling-cron-heartbeat.md → docs/08-scheduling-cron.md
+4 -46
View File
@@ -1,14 +1,13 @@
# 08 - Scheduling, Cron & Heartbeat
# 08 - Scheduling & Cron
Concurrency control and periodic task execution. The scheduler provides lane-based isolation and per-session serialization. Cron and heartbeat extend the agent loop with time-triggered behavior.
Concurrency control and periodic task execution. The scheduler provides lane-based isolation and per-session serialization. Cron extends the agent loop with time-triggered behavior.
> **Managed mode**: Cron jobs and run logs are stored in the `cron_jobs` and `cron_run_logs` PostgreSQL tables. Cache invalidation propagates via the `cache:cron` event on the message bus. In standalone mode, cron state is persisted to JSON files.
> Cron jobs and run logs are stored in the `cron_jobs` and `cron_run_logs` PostgreSQL tables. Cache invalidation propagates via the `cache:cron` event on the message bus.
### Responsibilities
- Scheduler: lane-based concurrency control, per-session message queue serialization
- Cron: three schedule kinds (at/every/cron), run logging, retry with exponential backoff
- Heartbeat: periodic agent wake-up, HEARTBEAT_OK detection, dedup within 24h
---
@@ -165,45 +164,6 @@ Jobs can be `active` or `paused`. Paused jobs skip execution during the due chec
---
## 5. Heartbeat -- 5 Steps
Periodically wakes the agent to check on events (calendar, inbox, alerts) and surfaces anything that needs attention.
> **Standalone mode only in practice**: HEARTBEAT.md is seeded from bootstrap templates only in standalone mode. In managed mode, HEARTBEAT.md is excluded from seeding (replaced by cron jobs). The heartbeat service still runs in both modes, but without a HEARTBEAT.md file with content, Step 2 skips execution.
```mermaid
flowchart TD
TICK["tick() -- every interval (default 30 min)"] --> S1{"Step 1:<br/>Within Active Hours?"}
S1 -->|Outside hours| SKIP1["Skip"]
S1 -->|Within hours| S2{"Step 2:<br/>HEARTBEAT.md exists<br/>and has meaningful content?"}
S2 -->|No| SKIP2["Skip"]
S2 -->|Yes| S3["Step 3: runner()<br/>Run agent with heartbeat prompt"]
S3 --> S4{"Step 4:<br/>Reply contains HEARTBEAT_OK?"}
S4 -->|OK| LOG["Log debug, discard reply"]
S4 -->|Has content| S5{"Step 5:<br/>Dedup -- same content<br/>within 24h?"}
S5 -->|Duplicate| SKIP3["Skip"]
S5 -->|New| DELIVER["deliver() via resolveTarget()<br/>then msgBus.PublishOutbound()"]
```
### Heartbeat Configuration
| Parameter | Default | Description |
|-----------|---------|-------------|
| Interval (`every`) | 30 minutes | Time between heartbeat wakes (`"0m"` to disable) |
| ActiveHours | (none) | Time window restriction, supports wrap-around midnight |
| Target | `"last"` | `"last"` (last-used channel), `"none"`, or explicit channel name |
| To | (empty) | Explicit chat ID (overrides target resolver) |
| Prompt | (default) | Custom system prompt for heartbeat runs |
| Model | (agent default) | Override LLM model for heartbeat runs |
| Session | `agent:{id}:heartbeat:main` | Custom session key |
| AckMaxChars | 300 | Content alongside HEARTBEAT_OK up to this length is still treated as OK |
### HEARTBEAT_OK Detection
Recognizes multiple formatting variants: `HEARTBEAT_OK`, `**HEARTBEAT_OK**`, `` `HEARTBEAT_OK` ``, `<b>HEARTBEAT_OK</b>`. Content accompanying the token is treated as an acknowledgment (OK) if it does not exceed `AckMaxChars`.
---
## File Reference
| File | Description |
@@ -212,13 +172,11 @@ Recognizes multiple formatting variants: `HEARTBEAT_OK`, `**HEARTBEAT_OK**`, ``
| `internal/scheduler/queue.go` | SessionQueue, Scheduler, drop policies, debounce |
| `internal/cron/service.go` | Cron run loop, schedule parsing, job lifecycle |
| `internal/cron/retry.go` | Retry with exponential backoff + jitter |
| `internal/heartbeat/service.go` | Heartbeat loop, HEARTBEAT_OK detection, active hours |
| `internal/store/cron_store.go` | CronStore interface (jobs + run logs) |
| `internal/store/file/cron.go` | File-based cron implementation (standalone mode) |
| `internal/store/pg/cron.go` | PostgreSQL cron implementation |
| `internal/store/pg/cron_scheduler.go` | PG job cache, due-job detection, execution |
| `cmd/gateway_cron.go` | makeCronJobHandler (routes cron execution to scheduler) |
| `cmd/gateway_agents.go` | setupHeartbeat, agent initialization |
| `cmd/gateway_agents.go` | Agent initialization |
| `internal/gateway/methods/cron.go` | RPC method handlers (list, create, update, delete, toggle, run, runs) |
---
docs/09-security.md
+1 -1
View File
@@ -368,5 +368,5 @@ When concurrency limits are hit, the error message is written for LLM reasoning:
| [04-gateway-protocol.md](./04-gateway-protocol.md) | WebSocket auth, RBAC, rate limiting |
| [06-store-data-model.md](./06-store-data-model.md) | API key encryption, agent access control pipeline, agent_links table |
| [07-bootstrap-skills-memory.md](./07-bootstrap-skills-memory.md) | Context file merging, virtual files |
| [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md) | Scheduler lanes, cron lifecycle |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | Scheduler lanes, cron lifecycle, /stop and /stopall |
| [10-tracing-observability.md](./10-tracing-observability.md) | Tracing and OTel export |
docs/10-tracing-observability.md
+2 -2
View File
@@ -2,7 +2,7 @@
Records agent run activities asynchronously. Spans are buffered in memory and flushed to the TracingStore in batches, with optional export to external OpenTelemetry backends.
> **Managed mode only**: Tracing requires PostgreSQL. In standalone mode, `TracingStore` is nil and no traces are recorded. The `traces` and `spans` tables store all tracing data. Optional OTel export sends spans to external backends (Jaeger, Grafana Tempo, Datadog) in addition to PostgreSQL.
> Tracing uses PostgreSQL. The `traces` and `spans` tables store all tracing data. Optional OTel export sends spans to external backends (Jaeger, Grafana Tempo, Datadog) in addition to PostgreSQL.
---
@@ -163,5 +163,5 @@ Delegation history is automatically recorded by `DelegateManager.saveDelegationH
| [01-agent-loop.md](./01-agent-loop.md) | Span emission during agent execution, cancel handling |
| [03-tools-system.md](./03-tools-system.md) | Delegation system, delegation history via agent tool |
| [06-store-data-model.md](./06-store-data-model.md) | traces/spans tables schema, delegation_history table |
| [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md) | /stop and /stopall commands |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | Scheduler lanes, /stop and /stopall commands |
| [09-security.md](./09-security.md) | Rate limiting, RBAC access control |
docs/11-agent-teams.md
+1 -1
View File
@@ -435,5 +435,5 @@ Teams emit events for real-time UI updates and observability.
|----------|-----------------|
| [03-tools-system.md](./03-tools-system.md) | Delegation system, agent links, quality gates, evaluate loop |
| [06-store-data-model.md](./06-store-data-model.md) | Team tables schema, delegation_history, handoff_routes |
| [08-scheduling-cron-heartbeat.md](./08-scheduling-cron-heartbeat.md) | Delegate scheduler lane (concurrency 100) |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | Delegate scheduler lane (concurrency 100), cron |
| [09-security.md](./09-security.md) | Delegation security, hook recursion prevention |
internal/agent/loop.go
+5 -5
View File
@@ -48,7 +48,7 @@ func (l *Loop) Run(ctx context.Context, req RunRequest) (*RunResult, error) {
Payload: map[string]interface{}{"message": req.Message},
})
// Create trace (managed mode only)
// Create trace
var traceID uuid.UUID
isChildTrace := req.ParentTraceID != uuid.Nil && l.traceCollector != nil
@@ -163,7 +163,7 @@ func (l *Loop) runLoop(ctx context.Context, req RunRequest) (*RunResult, error)
l.emit(event)
}
// Inject agent UUID into context for tool routing (managed mode)
// Inject agent UUID into context for tool routing
if l.agentUUID != uuid.Nil {
ctx = store.WithAgentID(ctx, l.agentUUID)
}
@@ -171,7 +171,7 @@ func (l *Loop) runLoop(ctx context.Context, req RunRequest) (*RunResult, error)
if req.UserID != "" {
ctx = store.WithUserID(ctx, req.UserID)
}
// Inject agent type into context for interceptor routing (managed mode)
// Inject agent type into context for interceptor routing
if l.agentType != "" {
ctx = store.WithAgentType(ctx, l.agentType)
}
@@ -200,7 +200,7 @@ func (l *Loop) runLoop(ctx context.Context, req RunRequest) (*RunResult, error)
cachedWs, loaded := l.userWorkspaces.Load(req.UserID)
if !loaded {
// First request for this user: get/create profile → returns stored workspace.
// Also seeds per-user context files on first chat (managed mode).
// Also seeds per-user context files on first chat.
ws := l.workspace
if l.ensureUserFiles != nil {
var err error
@@ -258,7 +258,7 @@ func (l *Loop) runLoop(ctx context.Context, req RunRequest) (*RunResult, error)
}
}
// Inject agent key into context for tool-level resolution (managed mode: multiple agents share tool registry)
// Inject agent key into context for tool-level resolution (multiple agents share tool registry)
ctx = tools.WithToolAgentKey(ctx, l.id)
// Security: truncate oversized user messages gracefully (feed truncation notice into LLM)
internal/agent/loop_history.go
+2 -2
View File
@@ -30,7 +30,7 @@ func (l *Loop) buildMessages(ctx context.Context, history []providers.Message, s
_, hasSpawn := l.tools.Get("spawn")
_, hasSkillSearch := l.tools.Get("skill_search")
// Per-user workspace: show the user's subdirectory in the system prompt (managed mode).
// Per-user workspace: show the user's subdirectory in the system prompt.
// Uses cached workspace from user_agent_profiles (includes channel isolation).
promptWorkspace := l.workspace
if l.agentUUID != uuid.Nil && userID != "" && l.workspace != "" {
@@ -51,7 +51,7 @@ func (l *Loop) buildMessages(ctx context.Context, history []providers.Message, s
}
}
// Group writer restrictions: filter context files + inject prompt (managed mode only)
// Group writer restrictions: filter context files + inject prompt
if l.groupWriterCache != nil && strings.HasPrefix(userID, "group:") {
senderID := store.SenderIDFromContext(ctx)
writerPrompt, filtered := l.buildGroupWriterPrompt(ctx, userID, senderID, contextFiles)
internal/agent/loop_types.go
+15 -15
View File
@@ -22,11 +22,11 @@ import (
// Bootstrap typically completes in 2-3 conversation turns.
const bootstrapAutoCleanupTurns = 3
// EnsureUserFilesFunc seeds per-user context files on first chat (managed mode).
// EnsureUserFilesFunc seeds per-user context files on first chat.
// Returns the effective workspace path (from user_agent_profiles) for caching.
type EnsureUserFilesFunc func(ctx context.Context, agentID uuid.UUID, userID, agentType, workspace, channel string) (effectiveWorkspace string, err error)
// ContextFileLoaderFunc loads context files dynamically per-request (managed mode).
// ContextFileLoaderFunc loads context files dynamically per-request.
type ContextFileLoaderFunc func(ctx context.Context, agentID uuid.UUID, userID, agentType string) []bootstrap.ContextFile
// BootstrapCleanupFunc removes BOOTSTRAP.md after a successful first run.
@@ -37,8 +37,8 @@ type BootstrapCleanupFunc func(ctx context.Context, agentID uuid.UUID, userID st
// Think → Act → Observe cycle with tool execution.
type Loop struct {
id string
agentUUID uuid.UUID // set in managed mode for context propagation
agentType string // "open" or "predefined" (managed mode)
agentUUID uuid.UUID // set for context propagation
agentType string // "open" or "predefined"
provider providers.Provider
model string
contextWindow int
@@ -63,7 +63,7 @@ type Loop struct {
hasMemory bool
contextFiles []bootstrap.ContextFile
// Per-user file seeding + dynamic context loading (managed mode)
// Per-user file seeding + dynamic context loading
ensureUserFiles EnsureUserFilesFunc
contextFileLoader ContextFileLoaderFunc
bootstrapCleanup BootstrapCleanupFunc
@@ -83,7 +83,7 @@ type Loop struct {
// Event callback for broadcasting agent events (run.started, chunk, tool.call, etc.)
onEvent func(event AgentEvent)
// Tracing collector (nil in standalone mode)
// Tracing collector (nil if not configured)
traceCollector *tracing.Collector
// Security: input scanning and message size limit
@@ -91,16 +91,16 @@ type Loop struct {
injectionAction string // "log", "warn" (default), "block", "off"
maxMessageChars int // 0 = use default (32000)
// Global builtin tool settings (from builtin_tools table, managed mode)
// Global builtin tool settings (from builtin_tools table)
builtinToolSettings tools.BuiltinToolSettings
// Thinking level for extended thinking support
thinkingLevel string
// Group writer cache for system prompt injection (managed mode)
// Group writer cache for system prompt injection
groupWriterCache *store.GroupWriterCache
// Team store for cross-session pending task detection (managed mode)
// Team store for cross-session pending task detection
teamStore store.TeamStore
}
@@ -158,11 +158,11 @@ type LoopConfig struct {
SandboxContainerDir string // e.g. "/workspace"
SandboxWorkspaceAccess string // "none", "ro", "rw"
// Managed mode: agent UUID for context propagation to tools
// Agent UUID for context propagation to tools
AgentUUID uuid.UUID
AgentType string // "open" or "predefined" (managed mode)
AgentType string // "open" or "predefined"
// Per-user file seeding + dynamic context loading (managed mode)
// Per-user file seeding + dynamic context loading
EnsureUserFiles EnsureUserFilesFunc
ContextFileLoader ContextFileLoaderFunc
BootstrapCleanup BootstrapCleanupFunc
@@ -175,16 +175,16 @@ type LoopConfig struct {
InjectionAction string // "log", "warn" (default), "block", "off"
MaxMessageChars int // 0 = use default (32000)
// Global builtin tool settings (from builtin_tools table, managed mode)
// Global builtin tool settings (from builtin_tools table)
BuiltinToolSettings tools.BuiltinToolSettings
// Thinking level: "off", "low", "medium", "high" (from agent other_config)
ThinkingLevel string
// Group writer cache for system prompt injection (managed mode)
// Group writer cache for system prompt injection
GroupWriterCache *store.GroupWriterCache
// Team store for cross-session pending task detection (managed mode)
// Team store for cross-session pending task detection
TeamStore store.TeamStore
}
internal/agent/resolver.go
+16 -16
View File
@@ -20,7 +20,7 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/tracing"
)
// ResolverDeps holds shared dependencies for the managed-mode agent resolver.
// ResolverDeps holds shared dependencies for the agent resolver.
type ResolverDeps struct {
AgentStore store.AgentStore
ProviderReg *providers.Registry
@@ -33,7 +33,7 @@ type ResolverDeps struct {
OnEvent func(AgentEvent)
TraceCollector *tracing.Collector
// Per-user file seeding + dynamic context loading (managed mode)
// Per-user file seeding + dynamic context loading
EnsureUserFiles EnsureUserFilesFunc
ContextFileLoader ContextFileLoaderFunc
BootstrapCleanup BootstrapCleanupFunc
@@ -49,30 +49,30 @@ type ResolverDeps struct {
SandboxContainerDir string
SandboxWorkspaceAccess string
// Dynamic custom tools (managed mode)
DynamicLoader *tools.DynamicToolLoader // nil if not managed
// Dynamic custom tools
DynamicLoader *tools.DynamicToolLoader
// Inter-agent delegation (managed mode)
AgentLinkStore store.AgentLinkStore // nil if not managed or no links
// Inter-agent delegation
AgentLinkStore store.AgentLinkStore
// Agent teams (managed mode)
TeamStore store.TeamStore // nil if not managed or no teams
// Agent teams
TeamStore store.TeamStore
// Builtin tool settings (managed mode)
BuiltinToolStore store.BuiltinToolStore // nil if not managed
// Builtin tool settings
BuiltinToolStore store.BuiltinToolStore
// MCP server store (managed mode) — for per-agent MCP tool loading
MCPStore store.MCPServerStore // nil if not managed or no MCP
// MCP server store — for per-agent MCP tool loading
MCPStore store.MCPServerStore
// Skill access store (managed mode) — for per-agent skill visibility filtering
SkillAccessStore store.SkillAccessStore // nil if not managed
// Skill access store — for per-agent skill visibility filtering
SkillAccessStore store.SkillAccessStore
// Group file writer cache (managed mode)
// Group file writer cache
GroupWriterCache *store.GroupWriterCache
}
// NewManagedResolver creates a ResolverFunc that builds Loops from DB agent data.
// This is the core of managed mode: agents are defined in Postgres, not config.json.
// Agents are defined in Postgres, not config.json.
func NewManagedResolver(deps ResolverDeps) ResolverFunc {
return func(agentKey string) (Agent, error) {
ctx := context.Background()
internal/agent/router.go
+6 -6
View File
@@ -8,7 +8,7 @@ import (
)
// ResolverFunc is called when an agent isn't found in the cache.
// Used in managed mode to lazy-create agents from DB.
// Used to lazy-create agents from DB.
type ResolverFunc func(agentKey string) (Agent, error)
const defaultRouterTTL = 10 * time.Minute
@@ -21,12 +21,12 @@ type agentEntry struct {
// Router manages multiple agent Loop instances.
// Each agent has a unique ID and its own provider/model/tools config.
// In managed mode, cached Loops expire after TTL (safety net for multi-instance).
// Cached Loops expire after TTL (safety net for multi-instance).
type Router struct {
agents map[string]*agentEntry
mu sync.RWMutex
activeRuns sync.Map // runID → *ActiveRun
resolver ResolverFunc // optional: lazy creation from DB (managed mode)
resolver ResolverFunc // optional: lazy creation from DB
ttl time.Duration
}
@@ -37,7 +37,7 @@ func NewRouter() *Router {
}
}
// SetResolver sets a resolver function for lazy agent creation (managed mode).
// SetResolver sets a resolver function for lazy agent creation.
func (r *Router) SetResolver(fn ResolverFunc) {
r.mu.Lock()
defer r.mu.Unlock()
@@ -51,7 +51,7 @@ func (r *Router) Register(ag Agent) {
r.agents[ag.ID()] = &agentEntry{agent: ag, cachedAt: time.Now()}
}
// Get returns an agent by ID. In managed mode, lazy-creates from DB via resolver.
// Get returns an agent by ID. Lazy-creates from DB via resolver if needed.
// Cached entries expire after TTL as a safety net for multi-instance deployments.
func (r *Router) Get(agentID string) (Agent, error) {
r.mu.RLock()
@@ -70,7 +70,7 @@ func (r *Router) Get(agentID string) (Agent, error) {
r.mu.Unlock()
}
// Try resolver (managed mode: create from DB)
// Try resolver (create from DB)
if resolver != nil {
ag, err := resolver(agentID)
if err != nil {
internal/agent/systemprompt.go
+1 -6
View File
@@ -153,12 +153,7 @@ func BuildSystemPrompt(cfg SystemPromptConfig) string {
lines = append(lines, buildSilentRepliesSection()...)
}
// 13. ## Heartbeats (full only)
if !isMinimal {
lines = append(lines, buildHeartbeatsSection()...)
}
// 14. ## Sub-Agent Spawning
// 13. ## Sub-Agent Spawning
if cfg.HasSpawn {
lines = append(lines, buildSpawnSection()...)
}
internal/agent/systemprompt_sections.go
-13
View File
@@ -199,19 +199,6 @@ func buildSilentRepliesSection() []string {
}
}
func buildHeartbeatsSection() []string {
return []string{
"## Heartbeats",
"",
"If you receive a heartbeat poll and there is nothing that needs attention, reply exactly:",
"HEARTBEAT_OK",
"",
"GoClaw treats a leading/trailing \"HEARTBEAT_OK\" as a heartbeat ack (and may discard it).",
"If something needs attention, do NOT include \"HEARTBEAT_OK\"; reply with the alert text instead.",
"",
}
}
func buildSpawnSection() []string {
return []string{
"## Sub-Agent Spawning",
internal/bootstrap/files.go
-3
View File
@@ -8,7 +8,6 @@
// USER.md — user profile
// IDENTITY.md— agent name, emoji, creature, vibe
// TOOLS.md — local tool notes
// HEARTBEAT.md— periodic check tasks
// BOOTSTRAP.md— first-run ritual (deleted after completion)
// MEMORY.md — long-term curated memory
package bootstrap
@@ -27,7 +26,6 @@ const (
IdentityFile = "IDENTITY.md"
UserFile = "USER.md"
UserPredefinedFile = "USER_PREDEFINED.md"
HeartbeatFile = "HEARTBEAT.md"
BootstrapFile = "BOOTSTRAP.md"
DelegationFile = "DELEGATION.md"
TeamFile = "TEAM.md"
@@ -44,7 +42,6 @@ var standardFiles = []string{
ToolsFile,
IdentityFile,
UserFile,
HeartbeatFile,
BootstrapFile,
}
internal/bootstrap/seed.go
-1
View File
@@ -18,7 +18,6 @@ var templateFiles = []string{
ToolsFile,
IdentityFile,
UserFile,
HeartbeatFile,
}
// ReadTemplate returns the content of an embedded template file.
internal/bootstrap/seed_store.go
+3 -5
View File
@@ -41,10 +41,8 @@ func SeedToStore(ctx context.Context, agentStore store.AgentStore, agentID uuid.
if name == UserFile {
continue
}
// TOOLS.md and HEARTBEAT.md are only useful in standalone mode.
// TOOLS.md: local tool notes (camera, SSH, device names).
// HEARTBEAT.md: replaced by cron jobs in managed mode.
if name == ToolsFile || name == HeartbeatFile {
// TOOLS.md: local tool notes (camera, SSH, device names) — not applicable.
if name == ToolsFile {
continue
}
if hasContent[name] {
@@ -83,7 +81,7 @@ func SeedToStore(ctx context.Context, agentStore store.AgentStore, agentID uuid.
}
// userSeedFilesOpen is the full set of files seeded per-user for open agents.
// TOOLS.md and HEARTBEAT.md excluded — only useful in standalone mode.
// TOOLS.md excluded — not applicable.
var userSeedFilesOpen = []string{
AgentsFile,
SoulFile,
internal/bootstrap/templates/HEARTBEAT.md
-5
View File
@@ -1,5 +0,0 @@
# HEARTBEAT.md
# Keep this file empty (or with only comments) to skip heartbeat API calls.
# Add tasks below when you want the agent to check something periodically.
internal/cache/cache.go
Vendored
+25
View File
@@ -0,0 +1,25 @@
package cache
import (
"context"
"time"
)
// Cache is a generic key-value cache interface.
// Implementations: InMemoryCache (this package), future: Redis, Memcache.
type Cache[V any] interface {
// Get retrieves a value by key. Returns the value and true if found, zero value and false if not.
Get(ctx context.Context, key string) (V, bool)
// Set stores a value with an optional TTL. If ttl is 0, the entry never expires.
Set(ctx context.Context, key string, value V, ttl time.Duration)
// Delete removes a single key.
Delete(ctx context.Context, key string)
// DeleteByPrefix removes all keys matching the given prefix.
DeleteByPrefix(ctx context.Context, prefix string)
// Clear removes all entries.
Clear(ctx context.Context)
}
internal/cache/memory.go
Vendored
+71
View File
@@ -0,0 +1,71 @@
package cache
import (
"context"
"strings"
"sync"
"time"
)
// entry wraps a cached value with expiration metadata.
type entry[V any] struct {
value V
expiresAt time.Time // zero means no expiry
}
func (e entry[V]) expired() bool {
return !e.expiresAt.IsZero() && time.Now().After(e.expiresAt)
}
// InMemoryCache is a thread-safe in-memory Cache implementation with TTL support.
type InMemoryCache[V any] struct {
data sync.Map
}
// NewInMemoryCache creates a new in-memory cache.
func NewInMemoryCache[V any]() *InMemoryCache[V] {
return &InMemoryCache[V]{}
}
func (c *InMemoryCache[V]) Get(_ context.Context, key string) (V, bool) {
raw, ok := c.data.Load(key)
if !ok {
var zero V
return zero, false
}
e := raw.(entry[V])
if e.expired() {
c.data.Delete(key)
var zero V
return zero, false
}
return e.value, true
}
func (c *InMemoryCache[V]) Set(_ context.Context, key string, value V, ttl time.Duration) {
e := entry[V]{value: value}
if ttl > 0 {
e.expiresAt = time.Now().Add(ttl)
}
c.data.Store(key, e)
}
func (c *InMemoryCache[V]) Delete(_ context.Context, key string) {
c.data.Delete(key)
}
func (c *InMemoryCache[V]) DeleteByPrefix(_ context.Context, prefix string) {
c.data.Range(func(key, _ any) bool {
if k, ok := key.(string); ok && strings.HasPrefix(k, prefix) {
c.data.Delete(key)
}
return true
})
}
func (c *InMemoryCache[V]) Clear(_ context.Context) {
c.data.Range(func(key, _ any) bool {
c.data.Delete(key)
return true
})
}
internal/cache/memory_test.go
Vendored
+92
View File
@@ -0,0 +1,92 @@
package cache
import (
"context"
"testing"
"time"
)
func TestInMemoryCache_GetSet(t *testing.T) {
c := NewInMemoryCache[string]()
ctx := context.Background()
c.Set(ctx, "hello", "world", 0)
val, ok := c.Get(ctx, "hello")
if !ok {
t.Fatal("expected key to exist")
}
if val != "world" {
t.Fatalf("expected 'world', got %q", val)
}
}
func TestInMemoryCache_TTLExpiry(t *testing.T) {
c := NewInMemoryCache[int]()
ctx := context.Background()
c.Set(ctx, "key", 42, 20*time.Millisecond)
// Should be present immediately
val, ok := c.Get(ctx, "key")
if !ok || val != 42 {
t.Fatal("expected key to be present before TTL expiry")
}
time.Sleep(40 * time.Millisecond)
_, ok = c.Get(ctx, "key")
if ok {
t.Fatal("expected key to be expired after TTL")
}
}
func TestInMemoryCache_Delete(t *testing.T) {
c := NewInMemoryCache[string]()
ctx := context.Background()
c.Set(ctx, "k", "v", 0)
c.Delete(ctx, "k")
_, ok := c.Get(ctx, "k")
if ok {
t.Fatal("expected key to be deleted")
}
}
func TestInMemoryCache_DeleteByPrefix(t *testing.T) {
c := NewInMemoryCache[string]()
ctx := context.Background()
c.Set(ctx, "agent:1:files", "a", 0)
c.Set(ctx, "agent:1:meta", "b", 0)
c.Set(ctx, "agent:2:files", "c", 0)
c.DeleteByPrefix(ctx, "agent:1:")
if _, ok := c.Get(ctx, "agent:1:files"); ok {
t.Error("agent:1:files should be deleted")
}
if _, ok := c.Get(ctx, "agent:1:meta"); ok {
t.Error("agent:1:meta should be deleted")
}
if _, ok := c.Get(ctx, "agent:2:files"); !ok {
t.Error("agent:2:files should still exist")
}
}
func TestInMemoryCache_Clear(t *testing.T) {
c := NewInMemoryCache[int]()
ctx := context.Background()
c.Set(ctx, "a", 1, 0)
c.Set(ctx, "b", 2, 0)
c.Set(ctx, "c", 3, 0)
c.Clear(ctx)
for _, k := range []string{"a", "b", "c"} {
if _, ok := c.Get(ctx, k); ok {
t.Errorf("key %q should be cleared", k)
}
}
}
internal/channels/quota.go
+1 -1
View File
@@ -26,7 +26,7 @@ type quotaCounts struct {
// QuotaChecker enforces per-user/group request quotas by counting top-level
// traces in the database. Results are cached in-memory for cacheTTL.
// Managed mode only — nil-safe (standalone mode passes nil).
// Nil-safe (nil means quota not configured).
type QuotaChecker struct {
db *sql.DB
config config.QuotaConfig
internal/channels/telegram/channel.go
+2 -2
View File
@@ -24,8 +24,8 @@ type Channel struct {
bot *telego.Bot
config config.TelegramConfig
pairingService store.PairingStore
agentStore store.AgentStore // for group file writer management (nil in standalone)
teamStore store.TeamStore // for /tasks, /task_detail commands (nil in standalone)
agentStore store.AgentStore // for group file writer management (nil if not configured)
teamStore store.TeamStore // for /tasks, /task_detail commands (nil if not configured)
placeholders sync.Map // localKey string → messageID int
stopThinking sync.Map // localKey string → *thinkingCancel
typingCtrls sync.Map // localKey string → *typing.Controller
internal/channels/zalo/personal/auth.go
+3 -3
View File
@@ -17,7 +17,7 @@ import (
func (c *Channel) authenticate(ctx context.Context) (*protocol.Session, error) {
sess := protocol.NewSession()
// 1. Preloaded credentials (managed mode: from factory).
// 1. Preloaded credentials (from DB via factory).
if c.preloadedCreds != nil {
slog.Info("zalo_personal: attempting login with preloaded credentials")
if err := protocol.LoginWithCredentials(ctx, sess, *c.preloadedCreds); err != nil {
@@ -26,7 +26,7 @@ func (c *Channel) authenticate(ctx context.Context) (*protocol.Session, error) {
return sess, nil
}
// 2. Saved file credentials (standalone).
// 2. Saved file credentials (config-based).
credPath := c.resolveCredentialsPath()
if cred := loadCredentials(credPath); cred != nil {
slog.Info("zalo_personal: attempting login with saved credentials", "path", credPath)
@@ -56,7 +56,7 @@ func (c *Channel) authenticate(ctx context.Context) (*protocol.Session, error) {
return sess, nil
}
// SetPreloadedCredentials sets credentials from DB (managed mode factory).
// SetPreloadedCredentials sets credentials from DB factory.
func (c *Channel) SetPreloadedCredentials(cred *protocol.Credentials) {
c.preloadedCreds = cred
}
internal/channels/zalo/personal/channel.go
+1 -1
View File
@@ -42,7 +42,7 @@ type Channel struct {
sess *protocol.Session
listener *protocol.Listener
// Pre-loaded credentials (managed mode: from DB, standalone: from file or QR).
// Pre-loaded credentials (from DB or from file/QR as fallback).
preloadedCreds *protocol.Credentials
requireMention bool
internal/channels/zalo/personal/factory.go
+1 -1
View File
@@ -27,7 +27,7 @@ type zaloInstanceConfig struct {
AllowFrom []string `json:"allow_from,omitempty"`
}
// Factory creates a Zalo Personal channel from DB instance data (managed mode).
// Factory creates a Zalo Personal channel from DB instance data.
// Does NOT trigger QR login — credentials must be provided.
func Factory(name string, creds json.RawMessage, cfg json.RawMessage,
msgBus *bus.MessageBus, pairingSvc store.PairingStore) (channels.Channel, error) {
internal/config/config.go
+2 -30
View File
@@ -65,16 +65,10 @@ type TailscaleConfig struct {
EnableTLS bool `json:"enable_tls,omitempty"` // use ListenTLS for auto HTTPS certs
}
// DatabaseConfig configures Postgres for managed mode.
// DatabaseConfig configures the PostgreSQL connection.
// PostgresDSN is NEVER read from config.json (secret) — only from env GOCLAW_POSTGRES_DSN.
type DatabaseConfig struct {
PostgresDSN string `json:"-"` // from env GOCLAW_POSTGRES_DSN only
Mode string `json:"mode,omitempty"` // "standalone" (default) or "managed"
}
// IsManagedMode returns true if the gateway is running in managed (multi-tenant) mode.
func (c *Config) IsManagedMode() bool {
return c.Database.Mode == "managed" && c.Database.PostgresDSN != ""
PostgresDSN string `json:"-"` // from env GOCLAW_POSTGRES_DSN only
}
// SkillsConfig configures the skills storage system.
@@ -126,8 +120,6 @@ type AgentDefaults struct {
Memory *MemoryConfig `json:"memory,omitempty"`
Compaction *CompactionConfig `json:"compaction,omitempty"`
ContextPruning *ContextPruningConfig `json:"contextPruning,omitempty"`
Heartbeat *HeartbeatConfig `json:"heartbeat,omitempty"`
// Bootstrap context truncation limits (matching TS bootstrapMaxChars / bootstrapTotalMaxChars)
BootstrapMaxChars int `json:"bootstrapMaxChars,omitempty"` // per-file max before truncation (default 20000)
BootstrapTotalMaxChars int `json:"bootstrapTotalMaxChars,omitempty"` // total budget across all files (default 24000)
@@ -178,26 +170,6 @@ type ContextPruningHardClear struct {
Placeholder string `json:"placeholder,omitempty"` // replacement text (default "[Old tool result content cleared]")
}
// HeartbeatConfig configures periodic agent heartbeats.
// Matching TS agents.defaults.heartbeat.
type HeartbeatConfig struct {
Every string `json:"every,omitempty"` // duration string: "30m", "1h", "0m"=disabled (default "30m")
ActiveHours *ActiveHoursConfig `json:"activeHours,omitempty"` // restrict to time window
Model string `json:"model,omitempty"` // optional model override
Session string `json:"session,omitempty"` // "main" (default) or explicit session key
Target string `json:"target,omitempty"` // "last" (default), "none", or channel ID
To string `json:"to,omitempty"` // optional recipient override (chat ID)
Prompt string `json:"prompt,omitempty"` // custom heartbeat prompt
AckMaxChars int `json:"ackMaxChars,omitempty"` // max chars after HEARTBEAT_OK before dropping (default 300)
}
// ActiveHoursConfig restricts heartbeats to a time window.
type ActiveHoursConfig struct {
Start string `json:"start,omitempty"` // "HH:MM" inclusive
End string `json:"end,omitempty"` // "HH:MM" exclusive
Timezone string `json:"timezone,omitempty"` // IANA timezone (default: local)
}
// MemoryConfig configures the agent memory system (SQLite + FTS5 + optional embeddings).
// Matching TS agents.defaults.memory.
type MemoryConfig struct {
internal/config/config_channels.go
+3 -3
View File
@@ -112,7 +112,7 @@ type ZaloPersonalConfig struct {
DMPolicy string `json:"dm_policy,omitempty"` // "pairing" (default), "allowlist", "open", "disabled"
GroupPolicy string `json:"group_policy,omitempty"` // "open" (default), "allowlist", "disabled"
RequireMention *bool `json:"require_mention,omitempty"` // require @bot mention in groups (default true)
CredentialsPath string `json:"credentials_path,omitempty"` // path to saved cookies JSON (standalone)
CredentialsPath string `json:"credentials_path,omitempty"` // path to saved cookies JSON
}
type FeishuConfig struct {
@@ -189,7 +189,7 @@ type QuotaWindow struct {
// IsZero returns true if no limits are set.
func (w QuotaWindow) IsZero() bool { return w.Hour == 0 && w.Day == 0 && w.Week == 0 }
// QuotaConfig configures per-user/group request quotas (managed mode only).
// QuotaConfig configures per-user/group request quotas.
// Config merge priority: Groups > Channels > Providers > Default.
type QuotaConfig struct {
Enabled bool `json:"enabled"`
@@ -210,7 +210,7 @@ type GatewayConfig struct {
RateLimitRPM int `json:"rate_limit_rpm,omitempty"` // rate limit: requests per minute per user (default 20, 0 = disabled)
InjectionAction string `json:"injection_action,omitempty"` // prompt injection action: "log", "warn" (default), "block", "off"
InboundDebounceMs int `json:"inbound_debounce_ms,omitempty"` // merge rapid messages from same sender (default 1000ms, -1 = disabled)
Quota *QuotaConfig `json:"quota,omitempty"` // per-user/group request quotas (managed mode only)
Quota *QuotaConfig `json:"quota,omitempty"` // per-user/group request quotas
}
// ToolsConfig controls tool availability, policy, and web search.
internal/config/config_load.go
+7 -2
View File
@@ -4,6 +4,7 @@ import (
"crypto/sha256"
"encoding/json"
"fmt"
"log/slog"
"os"
"path/filepath"
"strconv"
@@ -160,7 +161,11 @@ func (c *Config) applyEnvOverrides() {
// Database
envStr("GOCLAW_POSTGRES_DSN", &c.Database.PostgresDSN)
envStr("GOCLAW_MODE", &c.Database.Mode)
// Deprecation warning for GOCLAW_MODE (removed — PostgreSQL is always active)
if v := os.Getenv("GOCLAW_MODE"); v != "" {
slog.Warn("GOCLAW_MODE is deprecated; managed mode is now the only mode", "value", v)
}
// Telemetry
envStr("GOCLAW_TELEMETRY_ENDPOINT", &c.Telemetry.Endpoint)
@@ -240,7 +245,7 @@ func (c *Config) applyEnvOverrides() {
// src/config/defaults.ts.
//
// Go port does not have OAuth vs API-key distinction — we always treat it as
// API-key mode (heartbeat 30m).
// API-key mode.
func (c *Config) applyContextPruningDefaults() {
// Only apply when Anthropic is configured.
if c.Providers.Anthropic.APIKey == "" {
internal/config/config_secrets.go
+3 -3
View File
@@ -110,8 +110,8 @@ func (c *Config) StripSecrets() {
}
// StripMaskedSecrets strips only fields that still contain the mask value "***".
// Real values (user-entered via UI) are preserved. Used in standalone mode
// so that secrets entered via the config UI persist in config.json.
// Real values (user-entered via UI) are preserved, so that secrets entered
// via the config UI persist in config.json.
func (c *Config) StripMaskedSecrets() {
stripIfMasked := func(s *string) {
if *s == secretMask {
@@ -181,7 +181,7 @@ func (c *Config) ApplyDBSecrets(secrets map[string]string) {
}
// ExtractDBSecrets returns the config_secrets key-value pairs from the config.
// Used by managed mode to save secrets to the config_secrets table.
// Saves secrets to the config_secrets table.
func (c *Config) ExtractDBSecrets() map[string]string {
secrets := make(map[string]string)
internal/gateway/log_tee.go
+121
View File
@@ -0,0 +1,121 @@
package gateway
import (
"context"
"log/slog"
"sync"
"time"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// LogTee is a slog.Handler that forwards log records to subscribed WS clients
// while delegating to an underlying handler for normal output.
type LogTee struct {
inner slog.Handler
mu sync.RWMutex
clients map[string]*Client // client ID → client
}
// NewLogTee wraps an existing slog.Handler so log records are also forwarded
// to any WebSocket clients that have started log tailing.
func NewLogTee(inner slog.Handler) *LogTee {
return &LogTee{
inner: inner,
clients: make(map[string]*Client),
}
}
func (t *LogTee) Enabled(ctx context.Context, level slog.Level) bool {
return t.inner.Enabled(ctx, level)
}
func (t *LogTee) Handle(ctx context.Context, r slog.Record) error {
// Forward to subscribers (non-blocking).
t.mu.RLock()
n := len(t.clients)
t.mu.RUnlock()
if n > 0 {
entry := map[string]interface{}{
"timestamp": r.Time.UnixMilli(),
"level": levelName(r.Level),
"message": r.Message,
}
// Collect attributes into source hint.
var src string
r.Attrs(func(a slog.Attr) bool {
if a.Key == "component" || a.Key == "source" || a.Key == "module" {
src = a.Value.String()
return false
}
return true
})
if src != "" {
entry["source"] = src
}
evt := protocol.NewEvent("log", entry)
t.mu.RLock()
for _, c := range t.clients {
c.SendEvent(*evt)
}
t.mu.RUnlock()
}
return t.inner.Handle(ctx, r)
}
func (t *LogTee) WithAttrs(attrs []slog.Attr) slog.Handler {
return &LogTee{
inner: t.inner.WithAttrs(attrs),
clients: t.clients,
mu: t.mu,
}
}
func (t *LogTee) WithGroup(name string) slog.Handler {
return &LogTee{
inner: t.inner.WithGroup(name),
clients: t.clients,
mu: t.mu,
}
}
// Subscribe adds a client to the log tailing set.
func (t *LogTee) Subscribe(client *Client) {
t.mu.Lock()
t.clients[client.ID()] = client
t.mu.Unlock()
// Send an initial log entry so the client knows tailing started.
client.SendEvent(*protocol.NewEvent("log", map[string]interface{}{
"timestamp": time.Now().UnixMilli(),
"level": "info",
"message": "Log tailing started",
"source": "gateway",
}))
}
// Unsubscribe removes a client from the log tailing set.
func (t *LogTee) Unsubscribe(clientID string) {
t.mu.Lock()
delete(t.clients, clientID)
t.mu.Unlock()
}
func levelName(l slog.Level) string {
switch {
case l >= slog.LevelError:
return "error"
case l >= slog.LevelWarn:
return "warn"
case l >= slog.LevelInfo:
return "info"
default:
return "debug"
}
}
internal/gateway/methods/agent_links.go
+4 -4
View File
@@ -41,7 +41,7 @@ type linksListParams struct {
func (m *AgentLinksMethods) handleList(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.linkStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not configured"))
return
}
@@ -106,7 +106,7 @@ type linksCreateParams struct {
func (m *AgentLinksMethods) handleCreate(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.linkStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not configured"))
return
}
@@ -219,7 +219,7 @@ type linksUpdateParams struct {
func (m *AgentLinksMethods) handleUpdate(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.linkStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not configured"))
return
}
@@ -313,7 +313,7 @@ type linksDeleteParams struct {
func (m *AgentLinksMethods) handleDelete(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.linkStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "agent links not configured"))
return
}
internal/gateway/methods/agents.go
+14 -16
View File
@@ -24,13 +24,12 @@ type AgentsMethods struct {
cfg *config.Config
cfgPath string
workspace string
agentStore store.AgentStore // nil in standalone mode
interceptor *tools.ContextFileInterceptor // nil in standalone mode; invalidated on file writes
isManaged bool
agentStore store.AgentStore
interceptor *tools.ContextFileInterceptor // invalidated on file writes
}
func NewAgentsMethods(agents *agent.Router, cfg *config.Config, cfgPath, workspace string, agentStore store.AgentStore, isManaged bool, interceptor *tools.ContextFileInterceptor) *AgentsMethods {
return &AgentsMethods{agents: agents, cfg: cfg, cfgPath: cfgPath, workspace: workspace, agentStore: agentStore, isManaged: isManaged, interceptor: interceptor}
func NewAgentsMethods(agents *agent.Router, cfg *config.Config, cfgPath, workspace string, agentStore store.AgentStore, interceptor *tools.ContextFileInterceptor) *AgentsMethods {
return &AgentsMethods{agents: agents, cfg: cfg, cfgPath: cfgPath, workspace: workspace, agentStore: agentStore, interceptor: interceptor}
}
// isOwnerUser checks if the given user ID is in the configured owner IDs.
@@ -107,8 +106,7 @@ func (m *AgentsMethods) handleAgentWait(_ context.Context, client *gateway.Clien
}
func (m *AgentsMethods) handleList(ctx context.Context, client *gateway.Client, req *protocol.RequestFrame) {
// In managed mode, query the store so ALL accessible agents appear (not just router-cached ones).
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
userID := client.UserID()
if userID == "" {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInvalidRequest, "user context required"))
@@ -149,7 +147,7 @@ func (m *AgentsMethods) handleList(ctx context.Context, client *gateway.Client,
return
}
// Standalone mode: return router-cached agents.
// Fallback: return router-cached agents.
infos := m.agents.ListInfo()
client.SendResponse(protocol.NewOKResponse(req.ID, map[string]interface{}{
"agents": infos,
@@ -167,7 +165,7 @@ func (m *AgentsMethods) handleCreate(_ context.Context, client *gateway.Client,
Avatar string `json:"avatar"`
AgentType string `json:"agent_type"` // "open" (default) or "predefined"
OwnerIDs []string `json:"owner_ids,omitempty"` // first entry used as DB owner_id; falls back to "system"
// Per-agent config overrides (managed mode only)
// Per-agent config overrides
ToolsConfig json.RawMessage `json:"tools_config,omitempty"`
SubagentsConfig json.RawMessage `json:"subagents_config,omitempty"`
SandboxConfig json.RawMessage `json:"sandbox_config,omitempty"`
@@ -204,7 +202,7 @@ func (m *AgentsMethods) handleCreate(_ context.Context, client *gateway.Client,
ws = config.ExpandHome(ws)
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: create agent in DB ---
ctx := context.Background()
@@ -259,7 +257,7 @@ func (m *AgentsMethods) handleCreate(_ context.Context, client *gateway.Client,
// Invalidate router cache so resolver re-loads from DB
m.agents.InvalidateAgent(agentID)
} else {
// --- Standalone mode: config.json + filesystem ---
// --- Fallback: config.json + filesystem ---
if _, ok := m.cfg.Agents.List[agentID]; ok {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInvalidRequest, "agent already exists: "+agentID))
return
@@ -314,7 +312,7 @@ func (m *AgentsMethods) handleUpdate(_ context.Context, client *gateway.Client,
Workspace string `json:"workspace"`
Model string `json:"model"`
Avatar string `json:"avatar"`
// Per-agent config overrides (managed mode only)
// Per-agent config overrides
ToolsConfig json.RawMessage `json:"tools_config,omitempty"`
SubagentsConfig json.RawMessage `json:"subagents_config,omitempty"`
SandboxConfig json.RawMessage `json:"sandbox_config,omitempty"`
@@ -332,7 +330,7 @@ func (m *AgentsMethods) handleUpdate(_ context.Context, client *gateway.Client,
return
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: update agent in DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -397,7 +395,7 @@ func (m *AgentsMethods) handleUpdate(_ context.Context, client *gateway.Client,
m.agents.InvalidateAgent(params.AgentID)
} else {
// --- Standalone mode: config.json ---
// --- Fallback: config.json ---
spec, ok := m.cfg.Agents.List[params.AgentID]
if !ok {
if params.AgentID != "default" {
@@ -473,7 +471,7 @@ func (m *AgentsMethods) handleDelete(_ context.Context, client *gateway.Client,
var removedBindings int
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: delete from DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -495,7 +493,7 @@ func (m *AgentsMethods) handleDelete(_ context.Context, client *gateway.Client,
os.RemoveAll(ag.Workspace)
}
} else {
// --- Standalone mode: config.json ---
// --- Fallback: config.json ---
spec, ok := m.cfg.Agents.List[params.AgentID]
if !ok {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrNotFound, "agent not found: "+params.AgentID))
internal/gateway/methods/agents_create_owner_test.go
+1 -2
View File
@@ -126,7 +126,7 @@ func nullClient() *gateway.Client {
return &gateway.Client{}
}
// newManagedMethods returns AgentsMethods wired for managed mode with the given stub store.
// newManagedMethods returns AgentsMethods wired with the given stub store.
func newManagedMethods(t *testing.T, stub store.AgentStore) *AgentsMethods {
t.Helper()
return &AgentsMethods{
@@ -134,7 +134,6 @@ func newManagedMethods(t *testing.T, stub store.AgentStore) *AgentsMethods {
cfg: minimalConfig(),
workspace: t.TempDir(),
agentStore: stub,
isManaged: true,
}
}
internal/gateway/methods/agents_files.go
+7 -7
View File
@@ -14,7 +14,7 @@ import (
)
// allowedAgentFiles is the list of files exposed via agents.files.* RPCs.
// TOOLS.md and HEARTBEAT.md excluded — only useful in standalone mode.
// TOOLS.md excluded — not applicable.
var allowedAgentFiles = []string{
bootstrap.AgentsFile, bootstrap.SoulFile, bootstrap.IdentityFile,
bootstrap.UserFile, bootstrap.UserPredefinedFile, bootstrap.BootstrapFile, bootstrap.MemoryJSONFile,
@@ -32,7 +32,7 @@ func (m *AgentsMethods) handleFilesList(_ context.Context, client *gateway.Clien
params.AgentID = "default"
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: list from DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -76,7 +76,7 @@ func (m *AgentsMethods) handleFilesList(_ context.Context, client *gateway.Clien
return
}
// --- Standalone mode: filesystem ---
// --- Fallback: filesystem ---
ws := m.resolveWorkspace(params.AgentID)
files := make([]map[string]interface{}, 0, len(allowedAgentFiles))
@@ -130,7 +130,7 @@ func (m *AgentsMethods) handleFilesGet(_ context.Context, client *gateway.Client
return
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: read from DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -171,7 +171,7 @@ func (m *AgentsMethods) handleFilesGet(_ context.Context, client *gateway.Client
return
}
// --- Standalone mode: filesystem ---
// --- Fallback: filesystem ---
ws := m.resolveWorkspace(params.AgentID)
p := filepath.Join(ws, params.Name)
@@ -228,7 +228,7 @@ func (m *AgentsMethods) handleFilesSet(_ context.Context, client *gateway.Client
return
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: write to DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -261,7 +261,7 @@ func (m *AgentsMethods) handleFilesSet(_ context.Context, client *gateway.Client
return
}
// --- Standalone mode: filesystem ---
// --- Fallback: filesystem ---
ws := m.resolveWorkspace(params.AgentID)
os.MkdirAll(ws, 0755)
p := filepath.Join(ws, params.Name)
internal/gateway/methods/agents_identity.go
+2 -2
View File
@@ -40,7 +40,7 @@ func (m *AgentsMethods) handleIdentityGet(_ context.Context, client *gateway.Cli
"agentId": params.AgentID,
}
if m.isManaged && m.agentStore != nil {
if m.agentStore != nil {
// --- Managed mode: read identity from DB ---
ctx := context.Background()
ag, err := m.agentStore.GetByKey(ctx, params.AgentID)
@@ -70,7 +70,7 @@ func (m *AgentsMethods) handleIdentityGet(_ context.Context, client *gateway.Cli
}
}
} else {
// --- Standalone mode: config + filesystem ---
// --- Fallback: config + filesystem ---
result["name"] = m.cfg.ResolveDisplayName(params.AgentID)
if spec, ok := m.cfg.Agents.List[params.AgentID]; ok && spec.Identity != nil {
internal/gateway/methods/channel_instances.go
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// ChannelInstancesMethods handles channel instance CRUD via WebSocket RPC (managed mode).
// ChannelInstancesMethods handles channel instance CRUD via WebSocket RPC.
type ChannelInstancesMethods struct {
store store.ChannelInstanceStore
msgBus *bus.MessageBus
internal/gateway/methods/chat.go
+3 -4
View File
@@ -18,12 +18,11 @@ import (
type ChatMethods struct {
agents *agent.Router
sessions store.SessionStore
isManaged bool
rateLimiter *gateway.RateLimiter
}
func NewChatMethods(agents *agent.Router, sess store.SessionStore, isManaged bool, rl *gateway.RateLimiter) *ChatMethods {
return &ChatMethods{agents: agents, sessions: sess, isManaged: isManaged, rateLimiter: rl}
func NewChatMethods(agents *agent.Router, sess store.SessionStore, rl *gateway.RateLimiter) *ChatMethods {
return &ChatMethods{agents: agents, sessions: sess, rateLimiter: rl}
}
// Register adds chat methods to the router.
@@ -71,7 +70,7 @@ func (m *ChatMethods) handleSend(ctx context.Context, client *gateway.Client, re
}
userID := client.UserID()
if m.isManaged && userID == "" {
if userID == "" {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInvalidRequest, "user_id is required in managed mode — provide it in the connect handshake"))
return
}
internal/gateway/methods/config.go
+13 -24
View File
@@ -19,13 +19,12 @@ import (
type ConfigMethods struct {
cfg *config.Config
cfgPath string
managedMode bool
secretsStore store.ConfigSecretsStore // nil in standalone mode
eventBus bus.EventPublisher // nil-safe; broadcasts config change events
secretsStore store.ConfigSecretsStore
eventBus bus.EventPublisher // nil-safe; broadcasts config change events
}
func NewConfigMethods(cfg *config.Config, cfgPath string, managedMode bool, secretsStore store.ConfigSecretsStore, eventBus bus.EventPublisher) *ConfigMethods {
return &ConfigMethods{cfg: cfg, cfgPath: cfgPath, managedMode: managedMode, secretsStore: secretsStore, eventBus: eventBus}
func NewConfigMethods(cfg *config.Config, cfgPath string, secretsStore store.ConfigSecretsStore, eventBus bus.EventPublisher) *ConfigMethods {
return &ConfigMethods{cfg: cfg, cfgPath: cfgPath, secretsStore: secretsStore, eventBus: eventBus}
}
func (m *ConfigMethods) Register(router *gateway.MethodRouter) {
@@ -72,15 +71,9 @@ func (m *ConfigMethods) handleApply(ctx context.Context, client *gateway.Client,
return
}
// Branch on mode for secrets handling
if m.managedMode {
// Managed mode: extract secrets → save to config_secrets table, strip all from file
m.saveSecretsToStore(ctx, newCfg)
newCfg.StripSecrets()
} else {
// Standalone mode: only strip masked values, keep real values
newCfg.StripMaskedSecrets()
}
// Extract secrets → save to config_secrets table, strip all from file
m.saveSecretsToStore(ctx, newCfg)
newCfg.StripSecrets()
// Save to disk
if err := config.Save(m.cfgPath, newCfg); err != nil {
@@ -90,7 +83,7 @@ func (m *ConfigMethods) handleApply(ctx context.Context, client *gateway.Client,
// Update in-memory config and restore secrets
m.cfg.ReplaceFrom(newCfg)
if m.managedMode && m.secretsStore != nil {
if m.secretsStore != nil {
if secrets, err := m.secretsStore.GetAll(ctx); err == nil {
m.cfg.ApplyDBSecrets(secrets)
}
@@ -149,13 +142,9 @@ func (m *ConfigMethods) handlePatch(ctx context.Context, client *gateway.Client,
return
}
// Branch on mode for secrets handling
if m.managedMode {
m.saveSecretsToStore(ctx, merged)
merged.StripSecrets()
} else {
merged.StripMaskedSecrets()
}
// Extract secrets → save to config_secrets table, strip all from file
m.saveSecretsToStore(ctx, merged)
merged.StripSecrets()
// Save to disk
if err := config.Save(m.cfgPath, merged); err != nil {
@@ -165,7 +154,7 @@ func (m *ConfigMethods) handlePatch(ctx context.Context, client *gateway.Client,
// Update in-memory config and restore secrets
m.cfg.ReplaceFrom(merged)
if m.managedMode && m.secretsStore != nil {
if m.secretsStore != nil {
if secrets, err := m.secretsStore.GetAll(ctx); err == nil {
m.cfg.ApplyDBSecrets(secrets)
}
@@ -228,7 +217,7 @@ func (m *ConfigMethods) handleSchema(_ context.Context, client *gateway.Client,
}
// saveSecretsToStore extracts non-LLM/non-channel secrets from the config
// and persists them to the config_secrets table (managed mode only).
// and persists them to the config_secrets table.
func (m *ConfigMethods) saveSecretsToStore(ctx context.Context, cfg *config.Config) {
if m.secretsStore == nil {
return
internal/gateway/methods/logs.go
+50
View File
@@ -0,0 +1,50 @@
package methods
import (
"context"
"encoding/json"
"github.com/nextlevelbuilder/goclaw/internal/gateway"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// LogsMethods handles logs.tail (start/stop live log tailing).
type LogsMethods struct {
logTee *gateway.LogTee
}
func NewLogsMethods(logTee *gateway.LogTee) *LogsMethods {
return &LogsMethods{logTee: logTee}
}
func (m *LogsMethods) Register(router *gateway.MethodRouter) {
router.Register(protocol.MethodLogsTail, m.handleTail)
}
func (m *LogsMethods) handleTail(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
var params struct {
Action string `json:"action"`
}
if req.Params != nil {
json.Unmarshal(req.Params, &params)
}
switch params.Action {
case "start":
m.logTee.Subscribe(client)
client.SendResponse(protocol.NewOKResponse(req.ID, map[string]interface{}{
"status": "tailing",
}))
case "stop":
m.logTee.Unsubscribe(client.ID())
client.SendResponse(protocol.NewOKResponse(req.ID, map[string]interface{}{
"status": "stopped",
}))
default:
client.SendResponse(protocol.NewErrorResponse(
req.ID,
protocol.ErrInvalidRequest,
"action must be 'start' or 'stop'",
))
}
}
internal/gateway/methods/quota_methods.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
)
// QuotaMethods handles quota.usage — returns per-user quota consumption for the dashboard.
// Nil-safe: returns {enabled: false} when quotaChecker is nil (standalone mode).
// Nil-safe: returns {enabled: false} when quotaChecker is nil (quota not configured).
// When checker is nil but db is available, still queries today's summary from traces.
type QuotaMethods struct {
checker *channels.QuotaChecker
internal/gateway/methods/skills.go
+1 -2
View File
@@ -12,7 +12,6 @@ import (
)
// SkillsMethods handles skills.list, skills.get, skills.update.
// Uses store.SkillStore interface — PGSkillStore in managed mode, FileSkillStore in standalone.
type SkillsMethods struct {
store store.SkillStore
}
@@ -99,7 +98,7 @@ func (m *SkillsMethods) handleUpdate(_ context.Context, client *gateway.Client,
// Check if the store supports updates (PGSkillStore does, FileSkillStore doesn't)
updater, ok := m.store.(skillUpdater)
if !ok {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrNotFound, "skills.update not supported in standalone mode"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrNotFound, "skills.update not supported for file-based skills"))
return
}
internal/gateway/methods/teams.go
+2 -2
View File
@@ -52,7 +52,7 @@ func (m *TeamsMethods) Register(router *gateway.MethodRouter) {
func (m *TeamsMethods) handleList(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -81,7 +81,7 @@ type teamsCreateParams struct {
func (m *TeamsMethods) handleCreate(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
internal/gateway/methods/teams_crud.go
+5 -5
View File
@@ -20,7 +20,7 @@ type teamsGetParams struct {
func (m *TeamsMethods) handleGet(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -68,7 +68,7 @@ type teamsDeleteParams struct {
func (m *TeamsMethods) handleDelete(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -129,7 +129,7 @@ type teamsTaskListParams struct {
func (m *TeamsMethods) handleTaskList(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -172,7 +172,7 @@ type teamsUpdateParams struct {
func (m *TeamsMethods) handleUpdate(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -253,7 +253,7 @@ type teamsKnownUsersParams struct {
func (m *TeamsMethods) handleKnownUsers(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
internal/gateway/methods/teams_members.go
+2 -2
View File
@@ -22,7 +22,7 @@ type teamsAddMemberParams struct {
func (m *TeamsMethods) handleAddMember(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
@@ -108,7 +108,7 @@ type teamsRemoveMemberParams struct {
func (m *TeamsMethods) handleRemoveMember(_ context.Context, client *gateway.Client, req *protocol.RequestFrame) {
if m.teamStore == nil {
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not available (standalone mode)"))
client.SendResponse(protocol.NewErrorResponse(req.ID, protocol.ErrInternal, "teams not configured"))
return
}
internal/gateway/router.go
+2 -5
View File
@@ -168,10 +168,7 @@ func (r *MethodRouter) handleHealth(ctx context.Context, client *Client, req *pr
s := r.server
uptimeMs := time.Since(s.startedAt).Milliseconds()
mode := "standalone"
if s.cfg.Database.Mode == "managed" {
mode = "managed"
}
mode := "managed"
// Database status (real ping)
dbStatus := "n/a"
@@ -231,7 +228,7 @@ func (r *MethodRouter) handleStatus(ctx context.Context, client *Client, req *pr
sessionCount = len(r.server.sessions.List(""))
}
// In managed mode, agents are lazily resolved — router only has loaded agents.
// Agents are lazily resolved — router only has loaded agents.
// Query the DB store for the real total count.
agentTotal := len(agents)
if r.server.agentStore != nil {
internal/gateway/server.go
+29 -19
View File
@@ -33,16 +33,16 @@ type Server struct {
policyEngine *permissions.PolicyEngine
pairingService store.PairingStore
agentsHandler *httpapi.AgentsHandler // managed mode: agent CRUD API
skillsHandler *httpapi.SkillsHandler // managed mode: skill management API
tracesHandler *httpapi.TracesHandler // managed mode: LLM trace listing API
mcpHandler *httpapi.MCPHandler // managed mode: MCP server management API
customToolsHandler *httpapi.CustomToolsHandler // managed mode: custom tool CRUD API
channelInstancesHandler *httpapi.ChannelInstancesHandler // managed mode: channel instance CRUD API
providersHandler *httpapi.ProvidersHandler // managed mode: provider CRUD API
delegationsHandler *httpapi.DelegationsHandler // managed mode: delegation history API
builtinToolsHandler *httpapi.BuiltinToolsHandler // managed mode: builtin tool management API
agentStore store.AgentStore // managed mode: for context injection in tools_invoke
agentsHandler *httpapi.AgentsHandler // agent CRUD API
skillsHandler *httpapi.SkillsHandler // skill management API
tracesHandler *httpapi.TracesHandler // LLM trace listing API
mcpHandler *httpapi.MCPHandler // MCP server management API
customToolsHandler *httpapi.CustomToolsHandler // custom tool CRUD API
channelInstancesHandler *httpapi.ChannelInstancesHandler // channel instance CRUD API
providersHandler *httpapi.ProvidersHandler // provider CRUD API
delegationsHandler *httpapi.DelegationsHandler // delegation history API
builtinToolsHandler *httpapi.BuiltinToolsHandler // builtin tool management API
agentStore store.AgentStore // for context injection in tools_invoke
upgrader websocket.Upgrader
rateLimiter *RateLimiter
@@ -53,6 +53,8 @@ type Server struct {
version string
db interface{ PingContext(context.Context) error } // for health check DB ping
logTee *LogTee // optional; auto-unsubscribes clients on disconnect
httpServer *http.Server
mux *http.ServeMux
}
@@ -269,33 +271,33 @@ func (s *Server) SetPolicyEngine(pe *permissions.PolicyEngine) { s.policyEngine
// SetPairingService sets the pairing service for channel authentication.
func (s *Server) SetPairingService(ps store.PairingStore) { s.pairingService = ps }
// SetAgentsHandler sets the managed-mode agent CRUD handler.
// SetAgentsHandler sets the agent CRUD handler.
func (s *Server) SetAgentsHandler(h *httpapi.AgentsHandler) { s.agentsHandler = h }
// SetSkillsHandler sets the managed-mode skill management handler.
// SetSkillsHandler sets the skill management handler.
func (s *Server) SetSkillsHandler(h *httpapi.SkillsHandler) { s.skillsHandler = h }
// SetTracesHandler sets the managed-mode LLM trace listing handler.
// SetTracesHandler sets the LLM trace listing handler.
func (s *Server) SetTracesHandler(h *httpapi.TracesHandler) { s.tracesHandler = h }
// SetMCPHandler sets the managed-mode MCP server management handler.
// SetMCPHandler sets the MCP server management handler.
func (s *Server) SetMCPHandler(h *httpapi.MCPHandler) { s.mcpHandler = h }
// SetCustomToolsHandler sets the managed-mode custom tool CRUD handler.
// SetCustomToolsHandler sets the custom tool CRUD handler.
func (s *Server) SetCustomToolsHandler(h *httpapi.CustomToolsHandler) { s.customToolsHandler = h }
// SetChannelInstancesHandler sets the managed-mode channel instance CRUD handler.
// SetChannelInstancesHandler sets the channel instance CRUD handler.
func (s *Server) SetChannelInstancesHandler(h *httpapi.ChannelInstancesHandler) {
s.channelInstancesHandler = h
}
// SetProvidersHandler sets the managed-mode provider CRUD handler.
// SetProvidersHandler sets the provider CRUD handler.
func (s *Server) SetProvidersHandler(h *httpapi.ProvidersHandler) { s.providersHandler = h }
// SetDelegationsHandler sets the managed-mode delegation history handler.
// SetDelegationsHandler sets the delegation history handler.
func (s *Server) SetDelegationsHandler(h *httpapi.DelegationsHandler) { s.delegationsHandler = h }
// SetBuiltinToolsHandler sets the managed-mode builtin tool management handler.
// SetBuiltinToolsHandler sets the builtin tool management handler.
func (s *Server) SetBuiltinToolsHandler(h *httpapi.BuiltinToolsHandler) {
s.builtinToolsHandler = h
}
@@ -375,9 +377,17 @@ func (s *Server) unregisterClient(c *Client) {
defer s.mu.Unlock()
delete(s.clients, c.id)
s.eventPub.Unsubscribe(c.id)
if s.logTee != nil {
s.logTee.Unsubscribe(c.id)
}
slog.Info("client disconnected", "id", c.id)
}
// SetLogTee attaches a LogTee so that disconnecting clients are auto-unsubscribed.
func (s *Server) SetLogTee(lt *LogTee) {
s.logTee = lt
}
// StartTestServer creates a listener on :0 (random port) and returns the
// actual address and a start function. Used for integration tests.
func StartTestServer(s *Server, ctx context.Context) (addr string, start func()) {
internal/heartbeat/service.go
-412
View File
@@ -1,412 +0,0 @@
// Package heartbeat provides a periodic background agent runner.
// Matching TS src/infra/heartbeat-runner.ts.
//
// The heartbeat wakes the agent at regular intervals so it can check on
// things (calendar, inbox, alerts) and surface anything that needs attention.
// If nothing needs attention the agent replies with HEARTBEAT_OK which is
// silently dropped.
package heartbeat
import (
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/nextlevelbuilder/goclaw/internal/bus"
"github.com/nextlevelbuilder/goclaw/internal/config"
)
// Default heartbeat prompt matching TS.
const defaultPrompt = "Read HEARTBEAT.md if it exists (workspace context). Follow it strictly. " +
"Do not infer or repeat old tasks from prior chats. " +
"If nothing needs attention, reply HEARTBEAT_OK."
const defaultInterval = 30 * time.Minute
// DefaultInterval returns the default heartbeat interval (30m).
func DefaultInterval() time.Duration { return defaultInterval }
const defaultAckMaxChars = 300
const heartbeatOKToken = "HEARTBEAT_OK"
// AgentRunner is the callback the service uses to run an agent turn.
// It returns the agent's response text and an error.
type AgentRunner func(ctx context.Context, agentID, sessionKey, message, runID string) (string, error)
// DeliveryTarget holds resolved delivery info for a heartbeat alert.
type DeliveryTarget struct {
Channel string
ChatID string
}
// LastUsedResolver returns the last-used channel + chatID for an agent.
// Returns ("", "") if unknown.
type LastUsedResolver func(agentID string) (channel, chatID string)
// Config holds resolved runtime config for the heartbeat service.
type Config struct {
AgentID string
Interval time.Duration
ActiveHours *config.ActiveHoursConfig
Model string // unused for now, reserved for model override
SessionKey string
Target string // "last", "none", or channel name
To string // explicit chat ID
Prompt string
AckMaxChars int
Workspace string // for HEARTBEAT.md detection
}
// Service manages the periodic heartbeat loop.
type Service struct {
cfg Config
runner AgentRunner
msgBus *bus.MessageBus
lastUsed LastUsedResolver
mu sync.Mutex
running bool
cancel context.CancelFunc
lastContent string // dedup: last non-OK content
lastAlertAt time.Time // dedup: when last alert was sent
}
// NewService creates a heartbeat service.
func NewService(cfg Config, runner AgentRunner, msgBus *bus.MessageBus, lastUsed LastUsedResolver) *Service {
if cfg.Interval <= 0 {
cfg.Interval = defaultInterval
}
if cfg.Prompt == "" {
cfg.Prompt = defaultPrompt
}
if cfg.AckMaxChars <= 0 {
cfg.AckMaxChars = defaultAckMaxChars
}
if cfg.SessionKey == "" {
cfg.SessionKey = fmt.Sprintf("agent:%s:heartbeat:main", cfg.AgentID)
}
if cfg.Target == "" {
cfg.Target = "last"
}
return &Service{
cfg: cfg,
runner: runner,
msgBus: msgBus,
lastUsed: lastUsed,
}
}
// Start begins the heartbeat loop in a background goroutine.
func (s *Service) Start() {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
return
}
ctx, cancel := context.WithCancel(context.Background())
s.cancel = cancel
s.running = true
go s.loop(ctx)
slog.Info("heartbeat service started",
"agent", s.cfg.AgentID,
"interval", s.cfg.Interval,
"target", s.cfg.Target,
)
}
// Stop halts the heartbeat loop.
func (s *Service) Stop() {
s.mu.Lock()
defer s.mu.Unlock()
if !s.running {
return
}
s.cancel()
s.running = false
slog.Info("heartbeat service stopped", "agent", s.cfg.AgentID)
}
// IsRunning returns whether the heartbeat loop is active.
func (s *Service) IsRunning() bool {
s.mu.Lock()
defer s.mu.Unlock()
return s.running
}
// --- Internal loop ---
func (s *Service) loop(ctx context.Context) {
// Initial delay: wait one full interval before first heartbeat
// (matching TS behavior — don't fire immediately on startup).
timer := time.NewTimer(s.cfg.Interval)
defer timer.Stop()
for {
select {
case <-ctx.Done():
return
case <-timer.C:
s.tick(ctx)
timer.Reset(s.cfg.Interval)
}
}
}
func (s *Service) tick(ctx context.Context) {
// Check active hours
if s.cfg.ActiveHours != nil && !isInActiveHours(s.cfg.ActiveHours) {
slog.Debug("heartbeat skipped: outside active hours", "agent", s.cfg.AgentID)
return
}
// Check if HEARTBEAT.md is effectively empty
if s.isHeartbeatFileEmpty() {
slog.Debug("heartbeat skipped: HEARTBEAT.md empty", "agent", s.cfg.AgentID)
return
}
// Run agent turn
runID := fmt.Sprintf("heartbeat-%s-%d", s.cfg.AgentID, time.Now().UnixMilli())
reply, err := s.runner(ctx, s.cfg.AgentID, s.cfg.SessionKey, s.cfg.Prompt, runID)
if err != nil {
slog.Warn("heartbeat agent run failed", "agent", s.cfg.AgentID, "error", err)
return
}
// Normalize response: strip HEARTBEAT_OK token
content, isOK := stripHeartbeatToken(reply, s.cfg.AckMaxChars)
if isOK {
slog.Debug("heartbeat OK", "agent", s.cfg.AgentID)
return
}
// Dedup: skip if same content within 24h
s.mu.Lock()
if content == s.lastContent && time.Since(s.lastAlertAt) < 24*time.Hour {
s.mu.Unlock()
slog.Debug("heartbeat dedup: same content within 24h", "agent", s.cfg.AgentID)
return
}
s.lastContent = content
s.lastAlertAt = time.Now()
s.mu.Unlock()
// Deliver alert
s.deliver(content)
}
// deliver sends the heartbeat alert to the configured target.
func (s *Service) deliver(content string) {
if s.cfg.Target == "none" {
slog.Info("heartbeat alert (target=none, not delivered)",
"agent", s.cfg.AgentID,
"preview", truncate(content, 100),
)
return
}
channel, chatID := s.resolveTarget()
if channel == "" || chatID == "" {
slog.Warn("heartbeat alert: no delivery target resolved",
"agent", s.cfg.AgentID,
"target", s.cfg.Target,
)
return
}
slog.Info("heartbeat alert delivered",
"agent", s.cfg.AgentID,
"channel", channel,
"chatID", chatID,
"preview", truncate(content, 100),
)
s.msgBus.PublishOutbound(bus.OutboundMessage{
Channel: channel,
ChatID: chatID,
Content: content,
})
}
// resolveTarget determines where to deliver heartbeat alerts.
func (s *Service) resolveTarget() (channel, chatID string) {
// Explicit channel target
if s.cfg.Target != "" && s.cfg.Target != "last" && s.cfg.Target != "none" {
channel = s.cfg.Target
chatID = s.cfg.To
return
}
// "last" — ask the resolver
if s.lastUsed != nil {
channel, chatID = s.lastUsed(s.cfg.AgentID)
}
// Override chatID if explicitly set
if s.cfg.To != "" {
chatID = s.cfg.To
}
return
}
// isHeartbeatFileEmpty checks if HEARTBEAT.md exists and has meaningful content.
// Matching TS isHeartbeatContentEffectivelyEmpty().
func (s *Service) isHeartbeatFileEmpty() bool {
if s.cfg.Workspace == "" {
return true
}
path := filepath.Join(s.cfg.Workspace, "HEARTBEAT.md")
data, err := os.ReadFile(path)
if err != nil {
return true // file doesn't exist = empty
}
return isEffectivelyEmpty(string(data))
}
// isEffectivelyEmpty returns true if content has no meaningful text
// (only whitespace, markdown headers, empty list items, comments).
// Matching TS isHeartbeatContentEffectivelyEmpty().
func isEffectivelyEmpty(content string) bool {
for _, line := range strings.Split(content, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
// Skip markdown headers (# ...)
if strings.HasPrefix(line, "#") {
trimmed := strings.TrimLeft(line, "# ")
if trimmed == "" {
continue
}
// Header with text = not empty
return false
}
// Skip comments (<!-- -->)
if strings.HasPrefix(line, "<!--") {
continue
}
// Skip empty list items (- , * )
if (strings.HasPrefix(line, "- ") || strings.HasPrefix(line, "* ")) && strings.TrimSpace(line[2:]) == "" {
continue
}
// Any other non-empty line = meaningful content
return false
}
return true
}
// stripHeartbeatToken checks for HEARTBEAT_OK in the reply.
// Returns (remaining content, isOK).
// If HEARTBEAT_OK is found at start/end and remaining content <= ackMaxChars, treat as OK.
// Matching TS stripHeartbeatToken().
func stripHeartbeatToken(reply string, ackMaxChars int) (string, bool) {
trimmed := strings.TrimSpace(reply)
// Exact match
if trimmed == heartbeatOKToken {
return "", true
}
// Strip common markdown/HTML wrappers
stripped := trimmed
for _, prefix := range []string{"**", "<b>", "<strong>", "`"} {
suffix := strings.Replace(prefix, "<", "</", 1)
if prefix == "**" {
suffix = "**"
}
if prefix == "`" {
suffix = "`"
}
stripped = strings.TrimPrefix(stripped, prefix)
stripped = strings.TrimSuffix(stripped, suffix)
}
stripped = strings.TrimSpace(stripped)
if stripped == heartbeatOKToken {
return "", true
}
// Check if token appears at start or end
hasPrefix := strings.HasPrefix(trimmed, heartbeatOKToken)
hasSuffix := strings.HasSuffix(trimmed, heartbeatOKToken)
if !hasPrefix && !hasSuffix {
// Token in middle = not an ack, return full content
return trimmed, false
}
remaining := trimmed
if hasPrefix {
remaining = strings.TrimSpace(strings.TrimPrefix(remaining, heartbeatOKToken))
}
if hasSuffix {
remaining = strings.TrimSpace(strings.TrimSuffix(remaining, heartbeatOKToken))
}
// If remaining content is short enough, treat as OK
if len(remaining) <= ackMaxChars {
return "", true
}
// Long remaining content = not just an ack
return remaining, false
}
// --- Active Hours ---
// isInActiveHours checks if the current time is within the configured window.
// Matching TS heartbeat-active-hours.ts.
func isInActiveHours(cfg *config.ActiveHoursConfig) bool {
if cfg == nil || cfg.Start == "" || cfg.End == "" {
return true // no restriction
}
now := time.Now()
if cfg.Timezone != "" {
loc, err := time.LoadLocation(cfg.Timezone)
if err == nil {
now = now.In(loc)
}
}
startH, startM := parseHHMM(cfg.Start)
endH, endM := parseHHMM(cfg.End)
currentMin := now.Hour()*60 + now.Minute()
startMin := startH*60 + startM
endMin := endH*60 + endM
if startMin <= endMin {
// Normal range: e.g. 08:00 - 22:00
return currentMin >= startMin && currentMin < endMin
}
// Wrap-around: e.g. 22:00 - 06:00
return currentMin >= startMin || currentMin < endMin
}
// parseHHMM parses "HH:MM" into hours and minutes.
func parseHHMM(s string) (int, int) {
var h, m int
fmt.Sscanf(s, "%d:%d", &h, &m)
return h, m
}
// truncate returns the first n characters of s.
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "..."
}
internal/http/agents.go
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// AgentsHandler handles agent CRUD and sharing endpoints (managed mode only).
// AgentsHandler handles agent CRUD and sharing endpoints.
type AgentsHandler struct {
agents store.AgentStore
token string
internal/http/auth.go
+1 -1
View File
@@ -31,7 +31,7 @@ func tokenMatch(provided, expected string) bool {
}
// extractUserID extracts the external user ID from the request header.
// Returns "" if no user ID is provided (anonymous / standalone mode).
// Returns "" if no user ID is provided (anonymous).
// Rejects IDs exceeding MaxUserIDLength (VARCHAR(255) DB constraint).
func extractUserID(r *http.Request) string {
id := r.Header.Get("X-GoClaw-User-Id")
internal/http/builtin_tools.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// BuiltinToolsHandler handles built-in tool management endpoints (managed mode).
// BuiltinToolsHandler handles built-in tool management endpoints.
// Built-in tools are seeded at startup; only enabled and settings are editable.
type BuiltinToolsHandler struct {
store store.BuiltinToolStore
internal/http/channel_instances.go
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// ChannelInstancesHandler handles channel instance CRUD endpoints (managed mode).
// ChannelInstancesHandler handles channel instance CRUD endpoints.
type ChannelInstancesHandler struct {
store store.ChannelInstanceStore
agentStore store.AgentStore
internal/http/custom_tools.go
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// CustomToolsHandler handles custom tool CRUD endpoints (managed mode).
// CustomToolsHandler handles custom tool CRUD endpoints.
type CustomToolsHandler struct {
store store.CustomToolStore
token string
internal/http/mcp.go
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// MCPHandler handles MCP server management HTTP endpoints (managed mode).
// MCPHandler handles MCP server management HTTP endpoints.
type MCPHandler struct {
store store.MCPServerStore
token string
internal/http/provider_models.go
+12
View File
@@ -53,6 +53,8 @@ func (h *ProvidersHandler) handleListProviderModels(w http.ResponseWriter, r *ht
models, err = fetchGeminiModels(ctx, p.APIKey)
case "bailian":
models = bailianModels()
case "minimax_native":
models = minimaxModels()
default:
// All other types use OpenAI-compatible /models endpoint
apiBase := strings.TrimRight(p.APIBase, "/")
@@ -163,6 +165,16 @@ func bailianModels() []ModelInfo {
}
}
// minimaxModels returns a hardcoded list of MiniMax models.
// MiniMax does not expose a /v1/models endpoint.
func minimaxModels() []ModelInfo {
return []ModelInfo{
{ID: "MiniMax-Text-01", Name: "MiniMax Text 01"},
{ID: "MiniMax-M1", Name: "MiniMax M1"},
{ID: "MiniMax-M2.5", Name: "MiniMax M2.5"},
}
}
// fetchOpenAIModels calls an OpenAI-compatible /models endpoint.
func fetchOpenAIModels(ctx context.Context, apiBase, apiKey string) ([]ModelInfo, error) {
req, err := http.NewRequestWithContext(ctx, "GET", apiBase+"/models", nil)
internal/http/providers.go
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/store"
)
// ProvidersHandler handles LLM provider CRUD endpoints (managed mode).
// ProvidersHandler handles LLM provider CRUD endpoints.
type ProvidersHandler struct {
store store.ProviderStore
token string
internal/http/skills.go
+1 -1
View File
@@ -25,7 +25,7 @@ const maxSkillUploadSize = 20 << 20 // 20 MB
var slugRegexp = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*[a-z0-9]$`)
// SkillsHandler handles skill management HTTP endpoints (managed mode).
// SkillsHandler handles skill management HTTP endpoints.
type SkillsHandler struct {
skills *pg.PGSkillStore
baseDir string // filesystem base for skill content
internal/http/summoner.go
+1 -1
View File
@@ -29,7 +29,7 @@ const (
const frontmatterKey = "__frontmatter__"
// summoningFiles is the ordered list of context files the LLM should generate.
// Only personality files — operational files (AGENTS.md, TOOLS.md, HEARTBEAT.md)
// Only personality files — operational files (AGENTS.md, TOOLS.md)
// are kept as fixed templates from bootstrap.SeedToStore().
// USER_PREDEFINED.md is optional — generated only when description mentions user context.
var summoningFiles = []string{
internal/http/tools_invoke.go
+1 -1
View File
@@ -14,7 +14,7 @@ import (
type ToolsInvokeHandler struct {
registry *tools.Registry
token string
agentStore store.AgentStore // nil in standalone mode
agentStore store.AgentStore // nil if not configured
}
// NewToolsInvokeHandler creates a handler for the tools invoke endpoint.
internal/http/traces.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/store"
)
// TracesHandler handles LLM trace listing and detail endpoints (managed mode).
// TracesHandler handles LLM trace listing and detail endpoints.
type TracesHandler struct {
tracing store.TracingStore
token string
internal/mcp/manager.go
+9 -9
View File
@@ -48,32 +48,32 @@ type serverState struct {
}
// Manager orchestrates MCP server connections and tool registration.
// Supports two modes:
// - Standalone: reads from config.MCPServerConfig map (shared across all agents)
// - Managed: queries MCPServerStore per agent+user for permission-filtered servers
// Supports two sources:
// - Config-based: reads from config.MCPServerConfig map (shared across all agents)
// - DB-backed: queries MCPServerStore per agent+user for permission-filtered servers
type Manager struct {
mu sync.RWMutex
servers map[string]*serverState
registry *tools.Registry
// Standalone mode
// Config-based servers
configs map[string]*config.MCPServerConfig
// Managed mode
// DB-backed servers
store store.MCPServerStore
}
// ManagerOption configures the Manager.
type ManagerOption func(*Manager)
// WithConfigs sets static MCP server configs (standalone mode).
// WithConfigs sets static MCP server configs from the config file.
func WithConfigs(cfgs map[string]*config.MCPServerConfig) ManagerOption {
return func(m *Manager) {
m.configs = cfgs
}
}
// WithStore sets the MCPServerStore for managed mode.
// WithStore sets the MCPServerStore for DB-backed MCP server loading.
func WithStore(s store.MCPServerStore) ManagerOption {
return func(m *Manager) {
m.store = s
@@ -92,7 +92,7 @@ func NewManager(registry *tools.Registry, opts ...ManagerOption) *Manager {
return m
}
// Start connects to all configured MCP servers (standalone mode).
// Start connects to all config-file MCP servers.
// Non-fatal: logs warnings for servers that fail to connect and continues.
func (m *Manager) Start(ctx context.Context) error {
if len(m.configs) == 0 {
@@ -118,7 +118,7 @@ func (m *Manager) Start(ctx context.Context) error {
return nil
}
// LoadForAgent connects MCP servers accessible by a specific agent+user (managed mode).
// LoadForAgent connects MCP servers accessible by a specific agent+user.
// Previously registered MCP tools for this manager are cleared and reloaded.
func (m *Manager) LoadForAgent(ctx context.Context, agentID uuid.UUID, userID string) error {
if m.store == nil {
internal/memory/chunking.go
-68
View File
@@ -1,68 +0,0 @@
package memory
import (
"strings"
)
// ChunkText splits text into chunks at paragraph boundaries.
// Each chunk includes its starting line number in the source file.
func ChunkText(text string, maxChunkLen int) []TextChunk {
if maxChunkLen <= 0 {
maxChunkLen = 1000
}
lines := strings.Split(text, "\n")
var chunks []TextChunk
var current strings.Builder
startLine := 1
flush := func(endLine int) {
content := strings.TrimSpace(current.String())
if content != "" {
chunks = append(chunks, TextChunk{
Text: content,
StartLine: startLine,
EndLine: endLine,
})
}
current.Reset()
startLine = endLine + 1
}
for i, line := range lines {
lineNum := i + 1
// Paragraph boundary: empty line
if strings.TrimSpace(line) == "" && current.Len() > 0 {
// If current chunk is large enough, flush
if current.Len() >= maxChunkLen/2 {
flush(lineNum - 1)
continue
}
}
if current.Len() > 0 {
current.WriteString("\n")
}
current.WriteString(line)
// Force flush if too large
if current.Len() >= maxChunkLen {
flush(lineNum)
}
}
// Flush remaining
if current.Len() > 0 {
flush(len(lines))
}
return chunks
}
// TextChunk is a chunk of text with line number metadata.
type TextChunk struct {
Text string
StartLine int
EndLine int
}
internal/memory/embeddings.go
+69
View File
@@ -3,13 +3,82 @@ package memory
import (
"bytes"
"context"
"crypto/sha256"
"encoding/json"
"fmt"
"io"
"math"
"net/http"
"strings"
)
// ContentHash returns a short SHA256 hex digest of the content (first 16 bytes).
func ContentHash(text string) string {
h := sha256.Sum256([]byte(text))
return fmt.Sprintf("%x", h[:16])
}
// TextChunk is a chunk of text with line number metadata.
type TextChunk struct {
Text string
StartLine int
EndLine int
}
// ChunkText splits text into chunks at paragraph boundaries.
// Each chunk includes its starting line number in the source file.
func ChunkText(text string, maxChunkLen int) []TextChunk {
if maxChunkLen <= 0 {
maxChunkLen = 1000
}
lines := strings.Split(text, "\n")
var chunks []TextChunk
var current strings.Builder
startLine := 1
flush := func(endLine int) {
content := strings.TrimSpace(current.String())
if content != "" {
chunks = append(chunks, TextChunk{
Text: content,
StartLine: startLine,
EndLine: endLine,
})
}
current.Reset()
startLine = endLine + 1
}
for i, line := range lines {
lineNum := i + 1
// Paragraph boundary: empty line
if strings.TrimSpace(line) == "" && current.Len() > 0 {
if current.Len() >= maxChunkLen/2 {
flush(lineNum - 1)
continue
}
}
if current.Len() > 0 {
current.WriteString("\n")
}
current.WriteString(line)
// Force flush if too large
if current.Len() >= maxChunkLen {
flush(lineNum)
}
}
if current.Len() > 0 {
flush(len(lines))
}
return chunks
}
// EmbeddingProvider generates vector embeddings for text.
type EmbeddingProvider interface {
// Name returns the provider identifier (e.g., "openai", "voyage").
internal/memory/manager.go
-292
View File
@@ -1,292 +0,0 @@
package memory
import (
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"sync"
"github.com/nextlevelbuilder/goclaw/internal/bootstrap"
)
// ManagerConfig configures the memory manager.
type ManagerConfig struct {
DBPath string // path to SQLite database file
MemoryDir string // directory containing memory files (MEMORY.md, memory/*.md)
MaxChunkLen int // max characters per chunk (default 1000)
MaxResults int // default max search results (default 6)
VectorWeight float64
TextWeight float64
}
// DefaultManagerConfig returns sensible defaults.
func DefaultManagerConfig(workspace string) ManagerConfig {
return ManagerConfig{
DBPath: filepath.Join(workspace, "memory.db"),
MemoryDir: workspace,
MaxChunkLen: 1000,
MaxResults: 6,
VectorWeight: 0.7,
TextWeight: 0.3,
}
}
// Manager coordinates memory indexing and search.
type Manager struct {
store *SQLiteStore
provider EmbeddingProvider
config ManagerConfig
mu sync.RWMutex
watcher *Watcher
}
// NewManager creates a memory manager.
func NewManager(cfg ManagerConfig) (*Manager, error) {
// Ensure directory exists
if err := os.MkdirAll(filepath.Dir(cfg.DBPath), 0755); err != nil {
return nil, fmt.Errorf("create db dir: %w", err)
}
store, err := NewSQLiteStore(cfg.DBPath)
if err != nil {
return nil, err
}
return &Manager{
store: store,
config: cfg,
}, nil
}
// SetEmbeddingProvider sets the embedding provider for vector search.
// If not set, only FTS search is available.
func (m *Manager) SetEmbeddingProvider(provider EmbeddingProvider) {
m.mu.Lock()
defer m.mu.Unlock()
m.provider = provider
}
// IndexFile reads a file, chunks it, and stores the chunks in the database.
// Skips unchanged files (by content hash, matching TS manager-sync-ops.ts)
// and optionally generates embeddings.
func (m *Manager) IndexFile(ctx context.Context, path string) error {
absPath := path
if !filepath.IsAbs(path) {
absPath = filepath.Join(m.config.MemoryDir, path)
}
data, err := os.ReadFile(absPath)
if err != nil {
return fmt.Errorf("read %s: %w", path, err)
}
text := string(data)
relPath := path
if filepath.IsAbs(relPath) {
if rel, err := filepath.Rel(m.config.MemoryDir, relPath); err == nil {
relPath = rel
}
}
// Hash-based change detection (matching TS: compare SHA256 hash, skip if unchanged)
fileHash := ContentHash(text)
if existingHash, ok := m.store.GetFileHash(relPath); ok && existingHash == fileHash {
slog.Debug("skipping unchanged file", "path", relPath)
return nil
}
chunks := ChunkText(text, m.config.MaxChunkLen)
// Delete old chunks for this path
if err := m.store.DeleteByPath(relPath); err != nil {
return fmt.Errorf("delete old chunks for %s: %w", relPath, err)
}
// Collect texts for batch embedding
var textsToEmbed []string
var chunkEntries []Chunk
for i, tc := range chunks {
hash := ContentHash(tc.Text)
id := fmt.Sprintf("%s#%d", relPath, i)
entry := Chunk{
ID: id,
Path: relPath,
Source: "memory",
StartLine: tc.StartLine,
EndLine: tc.EndLine,
Hash: hash,
Text: tc.Text,
}
chunkEntries = append(chunkEntries, entry)
textsToEmbed = append(textsToEmbed, tc.Text)
}
// Generate embeddings if provider is available
m.mu.RLock()
provider := m.provider
m.mu.RUnlock()
if provider != nil && len(textsToEmbed) > 0 {
embeddings, err := provider.Embed(ctx, textsToEmbed)
if err != nil {
slog.Warn("embedding generation failed, indexing without vectors", "path", relPath, "error", err)
} else {
for i := range chunkEntries {
if i < len(embeddings) {
chunkEntries[i].Embedding = embeddings[i]
chunkEntries[i].Model = provider.Model()
}
}
}
}
// Store chunks
for _, entry := range chunkEntries {
if err := m.store.UpsertChunk(entry); err != nil {
slog.Warn("failed to store chunk", "id", entry.ID, "error", err)
}
}
// Update file metadata for future change detection
var mtime int64
var size int64
if info, err := os.Stat(absPath); err == nil {
mtime = info.ModTime().UnixMilli()
size = info.Size()
}
m.store.UpsertFile(relPath, "memory", fileHash, mtime, size)
slog.Debug("indexed file", "path", relPath, "chunks", len(chunkEntries))
return nil
}
// IndexAll scans the memory directory for .md files and indexes them all.
func (m *Manager) IndexAll(ctx context.Context) error {
memoryDir := m.config.MemoryDir
// Index MEMORY.md at root
memoryFile := filepath.Join(memoryDir, bootstrap.MemoryFile)
if _, err := os.Stat(memoryFile); err == nil {
if err := m.IndexFile(ctx, memoryFile); err != nil {
slog.Warn("failed to index MEMORY.md", "error", err)
}
}
// Index memory/*.md
memSubDir := filepath.Join(memoryDir, "memory")
if _, err := os.Stat(memSubDir); err == nil {
filepath.Walk(memSubDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
if strings.HasSuffix(path, ".md") {
if err := m.IndexFile(ctx, path); err != nil {
slog.Warn("failed to index file", "path", path, "error", err)
}
}
return nil
})
}
slog.Info("memory indexing complete", "chunks", m.store.ChunkCount())
return nil
}
// Search performs a hybrid search over indexed memory.
func (m *Manager) Search(ctx context.Context, query string, opts SearchOptions) ([]SearchResult, error) {
if opts.MaxResults <= 0 {
opts.MaxResults = m.config.MaxResults
}
m.mu.RLock()
provider := m.provider
m.mu.RUnlock()
hybridCfg := HybridSearchConfig{
VectorWeight: m.config.VectorWeight,
TextWeight: m.config.TextWeight,
}
if hybridCfg.VectorWeight == 0 && hybridCfg.TextWeight == 0 {
hybridCfg = DefaultHybridConfig()
}
return HybridSearch(ctx, m.store, provider, query, opts, hybridCfg)
}
// GetFile reads a memory file (or a section of it) and returns the content.
func (m *Manager) GetFile(path string, fromLine, numLines int) (string, error) {
absPath := path
if !filepath.IsAbs(path) {
absPath = filepath.Join(m.config.MemoryDir, path)
}
data, err := os.ReadFile(absPath)
if err != nil {
return "", fmt.Errorf("read %s: %w", path, err)
}
text := string(data)
if fromLine <= 0 && numLines <= 0 {
return text, nil
}
lines := strings.Split(text, "\n")
start := fromLine - 1 // 1-indexed to 0-indexed
if start < 0 {
start = 0
}
if start >= len(lines) {
return "", nil
}
end := len(lines)
if numLines > 0 {
end = start + numLines
if end > len(lines) {
end = len(lines)
}
}
return strings.Join(lines[start:end], "\n"), nil
}
// ChunkCount returns the number of indexed chunks.
func (m *Manager) ChunkCount() int {
return m.store.ChunkCount()
}
// StartWatcher starts a file watcher that monitors memory files for changes
// and re-indexes them automatically with debounce (matching TS chokidar pattern).
func (m *Manager) StartWatcher(ctx context.Context) error {
w, err := newWatcher(m)
if err != nil {
return fmt.Errorf("create watcher: %w", err)
}
if err := w.start(ctx); err != nil {
return fmt.Errorf("start watcher: %w", err)
}
m.watcher = w
return nil
}
// StopWatcher stops the file watcher if running.
func (m *Manager) StopWatcher() {
if m.watcher != nil {
m.watcher.stop()
m.watcher = nil
}
}
// Close shuts down the memory manager (including watcher).
func (m *Manager) Close() error {
m.StopWatcher()
return m.store.Close()
}
internal/memory/memory_test.go
-287
View File
@@ -1,287 +0,0 @@
package memory
import (
"context"
"os"
"path/filepath"
"testing"
)
func TestChunkText(t *testing.T) {
text := `# Title
First paragraph with some content.
More content in the same paragraph.
Second paragraph here.
And a second line.
Third paragraph is short.`
chunks := ChunkText(text, 100)
if len(chunks) < 2 {
t.Fatalf("expected at least 2 chunks, got %d", len(chunks))
}
// First chunk should start at line 1
if chunks[0].StartLine != 1 {
t.Errorf("first chunk start line = %d, want 1", chunks[0].StartLine)
}
// All chunks should have text
for i, c := range chunks {
if c.Text == "" {
t.Errorf("chunk %d has empty text", i)
}
}
}
func TestChunkText_SingleParagraph(t *testing.T) {
text := "Short text."
chunks := ChunkText(text, 1000)
if len(chunks) != 1 {
t.Fatalf("expected 1 chunk, got %d", len(chunks))
}
if chunks[0].Text != "Short text." {
t.Errorf("text = %q, want %q", chunks[0].Text, "Short text.")
}
}
func TestSQLiteStore_CRUD(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "test.db")
store, err := NewSQLiteStore(dbPath)
if err != nil {
t.Fatalf("NewSQLiteStore: %v", err)
}
defer store.Close()
// Insert chunk
chunk := Chunk{
ID: "test#0",
Path: "MEMORY.md",
Source: "memory",
StartLine: 1,
EndLine: 5,
Hash: ContentHash("hello world"),
Text: "hello world this is a test",
}
if err := store.UpsertChunk(chunk); err != nil {
t.Fatalf("UpsertChunk: %v", err)
}
if count := store.ChunkCount(); count != 1 {
t.Errorf("ChunkCount = %d, want 1", count)
}
// Get by path
chunks, err := store.GetChunksByPath("MEMORY.md")
if err != nil {
t.Fatalf("GetChunksByPath: %v", err)
}
if len(chunks) != 1 {
t.Fatalf("GetChunksByPath returned %d chunks, want 1", len(chunks))
}
if chunks[0].Text != "hello world this is a test" {
t.Errorf("chunk text = %q", chunks[0].Text)
}
// Delete by path
if err := store.DeleteByPath("MEMORY.md"); err != nil {
t.Fatalf("DeleteByPath: %v", err)
}
if count := store.ChunkCount(); count != 0 {
t.Errorf("after delete, ChunkCount = %d, want 0", count)
}
}
func TestSQLiteStore_FTSSearch(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "test.db")
store, err := NewSQLiteStore(dbPath)
if err != nil {
t.Fatalf("NewSQLiteStore: %v", err)
}
defer store.Close()
// Insert multiple chunks
chunks := []Chunk{
{ID: "memo#0", Path: "MEMORY.md", Source: "memory", StartLine: 1, EndLine: 3, Hash: "h1", Text: "The project uses Go for backend development with SQLite as the database"},
{ID: "memo#1", Path: "MEMORY.md", Source: "memory", StartLine: 4, EndLine: 6, Hash: "h2", Text: "Authentication is handled via JWT tokens and API keys"},
{ID: "notes#0", Path: "memory/notes.md", Source: "memory", StartLine: 1, EndLine: 2, Hash: "h3", Text: "Go is a compiled programming language designed at Google"},
}
for _, c := range chunks {
if err := store.UpsertChunk(c); err != nil {
t.Fatalf("UpsertChunk: %v", err)
}
}
// Search for "Go"
results, err := store.SearchFTS("Go", SearchOptions{MaxResults: 10})
if err != nil {
t.Fatalf("SearchFTS: %v", err)
}
if len(results) < 2 {
t.Errorf("expected at least 2 results for 'Go', got %d", len(results))
}
// Search for "authentication"
results, err = store.SearchFTS("authentication", SearchOptions{MaxResults: 10})
if err != nil {
t.Fatalf("SearchFTS: %v", err)
}
if len(results) != 1 {
t.Errorf("expected 1 result for 'authentication', got %d", len(results))
}
// Search with path filter
results, err = store.SearchFTS("Go", SearchOptions{MaxResults: 10, PathPrefix: "memory/"})
if err != nil {
t.Fatalf("SearchFTS with path filter: %v", err)
}
if len(results) != 1 {
t.Errorf("expected 1 result for 'Go' in memory/, got %d", len(results))
}
}
func TestCosineSimilarity(t *testing.T) {
// Identical vectors → 1.0
a := []float32{1, 0, 0}
b := []float32{1, 0, 0}
if sim := CosineSimilarity(a, b); sim < 0.99 {
t.Errorf("identical vectors: similarity = %f, want ~1.0", sim)
}
// Orthogonal vectors → 0.0
a = []float32{1, 0}
b = []float32{0, 1}
if sim := CosineSimilarity(a, b); sim > 0.01 {
t.Errorf("orthogonal vectors: similarity = %f, want ~0.0", sim)
}
// Opposite vectors → -1.0
a = []float32{1, 0}
b = []float32{-1, 0}
if sim := CosineSimilarity(a, b); sim > -0.99 {
t.Errorf("opposite vectors: similarity = %f, want ~-1.0", sim)
}
}
func TestManager_IndexAndSearch(t *testing.T) {
tmpDir := t.TempDir()
// Create test memory files
memoryMD := filepath.Join(tmpDir, "MEMORY.md")
os.WriteFile(memoryMD, []byte("# Project Notes\n\nThe project uses Go for backend.\nDatabase is SQLite.\n\n## Architecture\n\nMicroservices pattern with message bus."), 0644)
memDir := filepath.Join(tmpDir, "memory")
os.MkdirAll(memDir, 0755)
os.WriteFile(filepath.Join(memDir, "config.md"), []byte("# Config\n\nConfiguration uses JSON5 format.\nSupports hot-reload via file watcher."), 0644)
// Create manager
cfg := DefaultManagerConfig(tmpDir)
mgr, err := NewManager(cfg)
if err != nil {
t.Fatalf("NewManager: %v", err)
}
defer mgr.Close()
// Index all files
ctx := context.Background()
if err := mgr.IndexAll(ctx); err != nil {
t.Fatalf("IndexAll: %v", err)
}
if count := mgr.ChunkCount(); count == 0 {
t.Fatal("no chunks indexed")
}
// Search (FTS only, no embedding provider)
results, err := mgr.Search(ctx, "Go backend", SearchOptions{MaxResults: 5})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(results) == 0 {
t.Error("expected search results for 'Go backend'")
}
// Search for config (FTS porter stemmer matches "configuration" → "configur")
results, err = mgr.Search(ctx, "configuration reload", SearchOptions{MaxResults: 5})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(results) == 0 {
t.Error("expected search results for 'configuration reload'")
}
}
func TestManager_GetFile(t *testing.T) {
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "MEMORY.md")
os.WriteFile(testFile, []byte("line1\nline2\nline3\nline4\nline5"), 0644)
cfg := DefaultManagerConfig(tmpDir)
mgr, err := NewManager(cfg)
if err != nil {
t.Fatalf("NewManager: %v", err)
}
defer mgr.Close()
// Read entire file
text, err := mgr.GetFile("MEMORY.md", 0, 0)
if err != nil {
t.Fatalf("GetFile: %v", err)
}
if text != "line1\nline2\nline3\nline4\nline5" {
t.Errorf("full file = %q", text)
}
// Read lines 2-4
text, err = mgr.GetFile("MEMORY.md", 2, 3)
if err != nil {
t.Fatalf("GetFile: %v", err)
}
if text != "line2\nline3\nline4" {
t.Errorf("lines 2-4 = %q", text)
}
}
func TestEmbeddingCache(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "test.db")
store, err := NewSQLiteStore(dbPath)
if err != nil {
t.Fatalf("NewSQLiteStore: %v", err)
}
defer store.Close()
emb := []float32{0.1, 0.2, 0.3}
hash := ContentHash("test text")
// Cache miss
if _, ok := store.GetCachedEmbedding(hash, "openai", "text-embedding-3-small"); ok {
t.Error("expected cache miss")
}
// Cache write
if err := store.CacheEmbedding(hash, "openai", "text-embedding-3-small", emb); err != nil {
t.Fatalf("CacheEmbedding: %v", err)
}
// Cache hit
cached, ok := store.GetCachedEmbedding(hash, "openai", "text-embedding-3-small")
if !ok {
t.Fatal("expected cache hit")
}
if len(cached) != 3 || cached[0] != 0.1 {
t.Errorf("cached embedding = %v", cached)
}
}
Loaded 100 of 183 files, more files were not shown because too many files have changed in this diff. Show more