feat(providers): add ACP provider for external coding agents (#190)

* feat(providers): add ACP provider for orchestrating external coding agents (#189) Implement native Go ACP (Agent Client Protocol) client as a new Provider. Enables GoClaw to orchestrate any ACP-compatible agent (Claude Code, Codex CLI, Gemini CLI) as a subprocess via JSON-RPC 2.0 over stdio. - Add bidirectional JSON-RPC 2.0 transport over stdio pipes - Add subprocess process pool with idle TTL reaping and crash recovery - Add ACP session lifecycle (initialize, session/new, session/prompt) - Add tool bridge for agent-initiated fs/terminal/permission requests - Add workspace sandboxing, shell deny patterns, and env var filtering - Wire config-based and DB-based provider registration paths - Export DefaultDenyPatterns from tools package for reuse * feat(providers): add changelog entry for ACP provider integration * fix(tools): prevent workspace traversal bypass via /tmp/ fallback in resolveMediaPath Reject paths containing ".." in the isInTempDir fallback to prevent workspace escape where traversal path still resolves inside /tmp/. * fix(tools): block workspace-sibling paths in resolveMediaPath /tmp/ fallback When workspace is inside /tmp/, traversal paths like workspace/../X resolve to /tmp/ siblings that pass isInTempDir. Reject paths inside the workspace parent directory to prevent this escape. * feat(providers): add ACP provider web UI and live reload via pubsub Web UI for creating/editing ACP providers with dedicated form fields (binary, args, idle TTL, permission mode, work directory). ACP providers now update immediately without gateway restart via cache invalidation pubsub pattern. Frontend: - New ACPSection form component with i18n (en/vi/zh) - Provider form dialog integration with ACP state management - ACP type badge on providers list page - Settings field added to provider TypeScript types Backend: - ACP models handler (claude/codex/gemini) without API key requirement - Binary path validation + LookPath verification in verify handler - Provider CRUD emits cache.invalidate events via msgBus - Subscriber in gateway_managed.go re-registers ACP providers from DB - ACP core improvements from code review (helpers, jsonrpc, process, terminal, tool_bridge) --------- Co-authored-by: viettranx <viettranx@gmail.com>
2026-10-20 00:15:16 +00:00 · 2026-03-14 16:16:08 +07:00
1 parent 16d726d5a2
commit 5e2fa395c7
@@ -42,6 +42,7 @@ flowchart TD
    subgraph Providers["LLM Providers"]
        ANTH[Anthropic -- Native HTTP + SSE]
        OAI["OpenAI-Compatible -- HTTP + SSE<br/>(OpenAI, Gemini, DeepSeek, DashScope, +8)"]
+        ACP["ACP -- JSON-RPC 2.0 stdio<br/>(Claude Code, Codex, Gemini CLI)"]
    end

    subgraph Tools["Tool Registry"]
@@ -109,7 +110,8 @@ flowchart TD
 | `internal/gateway/` | WebSocket + HTTP server, client handling, method router |
 | `internal/gateway/methods/` | RPC method handlers: chat, agents, agent_links, teams, delegations, sessions, config, skills, cron, pairing, exec approval, usage, send |
 | `internal/agent/` | Agent loop (think, act, observe), router, resolver, system prompt builder, sanitization, pruning, tracing, memory flush, DELEGATION.md + TEAM.md injection |
-| `internal/providers/` | LLM providers: Anthropic (native HTTP + SSE streaming), OpenAI-compatible (HTTP + SSE, 12+ providers), DashScope (Qwen), extended thinking support, retry logic |
+| `internal/providers/` | LLM providers: Anthropic (native HTTP + SSE), OpenAI-compatible (HTTP + SSE, 12+ providers), DashScope (Qwen), ACP (JSON-RPC 2.0 subprocess), extended thinking support, retry logic |
+| `internal/providers/acp/` | ACP protocol implementation: ProcessPool (subprocess lifecycle), ToolBridge (fs/terminal), session management |
 | `internal/tools/` | Tool registry, filesystem ops, exec/shell, policy engine, subagent, delegation manager, team tools, evaluate loop, handoff, context file + memory interceptors, credential scrubbing, rate limiting, PathDenyable |
 | `internal/tools/dynamic_loader.go` | Custom tool loader: LoadGlobal (startup), LoadForAgent (per-agent clone), ReloadGlobal (cache invalidation) |
 | `internal/tools/dynamic_tool.go` | Custom tool executor: command template rendering, shell escaping, encrypted env vars |
@@ -30,7 +30,7 @@ The Anthropic provider uses `x-api-key` header authentication and the `anthropic

 ## 2. Supported Providers

-| Provider | Type | API Base | Default Model |
+| Provider | Type | API Base / Binary | Default Model |
 |----------|------|----------|---------------|
 | anthropic | Native HTTP + SSE | `https://api.anthropic.com/v1` | `claude-sonnet-4-5-20250929` |
 | openai | OpenAI-compatible | `https://api.openai.com/v1` | `gpt-4o` |
@@ -47,6 +47,7 @@ The Anthropic provider uses `x-api-key` header authentication and the `anthropic
 | bailian | OpenAI-compatible | `https://coding-intl.dashscope.aliyuncs.com/v1` | `qwen3.5-plus` |
 | zai | OpenAI-compatible | `https://api.z.ai/api/paas/v4` | `glm-5` |
 | zai_coding | OpenAI-compatible | `https://api.z.ai/api/coding/paas/v4` | `glm-5` |
+| acp | ACP (JSON-RPC 2.0 stdio) | `claude`, `codex`, `gemini` (binary name) | `claude` |

 ---

@@ -291,7 +292,167 @@ Standard OpenAI-compatible provider targeting the Alibaba Coding API.

 ---

-## 10. Agent Evaluators (Hook System)
+## 10. ACP Provider (Agent Client Protocol)
+
+The ACP provider enables GoClaw to orchestrate external coding agents (Claude Code, Codex CLI, Gemini CLI, or any ACP-compatible agent) as subprocesses via JSON-RPC 2.0 over stdio. This allows delegating complex code generation tasks to specialized agents while maintaining GoClaw's unified interface.
+
+### Architecture Overview
+
+```mermaid
+flowchart TD
+    AL["Agent Loop"] -->|Chat / ChatStream| ACP["ACPProvider"]
+    ACP --> PP["ProcessPool"]
+    PP -->|spawn| PROC["Subprocess<br/>json-rpc 2.0 stdio"]
+    PROC -->|initialize| AGT["Agent<br/>(Claude Code, Codex, etc.)"]
+
+    AGT -->|fs/readTextFile| TB["ToolBridge"]
+    AGT -->|fs/writeTextFile| TB
+    AGT -->|terminal/*| TB
+    AGT -->|permission/request| TB
+
+    TB -->|enforce| SB["Workspace Sandbox"]
+    TB -->|check| DEN["Deny Patterns"]
+    TB -->|handle| PERM["Permission Mode"]
+```
+
+### Configuration
+
+ACPConfig struct fields:
+
+```go
+type ACPConfig struct {
+	Binary   string   // agent binary name or path (e.g. "claude", "codex")
+	Args     []string // extra spawn args
+	Model    string   // default model/agent name (e.g. "claude")
+	WorkDir  string   // base workspace dir
+	IdleTTL  string   // process idle TTL (e.g. "5m")
+	PermMode string   // "approve-all" (default), "approve-reads", "deny-all"
+}
+```
+
+Example config.json:
+
+```json5
+{
+  "providers": {
+    "acp": {
+      "binary": "claude",
+      "args": ["--profile", "goclaw"],
+      "model": "claude",
+      "work_dir": "/tmp/workspace",
+      "idle_ttl": "5m",
+      "perm_mode": "approve-all"
+    }
+  }
+}
+```
+
+Database-based provider registration:
+
+- `provider_type = "acp"`
+- `api_base = "claude"` (binary name)
+- `settings = { "args": [...], "idle_ttl": "5m", "perm_mode": "approve-all", "work_dir": "..." }`
+
+### Session Management
+
+#### ProcessPool
+
+Manages subprocess lifecycle with idle TTL reaping and crash recovery:
+
+1. **GetOrSpawn** — Retrieve existing session or spawn new subprocess
+2. **Idle TTL** — Reap idle processes after configured duration (default 5m)
+3. **Crash Recovery** — Restart failed subprocesses transparently
+
+#### ToolBridge
+
+Handles agent → client requests for filesystem and terminal operations:
+
+- **fs/readTextFile** — Read file within workspace sandbox
+- **fs/writeTextFile** — Write file within workspace sandbox
+- **terminal/createTerminal** — Spawn terminal subprocess
+- **terminal/terminalOutput** — Fetch terminal output + exit status
+- **terminal/waitForTerminalExit** — Block until terminal exit
+- **terminal/releaseTerminal** — Clean up terminal resources
+- **terminal/killTerminal** — Force-terminate terminal
+- **permission/request** — Request user approval (approve-all, approve-reads, deny-all)
+
+### Content Handling
+
+ACP messages use `ContentBlock` with three types:
+
+```go
+type ContentBlock struct {
+	Type     string // "text", "image", "audio"
+	Text     string // text content
+	Data     string // base64 for image/audio
+	MimeType string // e.g., "image/png", "audio/wav"
+}
+```
+
+Request extraction:
+
+1. Extract system prompt + user message from GoClaw `ChatRequest.Messages`
+2. Prepend system prompt to first user message (ACP agents lack separate system API)
+3. Attach images as separate blocks
+
+Response collection:
+
+1. Accumulate `SessionUpdate` notifications during prompt execution
+2. Collect text blocks into response content
+3. Return finish reason mapped from `stopReason` ("maxContextLength" → "length", others → "stop")
+
+### Security & Sandboxing
+
+#### Workspace Isolation
+
+All file operations are scoped to `WorkDir`. Attempts to escape (e.g., `../../../etc/passwd`) are rejected.
+
+#### Deny Patterns
+
+Regex patterns (from config or tools policy) prevent access to sensitive paths:
+
+```
+[
+  "^/etc/",
+  "^\\.env",
+  "^secret",
+  "^[Cc]redentials"
+]
+```
+
+Each agent request is validated against deny patterns before execution.
+
+#### Permission Modes
+
+| Mode | Behavior |
+|------|----------|
+| `approve-all` | All requests approved (default) |
+| `approve-reads` | Read-only; filesystem writes denied |
+| `deny-all` | All requests denied |
+
+### Session Sequencing
+
+Per-session requests are serialized via `sessionMu` mutex to prevent concurrent tool access that could corrupt file state:
+
+```go
+unlock := p.lockSession(sessionKey)
+defer unlock()
+// ... execute Chat or ChatStream with guaranteed serial access
+```
+
+### Streaming vs Non-Streaming
+
+#### Chat (Non-Streaming)
+
+Returns complete response after agent execution finishes. Collects all text blocks and returns single `ChatResponse`.
+
+#### ChatStream
+
+Emits `StreamChunk` for each text delta via callback. Supports context cancellation by sending `session/cancel` notification. Returns combined response when complete.
+
+---
+
+## 11. Agent Evaluators (Hook System)

 Agent evaluators in the quality gate / hook system (see [03-tools-system.md](./03-tools-system.md)) use the same provider resolution as normal agent runs. When a quality gate is configured with `"type": "agent"`, the hook engine delegates to the specified reviewer agent, which resolves its own provider through the standard provider registry. No separate provider configuration is needed for evaluator agents.

@@ -307,6 +468,13 @@ Agent evaluators in the quality gate / hook system (see [03-tools-system.md](./0
 | `internal/providers/retry.go` | RetryDo[T] generic function, RetryConfig, IsRetryableError, backoff computation |
 | `internal/providers/schema_cleaner.go` | CleanSchemaForProvider, CleanToolSchemas, recursive schema field removal |
 | `internal/providers/dashscope.go` | DashScope provider: thinking budget, tools+streaming fallback |
+| `internal/providers/acp_provider.go` | ACPProvider implementation: orchestrates ACP agents as subprocesses |
+| `internal/providers/acp/types.go` | ACP protocol types: InitializeRequest, SessionUpdate, ContentBlock, etc. |
+| `internal/providers/acp/process.go` | ProcessPool: subprocess lifecycle, idle TTL reaping, crash recovery |
+| `internal/providers/acp/jsonrpc.go` | JSON-RPC 2.0 request/response marshaling over stdio |
+| `internal/providers/acp/tool_bridge.go` | ToolBridge: handles fs and terminal requests, workspace sandboxing |
+| `internal/providers/acp/terminal.go` | Terminal lifecycle: create, output, exit, release, kill |
+| `internal/providers/acp/session.go` | Session state tracking per ACP agent |
 | `cmd/gateway_providers.go` | Provider registration from config and database during gateway startup |

 ---
@@ -0,0 +1,63 @@
+# Changelog
+
+All notable changes to GoClaw Gateway are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+---
+
+## [Unreleased]
+
+### Added
+
+#### ACP Provider (Agent Client Protocol)
+- **New provider**: ACP provider enables orchestration of external coding agents (Claude Code, Codex CLI, Gemini CLI) as JSON-RPC 2.0 subprocesses over stdio
+- **ProcessPool**: Manages subprocess lifecycle with idle TTL reaping and automatic crash recovery
+- **ToolBridge**: Handles agent→client requests for filesystem operations and terminal spawning with workspace sandboxing
+- **Security features**: Workspace isolation, deny pattern matching, configurable permission modes (approve-all, approve-reads, deny-all)
+- **Streaming support**: Both streaming and non-streaming modes supported with context cancellation
+- **Config integration**: New `ACPConfig` struct in configuration with binary, args, model, work_dir, idle_ttl, perm_mode
+- **Database providers**: ACP providers can be registered in `llm_providers` table with encrypted credentials
+- **Files added**:
+  - `internal/providers/acp_provider.go` — ACPProvider implementation
+  - `internal/providers/acp/types.go` — ACP protocol types
+  - `internal/providers/acp/process.go` — Process pool management
+  - `internal/providers/acp/jsonrpc.go` — JSON-RPC 2.0 marshaling
+  - `internal/providers/acp/tool_bridge.go` — Request handling
+  - `internal/providers/acp/terminal.go` — Terminal lifecycle
+  - `internal/providers/acp/session.go` — Session tracking
+
+### Changed
+
+- Updated `02-providers.md` to document ACP provider architecture, configuration, session management, security, and streaming
+- Updated `00-architecture-overview.md` component diagram to include ACP provider
+- Updated Module Map in architecture overview to reference `internal/providers/acp/` package
+
+### Documentation
+
+- Added comprehensive ACP provider documentation with architecture diagrams, configuration examples, security model, and file reference
+- Added `17-changelog.md` for tracking project changes
+
+---
+
+## [Previous Releases]
+
+### v1.0.0 and Earlier
+
+- Initial release of GoClaw Gateway with Anthropic and OpenAI-compatible providers
+- WebSocket RPC v3 protocol and HTTP API
+- PostgreSQL multi-tenant backend with pgvector embeddings
+- Agent loop with think→act→observe cycle
+- Tool system: filesystem, exec, web, memory, browser, MCP bridge, custom tools
+- Channel adapters: Telegram, Discord, Feishu, Zalo, WhatsApp
+- Extended thinking support for Anthropic and select OpenAI models
+- Scheduler with lane-based concurrency control
+- Cron scheduling system
+- Agent teams with task delegation
+- Skills system with hot-reload
+- Tracing and observability with optional OpenTelemetry export
+- Browser automation via Rod
+- Code sandbox with Docker
+- Text-to-speech (OpenAI, ElevenLabs, Edge, MiniMax)
+- i18n support (English, Vietnamese, Chinese)
+- RBAC permission system
+- Device pairing with 8-character codes
+- MCP server integration with stdio, SSE, streamable-HTTP transports