mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-14 00:16:38 +00:00
fix(collaboration): harden delegated task isolation (#1486)
* feat(collaboration): isolate delegated artifacts and child runs Isolate delegated inputs and outputs behind secure artifact exchange lifecycles. Scope Agent Link tasks by tenant and root agent, and enforce delegation spawn-tree boundaries. Add process-wide child-run admission and preserve logical media paths across native, MCP, and sandbox execution. * fix(collaboration): harden delegated task isolation Enforce tenant and root-agent task scope across migrations and stores. Add exactly-once async completion delivery, delegated sandbox boundaries, and confined artifact and media recovery across runtime surfaces. * fix(collaboration): recover interrupted async tasks * fix(collaboration): normalize persisted child-run status --------- Co-authored-by: ntduc <ntduc@cpp.ai.vn>
This commit is contained in:
1 parent
496b7ffce6
commit
bb7712a9ff
186 files changed
+18858
-1233
No files matched your search
@@ -115,7 +115,7 @@ flowchart TD
|
||||
| `internal/skills/` | SKILL.md loader (5-tier hierarchy) + BM25 search + hot-reload via fsnotify |
|
||||
| `internal/channels/` | Channel manager + adapters: Telegram (forum topics, STT, bot commands), Feishu/Lark (streaming cards, media), Zalo OA, Zalo Personal, Discord, WhatsApp, Slack |
|
||||
| `internal/mcp/` | MCP server bridge (stdio, SSE, streamable-HTTP transports) |
|
||||
| `internal/scheduler/` | Lane-based concurrency control (main, subagent, cron, team lanes) with per-session serialization. Per-edition rate limits (`MaxSubagentConcurrent`, `MaxSubagentDepth`) with tenant-scoped concurrency |
|
||||
| `internal/scheduler/` | Lane-based concurrency control (main, subagent, cron, team lanes) with per-session serialization. Self-spawn and Agent Link callbacks additionally use process-wide child-run admission with per-root limits |
|
||||
| `internal/memory/` | Memory system (pgvector hybrid search) |
|
||||
| `internal/subagent/` | Subagent lifecycle: spawn, roster, task persistence (subagent_tasks table), announce queue (producer-consumer), auto-retry, per-edition rate limiting |
|
||||
| `internal/permissions/` | RBAC policy engine (admin, operator, viewer roles) |
|
||||
@@ -192,7 +192,7 @@ flowchart TD
|
||||
- **SSRF hardening (HTTPHandler)**: Caller supplies net.Dialer pinning resolved IP, blocking loopback/link-local/private ranges; no HTTP redirects (CheckRedirect returns ErrUseLastResponse)
|
||||
- **Auth header encryption**: `Authorization` + other sensitive fields in cfg.Config["headers"] encrypted at rest via AES-256-GCM; decrypted only at HTTP send-time
|
||||
- **Audit logging**: All hook invocations logged to `hook_executions` table (encrypted, PII-redacted) with dedup_key for idempotency
|
||||
- **Loop-depth guard (M5)**: SubagentStart checks recursion depth; max 3 levels prevents infinite delegation chains
|
||||
- **Hook recursion guard (M5)**: Hook dispatch tracks recursive hook re-entry and rejects it after 3 levels. Agent Link delegation uses separate admission, concurrency, and timeout safeguards.
|
||||
- **Circuit breaker**: Auto-disables hook after 3 consecutive failures in recent window (C4 mitigation)
|
||||
|
||||
### Pipeline Integration
|
||||
|
||||
@@ -111,7 +111,7 @@ Agents support three orchestration modes that determine which inter-agent tools
|
||||
- **Use case**: Agent with linked delegate targets
|
||||
- **Tools available**: `spawn`, `delegate` (dispatch to linked agents)
|
||||
- **Tools hidden**: `team_tasks`
|
||||
- **Resolution**: When `agent_links` table has rows with source = this agent
|
||||
- **Resolution**: When `agent_links` grants this agent an effective outbound delegation edge
|
||||
|
||||
### ModeTeam
|
||||
- **Use case**: Agent in a team (multiple agents collaborating)
|
||||
@@ -121,7 +121,9 @@ Agents support three orchestration modes that determine which inter-agent tools
|
||||
|
||||
**Mode Resolution Priority**: Team > Delegate > Spawn
|
||||
|
||||
The system prompt includes relevant details for each mode (delegate targets, team context, shared workspace paths).
|
||||
The system prompt includes relevant details for each mode. Agent Team runs use
|
||||
their existing shared workspace. Agent Link runs receive only delegation-scoped
|
||||
`inputs/` and `outputs/` aliases.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+54
-3
@@ -144,7 +144,7 @@ Memory layers: L1 (`memory_search`) returns ranked abstracts; L2 (`memory_expand
|
||||
|
||||
| Tool | Description |
|
||||
|---|---|
|
||||
| `delegate` | Inter-agent task delegation via agent_links (async/sync modes with timeout) |
|
||||
| `delegate` | Inter-agent task delegation via `agent_links` (async/sync, optional structured input files, isolated output publication) |
|
||||
|
||||
### Teams (`group:team`)
|
||||
|
||||
@@ -170,6 +170,19 @@ Memory layers: L1 (`memory_search`) returns ranked abstracts; L2 (`memory_expand
|
||||
| `read_document` | Extract and analyze documents (PDF, DOCX, images). Supports local-first extraction via pdftotext/pandoc before falling back to cloud vision (opt-in via config) |
|
||||
| `read_video` | Analyze/transcribe video content |
|
||||
|
||||
Image attachments keep absolute paths only in internal `MediaRef` storage.
|
||||
Model-visible `<media:image>` tags expose an exact media ID plus a logical path
|
||||
such as `.uploads/photo.jpg` or delegated `inputs/photo.jpg`. Image tools resolve
|
||||
those values against the active workspace; they never infer another agent's
|
||||
absolute workspace path. Unknown image IDs fail instead of falling back to an
|
||||
unrelated image. Exact `media_id` lookup is an in-process convenience; Claude
|
||||
CLI/MCP uses the workspace-confined logical `path` under signed bridge context.
|
||||
|
||||
For Agent Link artifact runs, Claude CLI MCP requests carry a signed delegation
|
||||
ID and staged-input root. Bridge tools therefore apply the same read-only
|
||||
`inputs/` policy as in-process tools, and generated media remains inside
|
||||
`outputs/` until the delegation manifest is validated and atomically published.
|
||||
|
||||
### Skills & Content
|
||||
|
||||
| Tool | Description |
|
||||
@@ -613,13 +626,51 @@ stateDiagram-v2
|
||||
|
||||
| Constraint | Default |
|
||||
|---|---|
|
||||
| Max concurrent | 8 (across all parents) |
|
||||
| Max concurrent | 20 executing descendants per root agent |
|
||||
| Max spawn depth | 1 |
|
||||
| Max children per agent | 5 |
|
||||
| Archive after | 60 min |
|
||||
| Max iterations | 20 per subagent |
|
||||
|
||||
**Actions:** `spawn` (async, returns immediately), `run` (sync, blocks until done), `list`, `cancel` (by ID / `"all"` / `"last"`), `steer` (cancel + respawn with new message).
|
||||
`maxSpawnDepth` applies only to an agent's own subagent tree. Agent Link
|
||||
delegation starts a new tree whose spawn depth, concurrency, fanout, retry, and
|
||||
model settings are resolved from the target agent.
|
||||
|
||||
**Actions:** `spawn(action="spawn", mode="async"|"sync")`, `get`, `list`,
|
||||
`cancel` (by ID / `"all"` / `"last"`), `steer` (cancel + respawn with new
|
||||
message), and `wait`.
|
||||
|
||||
An accepted async spawn returns its short runtime `task_id` plus a durable
|
||||
`completion_id`. Terminal status, the full text result, and workspace-safe
|
||||
logical media descriptors are written to `subagent_tasks` before the parent
|
||||
announcement is attempted. Announcement delivery retries a bounded number of
|
||||
times; if the inbound queue is still full or the gateway restarts after
|
||||
terminal persistence, the owning root agent can recover the result and logical
|
||||
file paths with:
|
||||
|
||||
```text
|
||||
spawn(action="get", completion_id="<uuid>")
|
||||
```
|
||||
|
||||
The lookup is scoped by tenant and immutable root-agent UUID. A same-key agent
|
||||
created later, another agent in the tenant, and delegation completion rows
|
||||
cannot satisfy the lookup.
|
||||
|
||||
Terminal persistence uses per-attempt database deadlines and a longer bounded
|
||||
retry window than announcement delivery. If the database remains unavailable
|
||||
for the entire window, GoClaw does not falsely mark the announcement as
|
||||
delivered; a live announcement is still attempted. On the next startup that
|
||||
can reach the database, every non-terminal completion row left by the previous
|
||||
process is marked `failed` with an interruption reason, so it cannot remain
|
||||
`queued` or `running` forever. The single-process gateway retries this
|
||||
reconciliation before accepting traffic when the database is temporarily
|
||||
unavailable. Graceful gateway shutdown drains this completion lifecycle before
|
||||
provider and database teardown.
|
||||
|
||||
Self-spawn and Agent Link callbacks also share a process safety cap (Standard:
|
||||
32; Lite: 2) and a bounded pending queue of 128. Inside an Agent Link artifact
|
||||
run, async `spawn` and async nested `delegate` are rejected; configured
|
||||
synchronous descendants complete before the outer artifact is published.
|
||||
|
||||
Subagents share the same `SecureCLIStore` as their parent — the credentialed binary gate cannot be bypassed by delegating exec to a child.
|
||||
|
||||
|
||||
+9
-3
@@ -461,13 +461,19 @@ Browser pairing allows web UI clients to authenticate without full admin credent
|
||||
|
||||
## 12. Delegation Security
|
||||
|
||||
Agent delegation is protected through delegation history tracking and concurrency controls.
|
||||
Agent delegation is protected through isolated artifact exchange, directional
|
||||
link permission checks, and bounded child-run admission.
|
||||
|
||||
| Control | Scope | Description |
|
||||
|---------|-------|-------------|
|
||||
| Per-agent load cap | B (all sources) | `other_config.max_delegation_load` limits total concurrent delegations targeting B |
|
||||
| Artifact boundary | One Agent Link run | Staged inputs are read-only; the delegatee writes only to ephemeral outputs that are validated before atomic publish |
|
||||
| Per-root limit | One root agent's self-spawn tree | Agent `subagents.maxConcurrent`, default 20 |
|
||||
| Process safety cap | All self-spawn and Agent Link callbacks | Standard 32 / Lite 2, with at most 128 independent pending chains |
|
||||
| Nested lifetime | Agent Link artifact run | Async descendants are rejected; sync descendants complete before publication |
|
||||
| Durable async result | Source/root agent UUID within one tenant | Terminal text and logical media paths use deadline-bounded persistence retries before announcement; single-process startup retries pre-traffic recovery that marks previous-process non-terminal rows failed, and successful terminal writes remain explicitly retrievable by completion UUID |
|
||||
|
||||
When concurrency limits are hit, the error message is written for LLM reasoning: *"Agent at capacity (5/5). Try a different agent or handle it yourself."*
|
||||
`agent_links.max_concurrent` is retained as compatibility metadata and is not
|
||||
currently enforced by runtime admission.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+52
-6
@@ -579,10 +579,17 @@ Each delegation link (lead→member) has its own settings:
|
||||
|
||||
| Layer | Scope | Default |
|
||||
|-------|-------|---------|
|
||||
| Per-link | Simultaneous delegations from lead to a specific member | 3 |
|
||||
| Per-agent | Total concurrent delegations targeting any single member | 5 |
|
||||
| Per-root agent | Executing self-spawn descendants across the full tree | 20 |
|
||||
| Process safety cap | Executing self-spawn and Agent Link callbacks | Standard 32 / Lite 2 |
|
||||
| Pending admission | Independent child-run chains waiting for capacity | 128 |
|
||||
|
||||
When limits are hit, the error message is written for LLM reasoning: "Agent at capacity (5/5). Try a different agent or handle it yourself."
|
||||
Each Agent Link delegation begins a new target-owned spawn tree. The source
|
||||
agent's `maxSpawnDepth` does not limit the target agent; any subagents created by
|
||||
the target use the target's own subagent settings.
|
||||
|
||||
`agent_links.max_concurrent` remains serialized compatibility metadata and is
|
||||
not currently an admission limit. Agent Team task scheduling keeps its existing
|
||||
lane/workspace contract.
|
||||
|
||||
---
|
||||
|
||||
@@ -593,7 +600,7 @@ Agent links define directed delegation relationships between agents, separate fr
|
||||
- **Target agent** (can receive delegations from)
|
||||
- **Direction** (outbound from source, inbound to target)
|
||||
- **Team context** (optional `team_id` if created by team setup)
|
||||
- **Concurrency limit** (max simultaneous delegations for this link)
|
||||
- **Reserved concurrency metadata** (`max_concurrent`; retained for compatibility, not currently enforced)
|
||||
|
||||
**Team-created links** are automatically created when a team is set up (lead → each member). Links remain even if the team is deleted or members are removed, allowing manual cleanup.
|
||||
|
||||
@@ -601,9 +608,48 @@ Agent links define directed delegation relationships between agents, separate fr
|
||||
|
||||
## 12. Delegation Context
|
||||
|
||||
### SenderID Clearing
|
||||
### Authorization and artifact scope
|
||||
|
||||
In sync delegations, the delegate agent's context has the `senderID` cleared. This is critical because delegations are system-initiated — the delegate should not inherit the caller's group writer permissions, which would incorrectly deny file writes. Each delegate agent has its own writer list.
|
||||
Agent Link dispatch preserves the caller's authorization scope for policy
|
||||
checks but attenuates filesystem authority. The delegatee receives staged
|
||||
read-only copies through `inputs/` and writes only to `outputs/`. After the run,
|
||||
regular-file, path, symlink, count, and size validation completes before an
|
||||
atomic publication to the caller's `.delegations/<delegation-id>/` directory.
|
||||
Agent Team shared-workspace behavior is unchanged.
|
||||
|
||||
Successful publications persist with `manifest.json` and logical `outputs/...`
|
||||
entries. Failed/cancelled exchanges are retained for 60 minutes for diagnostics
|
||||
and removed by a restart-safe janitor whose per-sweep item and filesystem-entry
|
||||
work are both bounded. Traces correlate by
|
||||
`delegation_id` and record only logical paths, sizes, hashes, status, and
|
||||
timestamps—never physical exchange or caller host paths.
|
||||
|
||||
Async Agent Link dispatch also creates a durable completion row before the
|
||||
child-run ticket is activated. The full terminal result and logical published
|
||||
artifact paths are persisted before the automatic parent announcement. If
|
||||
bounded announcement retries cannot enter the inbound queue, the source agent
|
||||
can retrieve the result after the run—or after a gateway restart—with:
|
||||
|
||||
```text
|
||||
delegate(action="get", delegation_id="<uuid>")
|
||||
```
|
||||
|
||||
Retrieval is scoped by tenant and the immutable UUID of the source agent. It
|
||||
does not grant the target agent, another linked source, or a later same-key
|
||||
agent access to the result. Delegation completion rows are excluded from
|
||||
`/subagents` and self-clone roster queries.
|
||||
|
||||
Terminal database writes use per-attempt deadlines plus an extended bounded
|
||||
retry window. If the database remains unavailable for that whole window, the
|
||||
live announcement is still attempted but is not recorded as durably delivered;
|
||||
a crash during the same outage may prevent the terminal result from being
|
||||
recorded. On the next startup that can reach the database, every non-terminal
|
||||
completion row from the previous process is marked `failed` with an interruption
|
||||
reason instead of remaining `queued` or `running` forever. The gateway retries
|
||||
that reconciliation before accepting traffic if the database is temporarily
|
||||
unavailable. The current runtime assumes one active gateway process. Graceful
|
||||
shutdown waits for accepted completion persistence before tearing down runtime
|
||||
dependencies.
|
||||
|
||||
### Trace Linking
|
||||
|
||||
|
||||
@@ -166,13 +166,20 @@ Uploaded files (from web chat, Telegram, Discord, etc.) are persisted to:
|
||||
|
||||
Uploads without a usable original filename fall back to `{uuid}.{ext}`. Legacy media refs may still resolve from `.media/{sessionHash}/{uuid}.{ext}`.
|
||||
|
||||
The `enrichDocumentPaths()` function injects the full path into `<media:document>` tags:
|
||||
The `enrichDocumentPaths()` function injects the exact media ID and a logical
|
||||
path relative to the active agent workspace into `<media:document>` tags:
|
||||
|
||||
```
|
||||
<media:document name="report.pdf" path="/app/workspace/.uploads/report-a1b2c3d4.pdf">
|
||||
<media:document name="report.pdf" id="..." path=".uploads/report-a1b2c3d4.pdf">
|
||||
```
|
||||
|
||||
Agents can read these files directly via exec — no copy to `/tmp` needed. For archive uploads such as `.zip`, inspect or extract with commands like `unzip -l "<path>"` or `unzip -q "<path>" -d <output-dir>`.
|
||||
Normal agent runs can read these workspace files directly via `exec` — no copy
|
||||
to `/tmp` is needed. Agent Link delegations instead receive selected files as
|
||||
read-only `inputs/...` paths in an isolated delegation exchange. Their `exec`
|
||||
calls fail closed unless an active sandbox is available, and generated files
|
||||
must be written under `outputs/` for validation and publication back to the
|
||||
caller. For archive uploads such as `.zip`, inspect or extract them within the
|
||||
authorized workspace or delegation paths.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user