- Iterate over all DNS responses instead of just the first IP to prevent
multi-IP SSRF bypass where a blocked IP appears after a public one
- Make resolver injectable via package-level lookupHost variable
- Replace real-DNS test (google.com) with stubbed resolver covering:
single public/private IP, DNS failure, empty results, invalid IP,
multi-IP public→private, private→public, all public, metadata IP,
IPv6 loopback
- Add fmt import to test file for stubbed error messages
- Add validateSelfHostedDomain() with SSRF protection (blocked CIDRs,
hostname resolution, localhost/.local/.localhost rejection)
- Validate port range 1-65535 for self-hosted domains with ports
- Mirror SSRF + port validation in frontend form
- Add comprehensive tests: SSRF blocked IPs, invalid ports, valid public
domains
Split bitrixDomainRegex into bitrixCloudDomainRegex (cloud SaaS domains)
and selfHostedDomainRegex (custom FQDNs). Users can now register portals
with self-hosted Bitrix24 instances like bx.mycompany.com instead of
being restricted to *.bitrix24.{com,eu,...} domains.
* fix(pipeline): preserve pending messages during compaction
When PruneStage or ThinkStage triggers mid-loop compaction,
ReplaceHistory() clears the pending buffer. If the current
iteration's assistant(tool_calls) message is in pending, it gets
discarded. The next iteration then sends tool-result messages
without a preceding tool_calls message, causing DeepSeek/OpenAI
to reject the request with HTTP 400:
Messages with role 'tool' must be a response to a preceding
message with 'tool_calls'
Fix: save pending before ReplaceHistory and restore it after.
Reproduction:
- Agent with 64K context window
- 4-5 consecutive tool calls (e.g. read_file) in one session
- Token count exceeds soft budget during ToolStage
- PruneStage compaction clears pending including assistant(tool_calls)
- Next iteration fails with HTTP 400
* test(pipeline): regression guards for pending preservation during compaction
Add focused unit tests covering both ReplaceHistory call sites that previously
dropped the pending buffer:
- PruneStage mid-loop compaction must preserve the in-flight
assistant(tool_calls) message so tool_calls -> tool_result pairing survives,
preventing the OpenAI/DeepSeek 400 on the next provider call.
- ThinkStage emergency overflow compaction must preserve already-staged pending
messages (e.g. iteration-budget nudges) across the retry.
Both tests fail without the preservation fix and pass with it.
Co-authored-by: ollielin <2137816+ollielin@users.noreply.github.com>
---------
Co-authored-by: Duy /zuey/ <duy@wearetopgroup.com>
Co-authored-by: ollielin <ollielin@tencent.com>
Co-authored-by: Thieu Nguyen <thieunguyen.tech@gmail.com>
Co-authored-by: ollielin <2137816+ollielin@users.noreply.github.com>
Anthropic returns HTTP 400 when temperature is sent to claude-opus-4-6,
claude-sonnet-4-6, and newer 4.x models. Agent summoning and chat both
hardcode temperature; skip the field for affected model IDs.
Co-authored-by: Ojus Save <ojusave@users.noreply.github.com>
* fix(sandbox): avoid shell in FsBridge writes
Replace sh -c with interpolated path by shell-free 'tee -- <path>' argv form,
piping content via stdin. Prevents command injection through filenames
containing shell metacharacters inside the sandbox container.
Co-authored-by: evgyur <evgyur@gmail.com>
* fix(security): fail-closed on pairing DB errors across channels
On IsPaired lookup error, deny instead of granting access. Covers the shared
CheckDMPolicy/CheckGroupPolicy helpers (Slack/Discord/Feishu/WhatsApp/Zalo) and
the four inline Telegram pairing checks.
Co-authored-by: Srini <srinis.k@gmail.com>
* fix(security): harden provider URL validation against SSRF
Enforce scheme check for all provider types; restrict local types (ollama,
claude_cli, acp) to an explicit localhost allowlist instead of skipping checks;
resolve remote hostnames and reject any IP in a private/reserved range via the
shared security.IsBlocked CIDR list (covers loopback, link-local, metadata,
multicast, and unspecified 0.0.0.0/::). Closes the wildcard-DNS bypass and the
local-type escape hatch. Operator opt-in via GOCLAW_ALLOW_PRIVATE_PROVIDER_URLS.
Exports security.IsBlocked as the single source of truth for blocked ranges.
Co-authored-by: Linh Vo Van <linh.vo@e-cq.net>
* feat(pipeline): add fail-closed tool call authorization gate
Gate tool execution against the server-side AllowedTools allowlist built from the
RBAC/tenant-aware filtered tool set. Resolve the tool-call prefix before the
allowlist lookup so prefixed agents are not wrongly blocked, re-check deny on lazy
MCP activation, and expand IsDenied to cover aliased tool names.
Co-authored-by: Huy Doan <tui@pm.me>
* fix(security): expand file-serve deny-list defense-in-depth
Add absolute-path deny prefixes (/home, /Users, /srv, /var/lib, /var/www, /opt)
and an explicit fail-closed log when no file-serving boundary is configured.
Co-authored-by: Linh Vo Van <linh.vo@e-cq.net>
* fix(providers): allow claude cli executable paths
Refs: #1185
---------
Co-authored-by: evgyur <evgyur@gmail.com>
Co-authored-by: Srini <srinis.k@gmail.com>
Co-authored-by: Linh Vo Van <linh.vo@e-cq.net>
Co-authored-by: Huy Doan <tui@pm.me>
Trivial nits from thieung's review of ba59e7b0:
- writeDocumentContent now closes the file explicitly and propagates
any Close() error so a delayed FS failure (e.g. disk-full at flush)
doesn't return success.
- Renamed slog `site=lexical` to `site=path_traversal` — the lexical
prefix check catches "../" / absolute-path escapes, not symlinks.
Umbrella key `security.vault_symlink_escape` stays.
- Non-master-tenant test now reuses newJSONRequest + WithContext,
dropping the duplicate mustJSON helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses second review on #1174 (thieung):
**B1 (blocking)** — first JSON content write into a fresh non-master
tenant returned 500 because workspace/tenants/{slug}/ doesn't exist
yet and EvalSymlinks fails with ErrNotExist. writeDocumentContent now
MkdirAlls the workspace before resolving (mirrors handleUpload). Added
a regression test that drives a non-master tenant context end-to-end
and asserts the file lands under workspace/tenants/{slug}/.
**I1** — replaced the Lstat→WriteFile sequence (TOCTOU window) with
os.OpenFile(O_WRONLY|O_CREATE|O_TRUNC|O_NOFOLLOW). The Lstat short-fail
stays as defense-in-depth + cleaner error logging. Portability handled
via build-tagged constants in vault_handler_documents_nofollow_{unix,
windows}.go — O_NOFOLLOW is no-op on Windows (desktop edition is
single-user with no untrusted tenants).
**I3** — every symlink/path-escape rejection now emits
slog.Warn("security.vault_symlink_escape", ...) with the rejection
site, resolved path and workspace, matching the storage.go pattern
so oncall can alert on attempted escapes.
**I4** — POST `content` field changed from `string` to `*string` to
match PUT. Nil pointer = "no write" (metadata-only stub); empty string
= "write a 0-byte file + fire event"; non-empty = write bytes. Updated
handler godoc to spell out the symmetry. Switched `werr == os.ErrInvalid`
to `errors.Is` per repo conventions.
**Tests added** — non-master tenant first content write (B1), PUT with
nil content (locks "no change" contract), handler-level ".." path
rejection, POST content:"" writes empty file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses review on #1174:
- writeDocumentContent now resolves symlinks before writing. A symlink
living inside the workspace but pointing outside (e.g. workspace/link
-> /etc) previously passed the lexical prefix check and os.WriteFile
followed it out of bounds. We now EvalSymlinks the workspace root and
the deepest existing ancestor of the target, re-check containment
before any mkdir/write, and refuse a symlink planted at the final
component via Lstat.
- publishDocUpserted starts enrichment progress (enrichProgress.Start)
before publishing EventVaultDocUpserted, matching handleUpload's
ordering so the worker pool can't drain the event before the progress
tracker registers it.
- Add regression tests: symlink dir escape, final-component symlink,
parent escape, empty content, disallowed extension, event-after-upsert
ordering, metadata-only no-write, and upsert-failure orphan behaviour.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously POST /v1/{agents/{id}/}vault/documents and the matching PUT
endpoint declared a body struct without a `content` field, so callers
passing JSON `{"path": ..., "title": ..., "content": "..."}` got back
HTTP 201 with a doc id, but the bytes were silently discarded:
- bindJSON ignored the `content` key (no matching field)
- the document row was inserted with content_hash="" and no file on disk
- the enrichment pipeline (summary, embeddings, links) was never
triggered because no EventVaultDocUpserted was published
The endpoint was effectively a metadata-only create — there was no client
path that combined "register doc + supply bytes" via JSON. /v1/vault/upload
exists, but it requires multipart, flattens paths to basename, and is
inconvenient for programmatic clients that already have UTF-8 text in
memory (CI jobs, agent skills, MCP tools).
This change makes `content` an optional field on both handlers:
- POST: when content is non-empty, validate the file extension against
the same allowedUploadExts whitelist used by /v1/vault/upload, write
bytes to <tenant-workspace>/<path>, compute SHA-256, store it as
content_hash on the doc, and publish EventVaultDocUpserted so the
enrichment worker picks it up — exactly the same downstream
behaviour as /v1/vault/upload.
- PUT: same logic; nil content = no change, non-nil content = rewrite
the file on disk (and an empty string truncates it).
- Omitting `content` preserves the historical metadata-only behaviour
(e.g. for callers that materialise files via filesystem rescan).
- Path validation reuses the existing rules: no `..`, no absolute
paths; an extra abs-path check rejects symlink escapes from the
workspace root.
Two small helpers (writeDocumentContent, publishDocUpserted) factor out
the disk write + event publish so both handlers share the same code,
and so they can be unit-tested in isolation later.
The fix is backwards-compatible: existing callers that don't send
`content` continue to work unchanged. Existing docs created without
content remain metadata-only stubs; a subsequent PUT with `content`
will materialise them.
Verified live on a development instance:
POST → row inserted, file at <workspace>/<path>, SHA-256 matches,
enrichment event published.
PUT → file rewritten, content_hash updated, updated_at refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements 3 coalescing layers to handle rapid multi-attachment inbounds:
- Bus debouncer: delays inbound messages 1s, merges duplicates
- Web chat debouncer: buffers client-side inbound frames for batch RPC
- Telegram album aggregator: collects album members via AfterFunc+Stop timer
Drops media-bypass shortcut (forces 1s media floor). Aggregator enforces:
- AfterFunc+Stop timer discipline with ordered drain on stop
- 2-tuple (album_id, sender) keying for isolation
- Dual DoS caps: max 10 albums per sender, max 100 messages per album
- merged_message_ids dedup seeding across all 3 surfaces
Closes#63
* feat(mcp): filter tools at registration + detect FastMCP session reset with force-reconnect
Two foundational MCP reliability improvements:
(1) Tool allow/deny filtering at BridgeTool registration: Previously the runtime grant-check at execute time surfaced "grant revoked" errors when the LLM called a tool it wasn't allowed to call. Filter upfront in both the per-agent registration path (manager_connect.go) and per-user registration path (loop_mcp_user.go). Adds tool_filter.go with IsToolAllowed + tests. This eliminates the "registered then runtime-denied" loop.
(2) FastMCP session reset detection + force-reconnect: FastMCP/Python mcp servers reject tools/call as "invalid during session initialization" when the upstream session lifecycle resets but our pool still holds the old Mcp-Session-Id. Detector matches three known phrasings (FastMCP, mcp-go, mcp-go transport ErrSessionTerminated) in session_reset.go. On detection, BridgeTool.Execute requests a force-reconnect via atomic CAS dedup so N concurrent failing calls collapse to one reconnect. Health loops skip ping while pending so a server answering ping in "initializing" state cannot clobber connected=true before the fresh Initialize completes. Includes 30s timeout, structured slog telemetry, concurrent CAS dedup test.
Files: tool_filter.go + tool_filter_test.go (new), session_reset.go + session_reset_test.go (new), manager_connect.go (connectServer/connectViaPool signatures + registerBridgeTools/registerPoolBridgeTools filter logic + reconnPending skip), manager.go (connectAndFilter + connectServer call signature changes), loop_mcp_user.go (filter-at-register block + WithForceReconnect wiring), pool.go (reconnPending skip), bridge_tool.go (session reset detection + WithForceReconnect callback).
* fix(mcp): self-heal grant cache + bypass per-user grant for system/empty userID
Two production fixes for "MCP tool: grant revoked" recurring on song-nhi-v2.
(1) System-user bypass in ListAccessible: Registration uses LoadForAgent(ctx, agentID, "") while execute uses IsAllowed(ctx, agentID, "system", ...). The LEFT JOIN on mcp_user_grants could match a stale disabled row keyed user_id='system' and silently filter the server out only at execute. Skip the join entirely for synthetic owner identities (userID="" or "system") so registration and execute see the same set. Applied to both PostgreSQL (mcp_servers_access.go) and SQLite (mcp_servers_access.go).
(2) Grant-checker no-cache on empty allowByServer: grant_checker.loadEntry now skips the cache write when allowByServer is empty. Without this, a single transient empty result pinned permanent denial until a bus invalidate fired. Re-queries until the empty condition clears, then caches normally. Includes TestStoreGrantChecker_EmptyEntryNotCached.
* feat(skills): add workspace-organizing built-in skill
Enforces purpose-based folder convention for agent workspaces with two
modes (flat for ad-hoc work, project for named multi-file work),
per-agent namespacing under shared/<agent_key>/ to prevent collisions
in team workspaces, and pre-write discovery via vault_search /
memory_search / knowledge_graph_search to surface related files and
avoid duplicates. Vault integration documents which folders are
Vault-indexable and how scope mirrors workspace scope.
Closes#71
* docs(changelog): add workspace-organizing skill entry
Refs #71
* fix(ci): base64-encode zuey SSH key secret to dodge libcrypto error
First real deploy_zuey_beta run failed at the new 'Sync zuey ops scripts
to VPS' step with 'Load key: error in libcrypto' / Permission denied,
because GitHub Secrets storage normalized newlines inside the multi-line
PEM block.
Switch the secret to ZUEY_SSH_PRIVATE_KEY_B64 (single-line base64) and
base64 -d it on the runner. Single-line storage is immune to whitespace
mangling. Add a ssh-keygen -y -f pre-flight check that fails fast with
a clear remediation message if either the base64 or the decoded key is
malformed.
Refs: workflow run 26499549166
* fix(ci): create SSH key at 0600 from birth via umask 077
Defense-in-depth from review of PR #79: wrap the base64-decode pipeline
in a subshell that sets umask 077 so the decoded private key file is
created at 0600 directly, instead of relying on a chmod 0600 after the
fact (which left a brief 0644 window).
* fix(exec): surface chdir error instead of misleading fork/exec
Linux Go forkAndExecInChild conflates chdir + execve failures into one
PathError naming only argv0. When agent workspace points to a stale
directory, exec reports "fork/exec /usr/bin/gh: no such file" even though
the binary exists — the real failure is chdir on cmd.Dir.
Two fixes:
- loop_context.go: fall back to l.workspace when per-user MkdirAll fails
(3 sites: user workspace, dispatched team, auto-resolved team)
- shell.go + credentialed_exec.go: preflight cwd via validateExecCwd
before exec.Command, so the error names the directory, not the binary
Regression test TestValidateExecCwd covers empty / existing / missing /
file-instead-of-dir.
* feat(upgrade): auto-normalize stale agent workspace paths
Detects non-portable workspace values left by deployment migrations
(Docker → bare-metal, host path drift) and rewrites them to the current
configured base. Registered as data hook 072_normalize_agent_workspaces;
runs automatically after `migrate up`.
Stale patterns:
- /app/workspace/* (Docker container path persisted on bare-metal)
- ~ prefix (Go doesn't expand tildes; MkdirAll creates literal ~ dir)
Base resolved with gateway precedence: GOCLAW_WORKSPACE env >
config.Agents.Defaults.Workspace > default ~/.goclaw/workspace.
Idempotent (skips rows already at proposed value). Conservative
(leaves absolute non-stale paths untouched as intentional custom config).
Thành Nguyễn