Trivial nits from thieung's review of ba59e7b0:
- writeDocumentContent now closes the file explicitly and propagates
any Close() error so a delayed FS failure (e.g. disk-full at flush)
doesn't return success.
- Renamed slog `site=lexical` to `site=path_traversal` — the lexical
prefix check catches "../" / absolute-path escapes, not symlinks.
Umbrella key `security.vault_symlink_escape` stays.
- Non-master-tenant test now reuses newJSONRequest + WithContext,
dropping the duplicate mustJSON helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses second review on #1174 (thieung):
**B1 (blocking)** — first JSON content write into a fresh non-master
tenant returned 500 because workspace/tenants/{slug}/ doesn't exist
yet and EvalSymlinks fails with ErrNotExist. writeDocumentContent now
MkdirAlls the workspace before resolving (mirrors handleUpload). Added
a regression test that drives a non-master tenant context end-to-end
and asserts the file lands under workspace/tenants/{slug}/.
**I1** — replaced the Lstat→WriteFile sequence (TOCTOU window) with
os.OpenFile(O_WRONLY|O_CREATE|O_TRUNC|O_NOFOLLOW). The Lstat short-fail
stays as defense-in-depth + cleaner error logging. Portability handled
via build-tagged constants in vault_handler_documents_nofollow_{unix,
windows}.go — O_NOFOLLOW is no-op on Windows (desktop edition is
single-user with no untrusted tenants).
**I3** — every symlink/path-escape rejection now emits
slog.Warn("security.vault_symlink_escape", ...) with the rejection
site, resolved path and workspace, matching the storage.go pattern
so oncall can alert on attempted escapes.
**I4** — POST `content` field changed from `string` to `*string` to
match PUT. Nil pointer = "no write" (metadata-only stub); empty string
= "write a 0-byte file + fire event"; non-empty = write bytes. Updated
handler godoc to spell out the symmetry. Switched `werr == os.ErrInvalid`
to `errors.Is` per repo conventions.
**Tests added** — non-master tenant first content write (B1), PUT with
nil content (locks "no change" contract), handler-level ".." path
rejection, POST content:"" writes empty file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses review on #1174:
- writeDocumentContent now resolves symlinks before writing. A symlink
living inside the workspace but pointing outside (e.g. workspace/link
-> /etc) previously passed the lexical prefix check and os.WriteFile
followed it out of bounds. We now EvalSymlinks the workspace root and
the deepest existing ancestor of the target, re-check containment
before any mkdir/write, and refuse a symlink planted at the final
component via Lstat.
- publishDocUpserted starts enrichment progress (enrichProgress.Start)
before publishing EventVaultDocUpserted, matching handleUpload's
ordering so the worker pool can't drain the event before the progress
tracker registers it.
- Add regression tests: symlink dir escape, final-component symlink,
parent escape, empty content, disallowed extension, event-after-upsert
ordering, metadata-only no-write, and upsert-failure orphan behaviour.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously POST /v1/{agents/{id}/}vault/documents and the matching PUT
endpoint declared a body struct without a `content` field, so callers
passing JSON `{"path": ..., "title": ..., "content": "..."}` got back
HTTP 201 with a doc id, but the bytes were silently discarded:
- bindJSON ignored the `content` key (no matching field)
- the document row was inserted with content_hash="" and no file on disk
- the enrichment pipeline (summary, embeddings, links) was never
triggered because no EventVaultDocUpserted was published
The endpoint was effectively a metadata-only create — there was no client
path that combined "register doc + supply bytes" via JSON. /v1/vault/upload
exists, but it requires multipart, flattens paths to basename, and is
inconvenient for programmatic clients that already have UTF-8 text in
memory (CI jobs, agent skills, MCP tools).
This change makes `content` an optional field on both handlers:
- POST: when content is non-empty, validate the file extension against
the same allowedUploadExts whitelist used by /v1/vault/upload, write
bytes to <tenant-workspace>/<path>, compute SHA-256, store it as
content_hash on the doc, and publish EventVaultDocUpserted so the
enrichment worker picks it up — exactly the same downstream
behaviour as /v1/vault/upload.
- PUT: same logic; nil content = no change, non-nil content = rewrite
the file on disk (and an empty string truncates it).
- Omitting `content` preserves the historical metadata-only behaviour
(e.g. for callers that materialise files via filesystem rescan).
- Path validation reuses the existing rules: no `..`, no absolute
paths; an extra abs-path check rejects symlink escapes from the
workspace root.
Two small helpers (writeDocumentContent, publishDocUpserted) factor out
the disk write + event publish so both handlers share the same code,
and so they can be unit-tested in isolation later.
The fix is backwards-compatible: existing callers that don't send
`content` continue to work unchanged. Existing docs created without
content remain metadata-only stubs; a subsequent PUT with `content`
will materialise them.
Verified live on a development instance:
POST → row inserted, file at <workspace>/<path>, SHA-256 matches,
enrichment event published.
PUT → file rewritten, content_hash updated, updated_at refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements 3 coalescing layers to handle rapid multi-attachment inbounds:
- Bus debouncer: delays inbound messages 1s, merges duplicates
- Web chat debouncer: buffers client-side inbound frames for batch RPC
- Telegram album aggregator: collects album members via AfterFunc+Stop timer
Drops media-bypass shortcut (forces 1s media floor). Aggregator enforces:
- AfterFunc+Stop timer discipline with ordered drain on stop
- 2-tuple (album_id, sender) keying for isolation
- Dual DoS caps: max 10 albums per sender, max 100 messages per album
- merged_message_ids dedup seeding across all 3 surfaces
Closes#63
* feat(mcp): filter tools at registration + detect FastMCP session reset with force-reconnect
Two foundational MCP reliability improvements:
(1) Tool allow/deny filtering at BridgeTool registration: Previously the runtime grant-check at execute time surfaced "grant revoked" errors when the LLM called a tool it wasn't allowed to call. Filter upfront in both the per-agent registration path (manager_connect.go) and per-user registration path (loop_mcp_user.go). Adds tool_filter.go with IsToolAllowed + tests. This eliminates the "registered then runtime-denied" loop.
(2) FastMCP session reset detection + force-reconnect: FastMCP/Python mcp servers reject tools/call as "invalid during session initialization" when the upstream session lifecycle resets but our pool still holds the old Mcp-Session-Id. Detector matches three known phrasings (FastMCP, mcp-go, mcp-go transport ErrSessionTerminated) in session_reset.go. On detection, BridgeTool.Execute requests a force-reconnect via atomic CAS dedup so N concurrent failing calls collapse to one reconnect. Health loops skip ping while pending so a server answering ping in "initializing" state cannot clobber connected=true before the fresh Initialize completes. Includes 30s timeout, structured slog telemetry, concurrent CAS dedup test.
Files: tool_filter.go + tool_filter_test.go (new), session_reset.go + session_reset_test.go (new), manager_connect.go (connectServer/connectViaPool signatures + registerBridgeTools/registerPoolBridgeTools filter logic + reconnPending skip), manager.go (connectAndFilter + connectServer call signature changes), loop_mcp_user.go (filter-at-register block + WithForceReconnect wiring), pool.go (reconnPending skip), bridge_tool.go (session reset detection + WithForceReconnect callback).
* fix(mcp): self-heal grant cache + bypass per-user grant for system/empty userID
Two production fixes for "MCP tool: grant revoked" recurring on song-nhi-v2.
(1) System-user bypass in ListAccessible: Registration uses LoadForAgent(ctx, agentID, "") while execute uses IsAllowed(ctx, agentID, "system", ...). The LEFT JOIN on mcp_user_grants could match a stale disabled row keyed user_id='system' and silently filter the server out only at execute. Skip the join entirely for synthetic owner identities (userID="" or "system") so registration and execute see the same set. Applied to both PostgreSQL (mcp_servers_access.go) and SQLite (mcp_servers_access.go).
(2) Grant-checker no-cache on empty allowByServer: grant_checker.loadEntry now skips the cache write when allowByServer is empty. Without this, a single transient empty result pinned permanent denial until a bus invalidate fired. Re-queries until the empty condition clears, then caches normally. Includes TestStoreGrantChecker_EmptyEntryNotCached.
* feat(skills): add workspace-organizing built-in skill
Enforces purpose-based folder convention for agent workspaces with two
modes (flat for ad-hoc work, project for named multi-file work),
per-agent namespacing under shared/<agent_key>/ to prevent collisions
in team workspaces, and pre-write discovery via vault_search /
memory_search / knowledge_graph_search to surface related files and
avoid duplicates. Vault integration documents which folders are
Vault-indexable and how scope mirrors workspace scope.
Closes#71
* docs(changelog): add workspace-organizing skill entry
Refs #71
* fix(ci): base64-encode zuey SSH key secret to dodge libcrypto error
First real deploy_zuey_beta run failed at the new 'Sync zuey ops scripts
to VPS' step with 'Load key: error in libcrypto' / Permission denied,
because GitHub Secrets storage normalized newlines inside the multi-line
PEM block.
Switch the secret to ZUEY_SSH_PRIVATE_KEY_B64 (single-line base64) and
base64 -d it on the runner. Single-line storage is immune to whitespace
mangling. Add a ssh-keygen -y -f pre-flight check that fails fast with
a clear remediation message if either the base64 or the decoded key is
malformed.
Refs: workflow run 26499549166
* fix(ci): create SSH key at 0600 from birth via umask 077
Defense-in-depth from review of PR #79: wrap the base64-decode pipeline
in a subshell that sets umask 077 so the decoded private key file is
created at 0600 directly, instead of relying on a chmod 0600 after the
fact (which left a brief 0644 window).
* fix(exec): surface chdir error instead of misleading fork/exec
Linux Go forkAndExecInChild conflates chdir + execve failures into one
PathError naming only argv0. When agent workspace points to a stale
directory, exec reports "fork/exec /usr/bin/gh: no such file" even though
the binary exists — the real failure is chdir on cmd.Dir.
Two fixes:
- loop_context.go: fall back to l.workspace when per-user MkdirAll fails
(3 sites: user workspace, dispatched team, auto-resolved team)
- shell.go + credentialed_exec.go: preflight cwd via validateExecCwd
before exec.Command, so the error names the directory, not the binary
Regression test TestValidateExecCwd covers empty / existing / missing /
file-instead-of-dir.
* feat(upgrade): auto-normalize stale agent workspace paths
Detects non-portable workspace values left by deployment migrations
(Docker → bare-metal, host path drift) and rewrites them to the current
configured base. Registered as data hook 072_normalize_agent_workspaces;
runs automatically after `migrate up`.
Stale patterns:
- /app/workspace/* (Docker container path persisted on bare-metal)
- ~ prefix (Go doesn't expand tildes; MkdirAll creates literal ~ dir)
Base resolved with gateway precedence: GOCLAW_WORKSPACE env >
config.Agents.Defaults.Workspace > default ~/.goclaw/workspace.
Idempotent (skips rows already at proposed value). Conservative
(leaves absolute non-stale paths untouched as intentional custom config).
ApplySystemConfigs and seedConfigForContext both handle tts.provider,
tts.auto, tts.mode, tts.max_length — but neither carries tts.timeout_ms.
Result: the dashboard /tts page Timeout (ms) input is inert; cfg.Tts.TimeoutMs
stays at config.json default; setupTTS passes 0 to provider constructors;
each provider falls back to its hardcoded default (e.g. edge = 30s).
Two-line fix to mirror the existing pattern for the other tts.* keys.
Duy /zuey/