Trivial nits from thieung's review of ba59e7b0:
- writeDocumentContent now closes the file explicitly and propagates
any Close() error so a delayed FS failure (e.g. disk-full at flush)
doesn't return success.
- Renamed slog `site=lexical` to `site=path_traversal` — the lexical
prefix check catches "../" / absolute-path escapes, not symlinks.
Umbrella key `security.vault_symlink_escape` stays.
- Non-master-tenant test now reuses newJSONRequest + WithContext,
dropping the duplicate mustJSON helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses second review on #1174 (thieung):
**B1 (blocking)** — first JSON content write into a fresh non-master
tenant returned 500 because workspace/tenants/{slug}/ doesn't exist
yet and EvalSymlinks fails with ErrNotExist. writeDocumentContent now
MkdirAlls the workspace before resolving (mirrors handleUpload). Added
a regression test that drives a non-master tenant context end-to-end
and asserts the file lands under workspace/tenants/{slug}/.
**I1** — replaced the Lstat→WriteFile sequence (TOCTOU window) with
os.OpenFile(O_WRONLY|O_CREATE|O_TRUNC|O_NOFOLLOW). The Lstat short-fail
stays as defense-in-depth + cleaner error logging. Portability handled
via build-tagged constants in vault_handler_documents_nofollow_{unix,
windows}.go — O_NOFOLLOW is no-op on Windows (desktop edition is
single-user with no untrusted tenants).
**I3** — every symlink/path-escape rejection now emits
slog.Warn("security.vault_symlink_escape", ...) with the rejection
site, resolved path and workspace, matching the storage.go pattern
so oncall can alert on attempted escapes.
**I4** — POST `content` field changed from `string` to `*string` to
match PUT. Nil pointer = "no write" (metadata-only stub); empty string
= "write a 0-byte file + fire event"; non-empty = write bytes. Updated
handler godoc to spell out the symmetry. Switched `werr == os.ErrInvalid`
to `errors.Is` per repo conventions.
**Tests added** — non-master tenant first content write (B1), PUT with
nil content (locks "no change" contract), handler-level ".." path
rejection, POST content:"" writes empty file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses review on #1174:
- writeDocumentContent now resolves symlinks before writing. A symlink
living inside the workspace but pointing outside (e.g. workspace/link
-> /etc) previously passed the lexical prefix check and os.WriteFile
followed it out of bounds. We now EvalSymlinks the workspace root and
the deepest existing ancestor of the target, re-check containment
before any mkdir/write, and refuse a symlink planted at the final
component via Lstat.
- publishDocUpserted starts enrichment progress (enrichProgress.Start)
before publishing EventVaultDocUpserted, matching handleUpload's
ordering so the worker pool can't drain the event before the progress
tracker registers it.
- Add regression tests: symlink dir escape, final-component symlink,
parent escape, empty content, disallowed extension, event-after-upsert
ordering, metadata-only no-write, and upsert-failure orphan behaviour.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously POST /v1/{agents/{id}/}vault/documents and the matching PUT
endpoint declared a body struct without a `content` field, so callers
passing JSON `{"path": ..., "title": ..., "content": "..."}` got back
HTTP 201 with a doc id, but the bytes were silently discarded:
- bindJSON ignored the `content` key (no matching field)
- the document row was inserted with content_hash="" and no file on disk
- the enrichment pipeline (summary, embeddings, links) was never
triggered because no EventVaultDocUpserted was published
The endpoint was effectively a metadata-only create — there was no client
path that combined "register doc + supply bytes" via JSON. /v1/vault/upload
exists, but it requires multipart, flattens paths to basename, and is
inconvenient for programmatic clients that already have UTF-8 text in
memory (CI jobs, agent skills, MCP tools).
This change makes `content` an optional field on both handlers:
- POST: when content is non-empty, validate the file extension against
the same allowedUploadExts whitelist used by /v1/vault/upload, write
bytes to <tenant-workspace>/<path>, compute SHA-256, store it as
content_hash on the doc, and publish EventVaultDocUpserted so the
enrichment worker picks it up — exactly the same downstream
behaviour as /v1/vault/upload.
- PUT: same logic; nil content = no change, non-nil content = rewrite
the file on disk (and an empty string truncates it).
- Omitting `content` preserves the historical metadata-only behaviour
(e.g. for callers that materialise files via filesystem rescan).
- Path validation reuses the existing rules: no `..`, no absolute
paths; an extra abs-path check rejects symlink escapes from the
workspace root.
Two small helpers (writeDocumentContent, publishDocUpserted) factor out
the disk write + event publish so both handlers share the same code,
and so they can be unit-tested in isolation later.
The fix is backwards-compatible: existing callers that don't send
`content` continue to work unchanged. Existing docs created without
content remain metadata-only stubs; a subsequent PUT with `content`
will materialise them.
Verified live on a development instance:
POST → row inserted, file at <workspace>/<path>, SHA-256 matches,
enrichment event published.
PUT → file rewritten, content_hash updated, updated_at refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements 3 coalescing layers to handle rapid multi-attachment inbounds:
- Bus debouncer: delays inbound messages 1s, merges duplicates
- Web chat debouncer: buffers client-side inbound frames for batch RPC
- Telegram album aggregator: collects album members via AfterFunc+Stop timer
Drops media-bypass shortcut (forces 1s media floor). Aggregator enforces:
- AfterFunc+Stop timer discipline with ordered drain on stop
- 2-tuple (album_id, sender) keying for isolation
- Dual DoS caps: max 10 albums per sender, max 100 messages per album
- merged_message_ids dedup seeding across all 3 surfaces
Closes#63
* feat(mcp): filter tools at registration + detect FastMCP session reset with force-reconnect
Two foundational MCP reliability improvements:
(1) Tool allow/deny filtering at BridgeTool registration: Previously the runtime grant-check at execute time surfaced "grant revoked" errors when the LLM called a tool it wasn't allowed to call. Filter upfront in both the per-agent registration path (manager_connect.go) and per-user registration path (loop_mcp_user.go). Adds tool_filter.go with IsToolAllowed + tests. This eliminates the "registered then runtime-denied" loop.
(2) FastMCP session reset detection + force-reconnect: FastMCP/Python mcp servers reject tools/call as "invalid during session initialization" when the upstream session lifecycle resets but our pool still holds the old Mcp-Session-Id. Detector matches three known phrasings (FastMCP, mcp-go, mcp-go transport ErrSessionTerminated) in session_reset.go. On detection, BridgeTool.Execute requests a force-reconnect via atomic CAS dedup so N concurrent failing calls collapse to one reconnect. Health loops skip ping while pending so a server answering ping in "initializing" state cannot clobber connected=true before the fresh Initialize completes. Includes 30s timeout, structured slog telemetry, concurrent CAS dedup test.
Files: tool_filter.go + tool_filter_test.go (new), session_reset.go + session_reset_test.go (new), manager_connect.go (connectServer/connectViaPool signatures + registerBridgeTools/registerPoolBridgeTools filter logic + reconnPending skip), manager.go (connectAndFilter + connectServer call signature changes), loop_mcp_user.go (filter-at-register block + WithForceReconnect wiring), pool.go (reconnPending skip), bridge_tool.go (session reset detection + WithForceReconnect callback).
* fix(mcp): self-heal grant cache + bypass per-user grant for system/empty userID
Two production fixes for "MCP tool: grant revoked" recurring on song-nhi-v2.
(1) System-user bypass in ListAccessible: Registration uses LoadForAgent(ctx, agentID, "") while execute uses IsAllowed(ctx, agentID, "system", ...). The LEFT JOIN on mcp_user_grants could match a stale disabled row keyed user_id='system' and silently filter the server out only at execute. Skip the join entirely for synthetic owner identities (userID="" or "system") so registration and execute see the same set. Applied to both PostgreSQL (mcp_servers_access.go) and SQLite (mcp_servers_access.go).
(2) Grant-checker no-cache on empty allowByServer: grant_checker.loadEntry now skips the cache write when allowByServer is empty. Without this, a single transient empty result pinned permanent denial until a bus invalidate fired. Re-queries until the empty condition clears, then caches normally. Includes TestStoreGrantChecker_EmptyEntryNotCached.
* feat(skills): add workspace-organizing built-in skill
Enforces purpose-based folder convention for agent workspaces with two
modes (flat for ad-hoc work, project for named multi-file work),
per-agent namespacing under shared/<agent_key>/ to prevent collisions
in team workspaces, and pre-write discovery via vault_search /
memory_search / knowledge_graph_search to surface related files and
avoid duplicates. Vault integration documents which folders are
Vault-indexable and how scope mirrors workspace scope.
Closes#71
* docs(changelog): add workspace-organizing skill entry
Refs #71
* fix(ci): base64-encode zuey SSH key secret to dodge libcrypto error
First real deploy_zuey_beta run failed at the new 'Sync zuey ops scripts
to VPS' step with 'Load key: error in libcrypto' / Permission denied,
because GitHub Secrets storage normalized newlines inside the multi-line
PEM block.
Switch the secret to ZUEY_SSH_PRIVATE_KEY_B64 (single-line base64) and
base64 -d it on the runner. Single-line storage is immune to whitespace
mangling. Add a ssh-keygen -y -f pre-flight check that fails fast with
a clear remediation message if either the base64 or the decoded key is
malformed.
Refs: workflow run 26499549166
* fix(ci): create SSH key at 0600 from birth via umask 077
Defense-in-depth from review of PR #79: wrap the base64-decode pipeline
in a subshell that sets umask 077 so the decoded private key file is
created at 0600 directly, instead of relying on a chmod 0600 after the
fact (which left a brief 0644 window).
* fix(exec): surface chdir error instead of misleading fork/exec
Linux Go forkAndExecInChild conflates chdir + execve failures into one
PathError naming only argv0. When agent workspace points to a stale
directory, exec reports "fork/exec /usr/bin/gh: no such file" even though
the binary exists — the real failure is chdir on cmd.Dir.
Two fixes:
- loop_context.go: fall back to l.workspace when per-user MkdirAll fails
(3 sites: user workspace, dispatched team, auto-resolved team)
- shell.go + credentialed_exec.go: preflight cwd via validateExecCwd
before exec.Command, so the error names the directory, not the binary
Regression test TestValidateExecCwd covers empty / existing / missing /
file-instead-of-dir.
* feat(upgrade): auto-normalize stale agent workspace paths
Detects non-portable workspace values left by deployment migrations
(Docker → bare-metal, host path drift) and rewrites them to the current
configured base. Registered as data hook 072_normalize_agent_workspaces;
runs automatically after `migrate up`.
Stale patterns:
- /app/workspace/* (Docker container path persisted on bare-metal)
- ~ prefix (Go doesn't expand tildes; MkdirAll creates literal ~ dir)
Base resolved with gateway precedence: GOCLAW_WORKSPACE env >
config.Agents.Defaults.Workspace > default ~/.goclaw/workspace.
Idempotent (skips rows already at proposed value). Conservative
(leaves absolute non-stale paths untouched as intentional custom config).
Resolve the PR #8/#9/#10 stack on current dev, including Bitrix24 install callback hardening, migration renumbering, duplicate-domain fail-closed routing, UI textarea/mobile cleanup, and review hardening.
* feat(ci): deploy zuey beta releases
Add automatic zuey VPS deployment to the dev beta release workflow after prerelease assets are published. Publish beta checksums and make the host upgrade script tolerate beta asset naming and checksum fallback.
* fix(deploy): allow active beta release reruns
Treat an existing target release as success only when it is already the active release and still contains the expected binary and migrations. This keeps automated beta deploy reruns from failing after a prior successful deploy while preserving fail-closed behavior for stale or partial release directories.
Thieu Nguyen