An agent's tts_voice_id (e.g. "en-US-GuyNeural" for Edge) was applied
regardless of the selected TTS provider, causing OpenAI and other
providers to reject unknown voice IDs.
- Add internal/audio/voice_compat.go with IsVoiceCompatible,
GetProviderDefaultVoice, and FilterVoiceForProvider helpers
- Update TtsTool.resolveVoiceAndModel to report whether the voice
came from an agent override (voiceFromAgent)
- Apply applyVoiceCompat per-provider branch in Execute so
incompatible agent voices fall back to the provider's default
with a slog.Warn for operator visibility
- Add voice_compat_test.go with table-driven tests for all cases
Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com>
Run pruning before ThinkStage so oversized histories compact before the provider request is built.
Persist V3 last_prompt_tokens with the post-flush persisted message count so post-run compaction calibration has current usage data.
Tests: go test ./internal/pipeline -run "TestNewDefaultPipeline_PrunesBeforeThink|TestFinalizeStage_UpdateMetadataReceivesPersistedMessageCount" -count=1; go test ./internal/agent -run "^$" -count=1
Co-authored-by: GoClaw Operator <operator@goclaw>
Follow-up to #1295. After scoping the Open Channel mention gate by
connector, customer messages on 1-to-1 connectors (Facebook Messenger,
Zalo OA, …) were STILL being silently dropped — by the second
mention-gate stanza further down in handleMessage, which checks the
channel-level c.RequireMention() flag without distinguishing Open
Channel traffic from native group chats (CRM deal/task chats).
For a channel like nguyen-dao-openline configured with RequireMention=true,
the flow was:
1. isOpenChannel block → shouldRequireMentionForOpenline("facebook|...")
returns false (facebook isn't in the whitelist) → no drop.
2. isGroup block (isOpenChannel implies isGroup) → c.RequireMention()=true
AND mentioned=false → DROP.
Net effect: the gate-by-connector decision was overridden one stanza
later, and every Facebook customer turn disappeared.
Fix: scope the drop check inside the isGroup block to native-group chats
only (!isOpenChannel). Open Channel sessions already own their mention
policy in the earlier isOpenChannel block; re-applying the channel-level
RequireMention flag here second-guesses that decision.
Importantly, the text-processing that follows the drop check
(MessageOriginal fallback, stripMention, bxConvertUserMentionsToReadable)
stays unconditional, so an Open Channel STAFF turn that does come in with
"[USER=<botID>]Bot[/USER]" BBCode still gets cleaned up the same way it
always did. The only thing the new guard suppresses is the drop itself.
Verified live: Facebook Messenger customer turn (IS_CONNECTOR=Y,
CHAT_ENTITY_ID=facebook|34|…) now reaches the agent loop and the bot
replies, matching the intent from #1295.
Surface parity: backend-only. No API contract / web UI / CLI change.
Co-authored-by: DangTinh311 <dangtinh31193@gmail.com>
Before: every Open Channel message required a bot @mention before the agent
picked it up — including external-customer turns on 1-to-1 connectors
(Facebook Messenger, Zalo OA, …) where the customer has no syntactic way
to produce a mention. The gate, designed for group-style connectors
(Zalo personal pools multiple customers into one Open Channel chat),
was silently dropping every legitimate customer message on every other
connector.
This change scopes the gate by connector:
- Internal staff (IS_CONNECTOR=N) — STILL gate. Operators can @-mention
and we don't want the bot jumping into operator-to-operator chatter
in the same session.
- External customer (IS_CONNECTOR=Y) on a whitelisted connector
(default: synity_zalo_personal) — STILL gate. Multiple customers
share the chat, so untagged turns would otherwise spam them.
- External customer on any other connector (Facebook, Zalo OA, the long
tail of 1-to-1 connectors, legacy payloads without CHAT_ENTITY_ID) —
do NOT gate. Every message is addressed to the bot by construction.
The whitelist is overridable via BITRIX24_REQUIRE_MENTION_CONNECTORS
(comma-separated connector codes — the leading token of CHAT_ENTITY_ID).
Default keeps synity_zalo_personal gated so a fresh deployment is safe.
Adding a future group-style connector (e.g. a Zalo group flavour) is an
env edit + restart, not a code change.
- mention_gate.go (new):
connectorCodeFromEntityID() parses "synity_zalo_personal|20|grp|960"
into "synity_zalo_personal".
requireMentionConnectorSet() reads the env once (sync.Once cache)
and falls back to the hardcoded default {synity_zalo_personal}.
shouldRequireMentionForOpenline() is the policy matrix.
- mention_gate_test.go (new): pins the parser, the env-override path,
and every row of the policy matrix.
- handle.go: replaces the unconditional Open Channel mention gate with
a call to shouldRequireMentionForOpenline(). Adds chat_entity_id to
the drop log so operators can see which connector tripped the gate.
- handle_test.go:
Update existing "ConnectorWithoutMentionDropped" test to set
ChatEntityID to the whitelisted connector — that's what the gate now
keys on.
Add new "OneToOneConnectorWithoutMentionForwarded" test pinning the
new behavior: facebook (not whitelisted) forwards every customer
message without a mention.
Surface parity: backend-only. No API contract / web UI / CLI change.
The env var follows the same operational model as the other BITRIX24_*
vars in docker-compose.yml; documenting it there is a no-op until the
default needs overriding.
Co-authored-by: DangTinh311 <dangtinh31193@gmail.com>
Without these tags the Bitrix-borne PDF / audio / image arrive as bare text
to the agent loop: the LLM never realizes an attachment exists and degrades
to "no file attached" replies (or worse, hallucinates a CRM document id and
fires crm.documentgenerator at it). The other channels (Telegram, Slack,
Discord) all prepend <media:*> tags before publish via the shared
media.BuildMediaTags helper — Bitrix24 was the lone holdout.
The agent loop's enrichInputMedia REPLACES tags it finds (it does NOT
insert new ones), so adding the tag at the channel boundary is the fix
that aligns Bitrix with the other channels AND unlocks enrich's
read_document / read_audio / read_image routing downstream.
- media_tags.go: mediaFilesToInfos() adapter from bus.MediaFile to
media.MediaInfo + classifyMediaType() MIME prefix → media.Type* mapping.
Unknown MIMEs fall back to TypeDocument (read_document handles generic
binaries safely; a bogus audio tag on a non-audio file would mislead the
LLM). Bitrix ships voice notes as audio/* — they're classified as
TypeAudio (read_audio handles both audio and voice identically).
- handle.go: prepend BuildMediaTags(...) output to text right before
HandleMessageMedia. No-op when no media files.
- media_tags_test.go: pins the MIME map, the conversion, and the
end-to-end tag shape through BuildMediaTags.
Surface parity: backend-only. No API / web UI / CLI impact.
Co-authored-by: DangTinh311 <dangtinh31193@gmail.com>
* fix(feishu): require exact bot_open_id match for mention detection
- Previously, when bot_open_id was empty, ALL mentions were treated as bot mentions
- This caused multiple agents in same group to all respond to any mention
- Now requires bot_open_id to be set AND match exactly for mentionedBot=true
- Fixes issue where CPPAI PM would respond even when other agent was mentioned
* fix(feishu): gate group pairing by target bot
---------
Co-authored-by: ntduc <ntduc@cpp.ai.vn>
WhatsApp LID-format chat IDs contain @ (e.g. 551152861098:5@s.whatsapp.net)
which is invalid in Docker container names, causing sandbox creation to fail
for any WhatsApp-triggered agent session.
Add @ to the sanitizeKey replacer alongside the existing : / . and space
characters. Adds a test case with a realistic WhatsApp LID key.
Fixes#1029
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 백엔드: LocaleKO 상수, IsSupported에 ko 추가, catalog_ko.go 185개 문자열
- 웹 UI: 38개 네임스페이스 JSON 파일 (locales/ko/)
- constants.ts SUPPORTED_LANGUAGES에 ko 추가
- index.ts에 ko 리소스 등록 및 getInitialLanguage에 ko 감지
- en/topbar.json languages에 한국어 옵션 추가
- webui handler: index.html에 no-cache 헤더 추가하여 배포 후 캐시 문제 방지
Both WS pairing (router.go Path 3a) and HTTP X-GoClaw-Sender-Id auth
were pinned to RoleOperator. Now they map tenant_users.role ->
permissions.Role (owner/admin/operator|member/viewer) so a tenant admin
who logs in via pairing actually gets admin in the dashboard. Falls
back to RoleOperator when the user has no membership row (backward
compatible). Tenant is inferred from a single membership when no hint
is provided.
Signed-off-by: imcvampire <anhnq0@gmail.com>
Mirror the existing /addwriter /removewriter /writers UX for the cron
permission type so group admins can grant cron-management access without
elevating to full file_writer. Bootstrap policy: first call in a group
is free (no cron managers yet); subsequent calls require an existing
cron manager OR a file_writer.
- new commands_cron_perm.go: handleCronPermCommand (add/remove),
handleListCronPerm; uses store.ConfigTypeCron with scope
"group:<channel>:<chatID>".
- commands.go: route /addcron, /removecron, /croners.
- commands_pairing.go: register the 3 commands in the bot menu.
* fix(skills): normalize backslash in ZIP entry names for Windows compatibility
ZIP archives created on Windows may use backslash (\) as path separator
instead of forward slash (/). When extracted on Linux, this causes files
like 'scripts\search.py' to be created as a single flat file instead of
the proper 'scripts/search.py' directory structure.
The ZIP specification (PKZIP APPNOTE) requires forward slashes, but many
Windows tools produce non-conforming archives.
This adds a strings.ReplaceAll backslash normalization step during skill
ZIP extraction to handle this edge case.
* test(skills): cover backslash zip extraction
---------
Co-authored-by: buidackim <221586880+buidackim@users.noreply.github.com>
Co-authored-by: Mỹ Ngân <goclaw-maintain@users.noreply.github.com>
* docs(openapi): add vault endpoints + schemas
The committed OpenAPI spec only documented 41 of ~286 actual routes. This
adds the 19 Vault endpoints — 16 under /v1/vault/* and 3 under
/v1/agents/{agentID}/vault/* — plus their request/response schemas.
Routes covered (matches internal/http/vault_handler_*.go and
vault_graph_handler.go registrations):
- documents: list, create, get, update, delete (per-agent and cross-agent)
- documents/{docID}/links: outlinks + backlinks + name lookup
- links: create, delete, batch get outlinks
- upload: multipart, allowed exts whitelist
- rescan: per-tenant workspace rescan w/ concurrency guard (409)
- tree: lazy sidebar listing
- search: hybrid FTS + vector
- enrichment: status + stop
- graph: lightweight nodes/edges for visualization
Schemas: VaultDocument, VaultDocumentInput, VaultDocumentUpdateInput,
VaultLink, VaultBacklink, VaultSearchResult, VaultTreeEntry,
VaultGraphNode, VaultGraphEdge. Tags: Vault, Vault Graph.
Pure spec addition — no handler changes. The insertion is surgical
(no reformatting of existing entries), so the diff is additive only.
* docs(openapi): cover the remaining HTTP endpoints
Continues the gap-fill started for Vault. Adds 150 more path operations
spanning every other HTTP-route file under internal/http/, with 19 new
component schemas and 8 new tags. After this commit the spec covers 210
paths vs 60 previously committed (the actual handler surface is ~258).
Coverage by area (file → routes added):
knowledge_graph_handlers.go 9 (Knowledge Graph: extract, graph, stats,
dedup scan/dismiss, merge, single entity)
vault_graph_handler.go 1 (kg/graph/compact)
memory_handlers.go 6 (documents CRUD with {path...}, chunks,
index, index-all)
oauth.go 8 (chatgpt + openai: status, start,
callback, logout × 2)
storage.go 4 (files {path...} read/delete, size, move;
POST upload merged into the existing
/v1/storage/files block so the path key
stays unique and YAML-strict parsers like
Swagger UI's loader accept the spec)
agents.go 22 (sync-workspace, shares CRUD, regenerate /
resummon / cancel-summon, system-prompt-
preview, export {preview,download/token},
import {preview,merge}, teams export/import,
instances + metadata, orchestration,
v3-flags)
evolution_handlers.go 3 (metrics, suggestions, suggestion review)
episodic_handlers.go 2 (list, search)
skills.go 18 (versions, files, files/{path...}, upload,
grants agent/user × revoke, install-deps
trio, runtimes, toggle, tenant-config,
export {preview}, import)
mcp.go + mcp_user_credentials.go 16 (test, reconnect, grants + per-user creds,
requests CRUD + review, export {preview},
import, agent-grants list)
providers.go 4 (verify-embedding, codex-pool-activity,
claude-cli auth-status, embedding/status)
tenants.go + users 7 (CRUD + tenant users + tenant-users list,
users/search)
secure_cli.go + agent_grants 17 (cli-credentials full CRUD + presets +
check-binary + test + user-credentials +
agent-grants)
system_configs.go 4 (list, get, put, delete)
backup_handler.go + s3 + tenant 12 (system backup preflight/download/s3 set,
tenant backup + restore)
packages.go 6 (list, install/uninstall, runtimes,
github-releases, shell-deny-groups)
builtin_tools.go 3 (tenant-config get/set/delete)
tts.go + voices.go + caps 6 (config, synthesize, test-connection,
capabilities, voices, voices/refresh)
channel_instances.go +contacts +pending 10 (channel writers, writer groups,
contacts resolve / merge / unmerge / merged,
pending-messages)
files.go + workspace_upload 3 (files {path...} read, files/sign,
team workspace upload + move, team events,
team task attachment download)
edition.go + openapi.go 2 (edition info, openapi.json)
api-keys + traces + usage 3 (revoke, traces/{traceID}/export,
usage/breakdown)
New schemas: KGEntity, KGRelation, KGDedupCandidate, KGStats, TraversalResult,
MemoryDocument, MemoryChunk, MemorySearchHit, OAuthStatus, OAuthStartResponse,
Skill, McpServer, Tenant, TenantUser, CLICredential, Package, EvolutionMetric,
EvolutionSuggestion, EpisodicSummary.
New tags: Tenants, CLI Credentials, Packages, Contacts, Teams, TTS, Evolution,
Episodic Memory.
Purely additive — no existing operation, schema, or tag is modified. The one
overlap (`/v1/storage/files`, previously documented as GET-only) is handled
by inserting the new POST method inside the existing path block in place
rather than re-declaring the key, so each path key appears exactly once and
the spec parses cleanly under both JSON and strict YAML loaders.
Verified by parsing HEAD and the new file with python json.load + yaml.safe_load
and diffing entry-by-entry. Under `git diff --histogram`/`--patience` the diff
shows zero deletions; the default myers algorithm reports cosmetic deletion
lines but they are diff-algorithm noise, not real changes.
Where a body field or response shape was clear from the handler, the spec
captures it; where the handler accepts free-form / additive JSON payloads
the request body is documented as a generic object (additionalProperties:
true) rather than a fabricated rigid schema. Better to be honest about
flexibility than to lock callers into a wrong type.
* docs(openapi): drop nonexistent DELETE /v1/api-keys/{id}
The spec carried a DELETE /v1/api-keys/{id} operation, but
internal/http/api_keys.go registers no such route — only
GET /v1/api-keys, POST /v1/api-keys, and
POST /v1/api-keys/{id}/revoke. The DELETE entry predates this PR
(it was already on dev), so the semantic-preservation check left
it untouched; this removes the stale op so the spec matches the
registered routes. The correct POST /v1/api-keys/{id}/revoke
operation is unchanged.
Verified: json.load parses, go build ./internal/http/ and
go vet ./internal/http/ both clean.
---------
Co-authored-by: Aaron Vu <aaron-tsar@users.noreply.github.com>
* fix(pipeline): defer non-tool messages until all tool results are emitted
Multi-tool responses could interleave synthetic user messages (loop
warnings, nudges) between tool results, breaking OpenAI-compatible
providers that require contiguous tool_result grouping.
Fix both sequential and parallel tool execution paths in tool_stage.go
to defer non-tool messages until after all tool results are appended.
Fix sanitizeHistory lookahead in loop_history_sanitize.go to reorder
interleaved non-tool messages in persisted session history.
Closes#1177
* perf(agent): cache BuildFilteredTools result per run
All filtering inputs (policy, disabled tools, bootstrap, channel,
orchestration mode, user MCP tools) are fixed for the lifetime of a
run. Only the final iteration differs (strips all tools). Cache the
tool definitions after the first call and reuse for iterations
0..maxIter-1, eliminating N-1 redundant 7-step policy evaluations per
agent run.
Currently the web_search tool resolves a tenant-configured provider
chain (e.g. tavily → exa → brave) and stops at the first success.
That's the right default for cost/latency, but it makes cross-engine
corroboration impossible at the agent level: the caller has no way to
ask "search the same query on Exa specifically" once Tavily has
already returned a hit.
Add an optional `provider` argument. When set, the chain is narrowed
to the named provider (case-insensitive); other providers are not
queried. The first-success-wins fallback is preserved when the arg
is omitted.
Use case (concrete): a research agent verifying source freshness
calls web_search twice for the same query — once with provider="tavily"
and once with provider="exa" — and compares the URLs returned. URLs
that appear in both engines are high-confidence original primary
sources; URLs only in one signal a republish or low-circulation outlet
worth verifying further.
Other changes:
- Cache key now includes the requested provider so per-engine
results don't collide. Without this, the second call would just
replay the first engine's cached result, defeating the purpose.
- Unknown provider returns a clear error listing what IS configured
for the tenant — better DX than a silent no-op.
Tests: 5 new cases covering narrowing, case-insensitivity, unknown
provider, default first-success behaviour preservation, and cache
isolation across providers.
* feat(cron): deterministic command payloads (run a shell command, no LLM)
Cron jobs always run an agent turn today, so deterministic work (health
probes, backups, syncs) pays model tokens on every fire. This adds a
"command" payload kind that runs a shell command directly in the gateway
process with zero model tokens, mirroring openclaw's command cron.
- store: CronPayload.Command (*CronCommandSpec — argv/cwd/env/input/
timeouts/output cap). Persists in the existing payload JSON blob, so
there is NO migration and no schema version bump.
- internal/cronexec: in-process runner with wall-clock + no-output
timeouts, per-stream output capping, and process-group termination so a
timed-out command's forked children are also killed.
- gateway_cron handler: command jobs run in-process and deliver stdout on
success (honoring the NO_REPLY sentinel). A non-zero exit / timeout
returns an error so the run is recorded as error and retried per
cron.max_retries; failures are NOT delivered, mirroring the agent path
(only successful output is announced — no channel spam).
- surfaces: cron.create RPC, the agent `cron` tool, and a new
`goclaw cron create` CLI all accept command payloads.
- security: gated by cron.command_enabled (default false). Commands run
with the gateway process's privileges, so the feature is opt-in per
gateway; when disabled the RPC and tool reject command payloads and the
handler refuses to run them.
- i18n (en/vi/zh), docs (08-scheduling-cron.md), and tests for the runner
and the handler command path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cron): gate command payloads on the update surfaces too
handleUpdate (RPC + agent tool) passed CronJobPatch.Command straight to
UpdateJob, which switches the payload to command kind for any non-nil
Command — without the command_enabled gate or ValidateCronCommandSpec that
create enforces. A normal job could therefore be mutated into a command job
(or persisted with an invalid spec, e.g. empty argv) on a gateway where
command cron is disabled, breaking the disabled-gateway contract.
Both update surfaces now require cron.command_enabled and validate the spec
before UpdateJob, matching create. The agent tool parses the command via the
same path as add and drops the raw keys so a shell-string command can't break
the generic patch unmarshal. Regression tests added for RPC and tool update
(command disabled + invalid argv), plus a positive enabled-valid case.
Addresses review feedback from @mrgoonie on #1279.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pairing requests and message handlers were silently using MasterTenantID
instead of the correct tenant because context.Context had no tenant set.
Root cause: instance_loader.go passed bare ctx (no tenant) to
startChannelWithTimeout; channel handlers also missing store.WithTenantID
at entry points.
Channels fixed:
- instance_loader.go: pass instCtx (with tenant) to startChannelWithTimeout
- Telegram, Bitrix24, Feishu: add store.WithTenantID at handler entry
- Facebook: add ctx param to handleMessagingEvent/handleCommentEvent + webhook_handler
- Pancake: add ctx param to handleMessagingEvent/handleCommentEvent + webhook_handler
Also fix consolidation pruner: PruneExpired SQL had tenant_id filter that
caused zero rows to ever be deleted (context had no tenant). Remove tenant
filter — pruning is a global maintenance operation.
Add regression tests:
- internal/channels/pancake/tenant_context_test.go (6 tests)
- internal/channels/facebook/tenant_context_test.go (4 tests)
- internal/consolidation/prune_expired_test.go (5 tests)
Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com>
The async webhook worker double-processed long-running agent calls: the
90s stale-running sweep reclaimed rows whose agent ran longer than 90s
(no heartbeat), re-running the agent and duplicating MCP side-effects.
Add a last_heartbeat_at column + a Heartbeat lease-renewal method (CAS on
lease_token). While an agent runs, the worker renews the lease every 30s;
ReclaimStale now keys off last_heartbeat_at instead of started_at, so a
live run is never reclaimed while a dead worker is still recovered within
the stale window. If a run loses its lease (reclaimed), the heartbeat
cancels the run context so the agent stops immediately and writes no
further side-effects.
- migrations/000085 (PG) + SQLite schema v52 + RequiredSchemaVersion 85
- WebhookCallStore.Heartbeat (PG + SQLite impls); ClaimNext/ReclaimStale
switched to last_heartbeat_at
- worker heartbeat goroutine + cancel-on-lease-loss; invokeAgent honors ctx
- store + worker tests
setupToolRegistry wires the filesystem tools' AllowPaths from the config/JSON5
default before ApplySystemConfigs overlays system_configs['allowed_paths'], so
DB-configured allowed paths never reached read_file / list_files / write_file /
edit / send_file. Only the rate limiter was re-applied after the overlay (#1111);
the AllowPaths analogue was missing, so agents were denied access to configured
shared directories outside their workspace even though the DB value was present
(visible as a read_file "access denied" log whose allowedPrefixes omit the
configured path).
Extract the user-allowed-path application into applyUserAllowedPaths and re-run it
from runGateway after the overlay, mirroring the rate-limiter re-apply. The helper
is idempotent (AllowPaths is additive and the prefix check is membership-based),
so the initial wiring call plus the re-apply is safe.
Test: cmd/gateway_tools_wiring_test.go asserts a path outside the workspace is
denied before the grant and readable after, that an unrelated path stays denied,
that repeated application is safe, and that an empty list is a no-op.
Add RFC 2544 benchmarking range (198.18.0.0/15) and reserved range
(240.0.0.0/4) to both SSRF blocklists (security package + web_fetch
tool) to prevent internal network access via these special-use IPs.
Closes#1218
A run records its terminal run.status (completed/failed/cancelled) only
when the agent loop returns. If the gateway process is killed mid-run
(restart, crash, or SIGKILL after a slow graceful shutdown), the terminal
event is never emitted, so the run's last run.status item stays "started"
forever — it shows as perpetually running in the timeline and is never
counted as failed.
Mirror the cron scheduler's startup reset (recomputeStaleJobs flips stale
'running' jobs to 'interrupted'): add RunTimelineStore.RecoverInterruptedRuns,
invoked once on gateway startup. It finds runs that have a started run.status
but no terminal sibling and appends a terminal failed run.status item, marked
interrupted in metadata so it stays distinguishable from a genuine agent
failure. Running only at startup means nothing from before is still
executing, so there is no false-positive risk — unlike a periodic sweep
(see the deliberately-disabled tracing stale-trace loop).
- store: RecoverInterruptedRuns on the interface + PG and SQLite impls
- cmd/gateway_managed: invoke once after the timeline recorder is built
- tests: SQLite store test — orphan recovered, completed run untouched,
idempotent on re-run
Server-side webhook agent runs (sync, async, and admin test) used Stream:false,
so OpenAI-compatible routers that only cache streaming requests never populated
or served their prompt cache. Webhook runs paid full input-token price on every
turn even with a stable session and an identical multi-turn prefix, while WS chat
(Stream:true) got cache hits from the 3rd turn.
Add gateway.webhook_stream (default true, env GOCLAW_WEBHOOK_STREAM) and apply it
to all three run sites. ChatStream returns the fully assembled response, so the
payload returned to the caller is unchanged. Set to false to restore non-streaming.
Multi-tool responses could interleave synthetic user messages (loop
warnings, nudges) between tool results, breaking OpenAI-compatible
providers that require contiguous tool_result grouping.
Fix both sequential and parallel tool execution paths in tool_stage.go
to defer non-tool messages until after all tool results are appended.
Fix sanitizeHistory lookahead in loop_history_sanitize.go to reorder
interleaved non-tool messages in persisted session history.
Closes#1177
Add total-count pagination to the webhook admin endpoints and the web UI.
Store:
- WebhookStore/WebhookCallStore gain Count; WebhookListFilter gains
IncludeRevoked + Query (PG + SQLite, parameterized, tenant-scoped)
API:
- GET /v1/webhooks and GET /v1/webhooks/{id}/calls return
{items, total, limit, offset} with server-side search + revoked filtering
Web UI:
- server-driven list pager + search/revoked filter; call-history dialog uses
the real total (fixes the full-page "has more" boundary bug)
- i18n pager labels (en/vi/zh)
Tests: store pagination integration test; mock stores implement Count.
No schema migration (read-only COUNT).
Replace the hardcoded 30s webhook agent-run deadline with a configurable
timeout (default 600s, cap 3600s) for both the async worker and the
sync/test HTTP handler. Legitimate multi-step runs were dying at 30s
mid-tool-call; the same request over WebSocket completed fine.
- internal/webhooks/timeout.go: ResolveTimeoutSec helper (<=0 → 600s, cap 3600s)
- WorkerConfig.AsyncAgentTimeout (async worker) + WebhookLLMHandler.syncTimeout
(sync + admin test), wired from config
- config keys gateway.webhook_{async,sync}_timeout_sec, also settable via
GOCLAW_WEBHOOK_{ASYNC,SYNC}_TIMEOUT_SEC env (env overrides config)
- tests: timeout bounds + config file/env override
The agent config UI (tools-profile-section) and i18n already expose a
per-agent "Rate Limit (per hour)" field saved into tools_config, but the
backend ignored it — the tool rate limiter only ever used the global
tools.rate_limit_per_hour. Wire the field through: ToolPolicySpec gains
RateLimitPerHour; the agent loop threads it into the tool-execution
context; the registry passes it to the limiter, which uses it in place
of the global max when > 0 (0 inherits the global).
The per-run session reset was gated on `!job.Stateless`, inverting the
flag: stateless jobs (the token-saving default) skipped the reset and
accumulated unbounded history, while stateful jobs were wiped every run.
Reset for `job.Stateless` instead, and clear BOTH session layers — the
goclaw session store AND the Claude CLI on-disk .jsonl. claude-cli
resumes its own session by a deterministic per-key UUID, so without
clearing the .jsonl a "stateless" run still replayed the entire
accumulated history (and silently grew it run after run).
Follow-up to #1260. A model (notably the claude-cli proxy under a
degraded session) sometimes emits a tool call as a bare
<invoke name="...">...</invoke> block with no <function_calls> wrapper.
fullToolCallBlockPattern only matched the wrapped form, so the bare
block slipped through to the tag-only strip, leaking the inner
<parameter> command text into the reply while the tool never ran.
Match and remove complete bare <invoke> blocks too (after the wrapped
form), add "<invoke name=" as a detection indicator, and log the
dropped tool name(s). Partial/unterminated artifacts still fall through
to the existing tag strip.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stripGarbledToolXML removed only the tags of a model-emitted
<function_calls> block, leaking the inner <parameter> argument values
into the user-facing reply, and the dropped call was completely silent.
Detect a complete tool-call block, remove it whole, and log the
attempted tool name(s) at WARN so the no-op is diagnosable. Partial
artifacts still fall through to the existing tag-level strip.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Allow overriding the cron job execution timeout via env var, matching
the existing GOCLAW_* env var pattern used throughout the codebase.
Falls back to the existing default (10m) when unset or invalid, since
CronConfig.JobTimeoutDuration() already handles parse errors with a
slog.Warn and the DefaultJobTimeout constant.
Co-authored-by: will.nguyen <will.nguyen@gemcommerce.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Local pkg-helper installs an arm64 mcp-server binary on macOS hosts; the
existing allowlist (deno, bun, uvx, ...) didn't cover it so MCP server
config validation rejected legitimate entries.
* refactor(bitrix24): rename "Path B" framing to maintainer-specified naming [B24:2794]
Per maintainer hard rule #10 (no generic "Path A/B" framing) from PR #1061
review. The Bitrix24 MCP auto-onboard flow is Bitrix-specific glue
("Bitrix24 OAuth -> existing mcp_user_credentials bridge"), NOT a generic
MCP architecture pattern.
Naming convention applied consistently:
- First mention per file: full "Bitrix24 OAuth -> existing
mcp_user_credentials bridge" (matches maintainer comment verbatim).
- Subsequent mentions in same file: shortened "mcp_user_credentials bridge".
- Test/log context referencing literal endpoint /api/auto-onboard: keep
"auto-onboard" reference (it's the actual API endpoint name).
Changes are documentation-only:
- Rename in code comments + test descriptions + plan docs.
- Clarify framing in mcp_client.go + provisioner.go doc comments to
emphasize Bitrix-specific glue (not generic MCP infra).
- Reuse existing mcp_user_credentials table + MCPServerStore methods
(no schema / store / abstraction change).
Files:
- cmd/gateway.go (factory registration doc)
- internal/channels/bitrix24/{channel,factory,mcp_client,provisioner}.go
- internal/channels/bitrix24/{mcp_client,provisioner}_test.go
- plan/goclaw-mcp-integration.md (21 occurrences)
Verified: go build + MCP-related tests pass (TestProvision*,
TestInitMCPProvisioner*, TestMCPClient*).
Phase 1 of Path C execution per
plans/reports/decision-log-260519-1555-bitrix24-pr-fork-decision.md.
* fix: confine outbound media paths to agent workspace [B24:2794]
Tool MEDIA:<path> output reached channel file-upload sinks (Bitrix
imbot.v2.File.upload, Telegram sendDocument, etc.) verbatim via
parseMediaResult, with no workspace-boundary check. A malicious or buggy
tool emitting MEDIA:/etc/passwd could exfiltrate arbitrary files to chat.
Extract the EvalSymlinks+Rel containment from extractMediaFromContent into
a shared confineToWorkspace helper and apply it at the parseMediaResult
sink in processToolResult. Fixing at the source/egress boundary protects
every channel at once rather than per-channel. Paths that escape the
workspace are dropped and logged (security.media_path_rejected).
Add TestConfineToWorkspace (boundary unit) and
TestParseMediaResultConfinedToWorkspace (sink regression for H2).
* feat(bitrix24): support inbound + outbound media via imbot.v2 File API [B24:2794]
Bitrix24 channel was text-only; attachments were parsed but dropped.
- Inbound: download chat files via imbot.v2.File.download (one-time URL),
forward to the agent with MIME preserved (internal/channels/bitrix24/download.go).
- Outbound: upload agent media to the chat via imbot.v2.File.upload
(internal/channels/bitrix24/send_media.go).
- Add BaseChannel.HandleMessageMedia to preserve MIME/filename through the bus.
- Per-channel media_max_mb cap (default 20) applies to both directions.
Tests: 92 pass (internal/channels/bitrix24 + internal/channels), go vet clean (PG + sqliteonly).
* refactor(bitrix24): migrate messaging/bot-list/unregister to imbot v2 API [B24:2794]
Move outbound REST calls to the imbot v2 family (keeps register on v1):
- imbot.message.add -> imbot.v2.Chat.Message.send (fields.message shape, live-verified)
- imbot.bot.list (+ legacy imbot.list fallback) -> imbot.v2.Bot.list; add botListRows
to normalize the v2 {bots:[...]} envelope, legacy array, and id-keyed map forms
- imbot.unregister -> imbot.v2.Bot.unregister
Bot registration stays on v1 imbot.register: v2 imbot.v2.Bot.register changes the
event-delivery model (per-event handler URLs -> eventMode), which would require
rewriting the inbound event parser. No user-facing behavior change.
Tests: bitrix24 package green; go vet ./... clean.
* feat(bitrix24): route whisper via v1 SKIP_CONNECTOR + add v2 replyId [B24:2794]
Bot was leaking HiddenMessage (whisper) replies to the external Zalo
connector because every outbound call went through imbot.v2.Chat.Message.send,
which has no equivalent of the v1 SKIP_CONNECTOR flag. Branch the outbound
path on inbound visibility:
whisper → imbot.message.add + SKIP_CONNECTOR=Y (v1, send_v1.go)
public → imbot.v2.Chat.Message.send + fields.replyId (v2, send_v2.go)
Pipeline:
events.go parse data[PARAMS][PARAMS][COMPONENT_ID]=HiddenMessage
into EventParams.IsHiddenMessage (form + JSON variants)
handle.go set bitrix_visibility on InboundMessage.Metadata
consumer forward visibility + message_id into OutboundMessage
send.go resolveSendOptions + sendChunk dispatcher +
shared callWithRateLimitRetry helper
metadata_keys.go single source of truth for the keys + values
Defaults preserve pre-refactor behaviour: callers that don't populate
bitrix_visibility still go through v2 public, and replyId is omitted
unless a numeric bitrix_message_id arrives in metadata.
Tests:
TestParseEvent_FormURLEncoded_IsHiddenMessage (3 cases)
TestParseEvent_JSON_IsHiddenMessage (3 cases)
TestResolveSendOptions (8 cases)
TestSend_BranchesOnVisibility (4 cases)
* feat(bitrix24): openline sender-tag echo on replies [B24:2794]
Openline sender-tag echo (this change):
- Capture the connector sender tag ("[name #id]:" or "[name] #id:") from
inbound openline group messages, strip it from the body the agent sees,
and re-prepend the canonical "[name] #id:" form to the reply so the Open
Channel connector routes the answer back to the right external user.
- New sender_prefix.go helper (+ test) accepts both inbound layouts and
emits one canonical form; scoped to messages carrying the tag, so plain
chats are unaffected.
- metadata_keys.go: MetaKeySenderPrefix; handle.go capture/strip/stash;
gateway_consumer_normal.go forwards the key; send.go prepends it on the
first chunk before chunking.
Bundled bitrix24 channel-core work already on this branch:
- handle.go: @mention is the sole trigger for both staff and connector
customers; unmentioned traffic is dropped (was: drop all connector msgs).
- isGroupMessageType: treat SONET_GROUP "B" as a group.
- handle_test.go, mcp_client_test.go: cover the above.
* feat(bitrix24): accept colon-less openline sender tag, echo [name] #id [B24:2794]
The Open Channel connector dropped the trailing colon from its sender tag:
inbound now arrives as "[Name] #id <msg>" (was "[Name] #id: <msg>"). The
id-bearing patterns required the colon, so the tag fell through to the
name-only branch and the reply echoed "[Name]" — dropping the #id the
connector needs to route the answer back.
- sender_prefix.go: make the trailing ":" optional on both id layouts
([name #id] / [name] #id, with or without colon) and echo the canonical
"[name] #id" (no colon) to match the connector's current format. Bare
"[name]" (no id) still echoes "[name]" for Open Channel only.
- handle.go: gate the bare name-only layout to Open Channel (isOpenChannel)
so ordinary group chats starting with "[x] ..." are left untouched.
- sender_prefix_test.go: cover colon/no-colon x id-inside/id-outside, the
name-only openline case, and the non-openline no-op.
* fix: security and robustness fixes from the bitrix24 channel review [B24:2794]
- download.go: block redirect-based SSRF on inbound media. CheckRedirect
re-validates each hop (http(s) only, reject private/loopback/link-local
hosts, cap hops); the initial portal-domain pin is no longer bypassable
via a 3xx to an internal service. Public-host redirects still allowed.
- handle.go: extract/echo the openline sender tag only for Open Channel
sessions (was: any group chat), removing bogus prefixes in CRM group
chats and narrowing the forged-tag misroute surface.
- loop_tools.go + loop_media.go: confine result.Media to the agent / team /
tenant-allowed roots (new confineToAnyRoot) before a channel uploads it,
so a prompt-injected out-of-workspace path (e.g. /etc/passwd) cannot
exfiltrate, while legitimate cross-workspace media (team files, delegatee
output) still flows.
- send_media.go: bounded outbound read via io.LimitReader replaces the
os.Stat + os.ReadFile pair, closing the TOCTOU size-cap bypass; cap a
single message's outbound attachments at 10 (mirrors inbound).
- register.go: paginate imbot.v2.Bot.list (limit/offset + hasNextPage,
capped at 40 pages) so verify/lookup see bots past the first 50.
- mcp_client.go: redact access_token / refresh_token / client_secret from an
echoed MCP error body before it is logged or returned (+ test).
* fix(security): validate resolved dial IP on Bitrix media redirects [B24:2794]
The inbound media download redirect guard only string-checked the redirect
hostname (isPrivateOrLoopback on req.URL.Hostname()), so a redirect to a public
hostname that resolves to 127.0.0.1 / 169.254.169.254 / an RFC1918 address — or a
DNS-rebinding swap between check and dial — still passed the guard and the client
would connect. Reported in PR review.
Add security.NewRedirectFollowingSafeClient: it follows redirects but validates
the RESOLVED destination IP of every hop at dial time via net.Dialer.Control,
reusing the existing blocked-CIDR list. The IP it checks is the IP actually
dialed, so both redirect-to-internal and DNS rebinding are refused, while
legitimate public CDN redirects still succeed. download.go now uses it instead of
the hostname-string guard.
Tests: deterministic dial-control table (loopback / link-local / private /
multicast / unspecified / public, v4 + v6), malformed/non-IP addr, test bypass,
loopback-dial-blocked client wiring, and redirect cap + scheme checks.
* feat(bitrix24): per-participant Zalo openline identity from 3-token sender tag [B24:2794]
Parse the connector's "[Name] #uid #msgId" sender tag so each external
customer in a shared Open Channel group gets its own contact + USER.md
instead of collapsing onto the connector proxy id. Identity minting is
gated on IS_CONNECTOR=Y to reject operator forged tags. Echo back the
msgId only ("#msgId") on replies; keep the legacy single-number and
name-only layouts unchanged. Zero DB migration.
- sender_prefix.go: parseOpenlineSenderTag() classifies 3-token / legacy / name-only
- handle.go: synthetic senderID "openlines:{instance}:{chat}:{uid}" + participant_user_id metadata, gated on FromIsConnector
- gateway_consumer_normal.go: deriveGroupUserID() routes participant -> per-person scope, group fallback otherwise
- send.go: buildAddressMention numeric-id guard so synthetic ids don't emit invalid [USER=...] BBCode
- MetaKeyMessageID kept as Bitrix MESSAGE_ID (drives v2 fields.replyId); connector msgId surfaced only via echo prefix
---------
Co-authored-by: DangTinh311 <dangtinh31193@gmail.com>
Co-authored-by: Chinh Dang <chinhdang@192.168.68.104>
