mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-11 12:18:59 +00:00
dev
394
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 thotam
|
3f90057c24 |
fix(pipeline): stop aborting runs on heuristic context budget estimates (#1587)
Runs on models without a registered tokenizer (e.g. 9router brand models)
ended with the generic "Agent couldn't generate a response" fallback even
though the real request used about 55% of the context window.
PruneStage counted history with TokenCounter, which falls back to a
chars/2 heuristic for unregistered models and overcounted about 1.8x.
Once over budget it ran memory flush (~35s, invisible in traces), then
mid-loop compaction, which cannot summarize a history made only of tool
call/result pairs. The callback reported the untouched history as
compacted, PruneStage still saw it over budget and returned AbortRun
before any LLM call, and FinalizeStage replaced the empty reply with the
fallback.
- PruneStage and ContextStage overhead count with the request guard's
BudgetCounter. PruneStage no longer controls loop flow; the final
request guard in ThinkStage decides.
- CompactMessages returns ErrNotCompacted when history is unchanged.
Callers stop counting it as a compaction and do not retry it in the
same run, while post-run summarization still sees the pressure.
- When the guard exhausts every reduction step, ThinkStage stops the run
with a localized chat.context_budget_exceeded notice instead of an
error, so the run's tool results are still persisted. The stop reason
marks the trace and agent span as error; team tasks, cron and
heartbeat treat it as a failure via RunOutcome.Failure().
- Memory flush and mid-loop compaction emit event spans.
- Web and desktop UIs treat an unset context_pruning as enabled (the
backend default since
|
||
|
yatulandClaude Opus 5
|
305bd7c50d |
fix(teams): grant a delegated lead read access to its own team (#1537)
* fix(teams): scope team tasks to the delegation origin, not the delivery channel
A delegated run is delivered on the internal "delegate" channel while its real
origin is preserved separately — buildAgentLinkRunRequest is explicit about it:
// preserves the origin's authorization-bearing identity while keeping
// delegation on its internal delivery channel.
Channel: "delegate",
WorkspaceChannel: req.Channel,
WorkspaceChatID: req.ChatID,
The team tools did not consult that origin, so a task created by a lead reached
through delegate was stamped with the delivery channel. Nothing is registered
for it, so every notification about that task — completion, failure, blocker
escalation, ask_user — was dropped:
unknown channel for outbound message channel=delegate
The delegatee's first answer still arrived, because it travels back as the
delegation result rather than through a channel; everything the lead said after
the delegation closed was lost. One session produced 16 such drops, including a
blocker escalation the user needed to see.
Add OriginChannelFromCtx / OriginChatIDFromCtx next to the existing workspace
scope propagation helpers, and use them where team scoping and notification
routing are decided: task records, list/search scoping, dispatch fallbacks,
event payloads, escalation tasks, ask_user and leader notifications. The
resolution is the identity when no delegation origin is present, so
non-delegated flows are unchanged.
Deliberately not touched: the channel used in authorization decisions
(checkTeamAccess, requireLead, approve/reject lead bypass). Those ask "how did
this call arrive", not "where should the answer go", and widening them is a
separate question.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(teams): cover delegated-lead completion routing and origin isolation
Triage on #1529 named two gates this PR had not met: regression coverage for
delegated lead task completion, and a guard that unrelated origins cannot
receive the notification. The existing tests only covered what create persists.
TestDelegatedLeadCompletionNotifiesOrigin completes a task raised in a delegated
context and asserts the completion event is addressed to the caller's origin.
Reverting team_event_helpers.go to the delivery channel fails it with
"addressed to delegate/system".
TestCompletionNotificationStaysWithinItsOwnOrigin puts two tasks from different
origins on one board, completes one, and asserts no completion notification
carries the other origin.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(teams): grant a delegated lead read access to its own team
Fixes #1535. Second attempt: the first one set RunRequest.TeamWorkspace on the
delegated run, which loop_context.go rejects outright and deliberately so — that
field also becomes ToolWorkspace, which would displace the exchange outputs
directory. Every delegation then died at setup with "invalid delegation artifact
workspace". This does not go near that field.
The separation the lead needs already exists in the code. A lead addressed
directly resolves its team at loop_context.go:285-321 and gets
ctx = tools.WithToolTeamWorkspace(ctx, wsDir)
ctx = tools.WithToolTeamRoot(ctx, teamRoot)
with no WithToolWorkspace call — a read allowance, not an override. Only the
!isArtifactDelegation gate keeps a delegated lead out of it. So the grant is
made inside the artifact branch instead, from the same inputs, and the guard on
req.TeamWorkspace stays exactly as strict as it was.
Both paths are needed. The workspace alone covers only the lead's own chat leaf;
the deliverable in the report sits at teams/<id>/system/review-....md, written by
a member under a different chat scope. buildAllowedPrefixes adds the team root
for reads and not for writes (filesystem.go:366-370), which is precisely the
asymmetry this case wants: the lead reads the team's output, per-chat write
isolation is untouched.
Team ID is deliberately not set. It switches on the workspace interceptor's
write validation, file-change broadcast and task attachment (workspace_
interceptor.go:36,114,203) — none of which a delegated run should trigger, and
none of which reading needs.
Ambiguity is not resolved silently, as triage asked: an agent leading more than
one active team gets nothing. store.GetTeamForAgent would answer in one call but
it is ORDER BY (lead_agent_id = $1) DESC LIMIT 1 and would quietly pick a team.
Hermeticity is unchanged: send_file and message still refuse inside an artifact
run, publication still happens through delegation outputs on completion.
Tests: TestInjectContext_DelegatedLeadReadsTeamWithoutDisplacingOutputs drives
injectContext through the guard that broke the first attempt and pins both
halves — the run is not refused, and ToolWorkspace stays the outputs directory.
Resolution rules, the shared/isolated path shapes, read-not-write on the team
root, and send_file staying blocked are covered alongside.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(teams): pin that the delegated lead resolver does not re-scope by tenant
l.dataDir arrives already tenant-scoped from resolver.go (config.TenantDataDir),
which is why resolveDelegatedLeadTeamRead must not apply TenantLayer itself.
That was carried only by a comment, here and in the sibling branch of
injectContext, and nothing failed if someone added the layer back.
The failure it guards against is quiet: the path stays plausible, it just gains
a second tenant segment — teams/<id> under tenants/<slug>/tenants/<slug> — so
the lead is handed a directory its own tasks never write to, and reads come back
empty rather than denied.
Verified the assertion has teeth by reintroducing the layer: the subtest fails
with "tenant segment appears 2 times". Prompted by #1546 in this same area,
where a contract that lived outside the unit under test went unpinned and the
feature shipped green and did nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
isaacgao4396andisaacgao4396
|
169e0bafaf |
fix(tools): use_skill inlines content when read_file isn't granted (#1547)
* feat(store): add context helper for per-iteration tool allowlist * feat(pipeline): surface resolved tool allowlist to tool dispatch via context * fix(tools): use_skill inlines content when read_file isn't granted Fixes #1477 --------- Co-authored-by: isaacgao4396 <gaoyuan4396@gmail.com> |
||
|
HaiDuongandmor-phongdt
|
b39f0decb9 |
fix(skills): five defects in slash activation, grants, subagent tool policy, preview and import (#1534)
* fix(agent): filter skill slash commands by the agent's grants The inline <available_skills> block was already filtered by visibility and agent grants, but slash activation read the loader's full skill list. A /<slug> command could therefore activate a skill the agent was never granted, and the not-found suggestions disclosed that such skills existed. Resolve slash commands against the agent's allow list via FilterSkills. The list is filtered before matching, so /list-skills, /help and the near-match suggestions are all covered by the same change. The allow-list convention is unchanged: nil means every skill (the fallback when the access store errors), an empty slice means none, and a populated slice is an explicit set of slugs. * fix(agent): split slash commands on any whitespace The tokenizer separated the skill name from the rest of the message with strings.Cut(after, " ") — a literal space. A user who typed the command and pressed Enter before the rest of the message sent "/ck:git\nreview the diff", which parsed to the target "ck:git\nreview" and matched no skill. The reply was "skill not found" followed by near-matches that included the skill they had just named, because the similarity fallback searched the mangled string and still landed beside it. Multi-line messages are ordinary in Slack and Telegram, so this was reachable in normal use. Introduce cutFirstField, which splits on the first run of whitespace, and use it everywhere the literal-space split appeared. The match loop had the same assumption in strings.HasPrefix(raw, value+" "); hasFieldPrefix replaces it and decodes the following rune properly, so a multi-byte skill name is not truncated. That also stops a plain string prefix from matching: a command for "frontend-design-extra" no longer resolves to "frontend-design". * fix(tools): inherit the parent agent's tool policy in spawned subagents buildSubagentToolsRegistry clones the parent registry, then overwrites exec, read_file, write_file and list_files with freshly constructed tools. A fresh tool carries none of the hardening the gateway applies to the parent's instances at startup: exec path denials and their exemptions, the shell deny-group toggles, the command keyword allowlist, and the read/write/list deny prefixes covering config.json, the internal databases and delegate/. Spawning a subagent therefore widened what an agent could reach — the parent was blocked from the data dir, the subagent was not. Verified by disabling the new call: the subagent's exec read config.json out of the denied data dir, read_file did the same, and write_file overwrote it. Copy the policy from the live parent instances rather than re-deriving it, so there is one source of truth and a deny path reloaded later through config pub/sub reaches subagents without a second wiring site that can drift. Allow-prefixes are inherited alongside the denials: they are what make the denied roots usable at all, since the skills store sits under the denied data dir. Copying denials without them would leave a subagent unable to read the skills it is told to use. * fix(agent): use one inline-vs-search decision for prompt and preview The system-prompt preview exists to show the prompt an agent actually gets, but it decided between inline skills and search mode on its own terms: it counted tokens with the fallback counter over the fully rendered XML — tags and <location> paths included, at roughly runes/2 — while the prompt builder estimated name+description at chars/4. On the same eighteen skills that read 3087 against 944, so the preview reported search mode for an agent that was running inline. Extract shouldInlineSkills and call it from both paths. The estimate keeps the prompt builder's rule, mirroring BuildSummary's 200-rune description truncation, so the decision still costs no rendering. The preview's loader interface gains FilterSkills to feed it. Its behaviour on an access-store error is unchanged: an empty allow list yields no skills rather than falling back to showing every skill. * fix(http): keep reference files when importing skills Export archives the whole skill directory, but import recognised only metadata.json, SKILL.md and grants.jsonl. The switch had no default, so every other file was discarded without a log line. A skill whose SKILL.md cites references/*.md arrived without them, the import reported success, and the skill failed at the first read. Collect the remaining entries and write them under the skill directory with their structure intact. Archive entry names are attacker-controlled, so each path goes through sanitizeRelPath: a traversal attempt collapses to a relative path and the write stays inside the skill directory. * test(agent): make the inline-decision table exercise both gates The "100 skills, 200-char descriptions" case claimed to cover the token ceiling, but 100 exceeds the count ceiling so the count check short-circuited and the token branch never ran — deleting that branch would not have failed the test. Split the table so each case isolates one gate: tiny descriptions for the count rows, a count safely under the ceiling for the token rows. Removing the token comparison now fails the over-the-ceiling case and nothing else. * fix(agent): gate slash commands on the managed tier only The allow list comes from a query over the `skills` table, so filesystem-tier skills — the workspace, .agents and ~/.agents directories of the five-tier loader — have no row in it. Filtering every skill against the list made those four tiers unreachable by slash while skill_search still found them unfiltered, which turned a security fix into a functional regression. Apply the list to managed skills only. Builtins are seeded with is_system and returned unconditionally, so they stay reachable either way. This closes slash activation of ungranted managed skills. It does not close skill_search and use_skill, which still read the loader's full list; that is a wider change and wants its own review. * fix(http): stop imported skill files from replacing guarded ones Two ways the auxiliary write could go wrong. GuardSkillContent scans SKILL.md before anything reaches disk, but the switch that routes archive entries compares the raw path. An entry named "./SKILL.md" does not match it, so it fell through to the auxiliary set — where sanitizeRelPath collapses it back to "SKILL.md" and the write replaced the file that had just been scanned. Reject any auxiliary path that resolves to a name the import handles by itself, and log the attempt. Tar directory entries carry a trailing separator and no content. Written as files they took the name a real directory needed, so everything beneath was dropped — and whether that happened depended on map iteration order, making it intermittent. Skip them; MkdirAll creates what is needed. Also cap the number of auxiliary files per skill and log when the cap trims an archive, so a truncated import is visible rather than silent. --------- Co-authored-by: mor-phongdt <phong.dangtuan@mor.com.vn> |
||
|
Justin Truong
|
d13cc0804a |
fix(providers): carry tool name on tool results for Gemini
Gemini drops tool_call_id and pairs functionCall/functionResponse by function name, so its OpenAI-compat shim requires a non-empty FunctionResponse.name. The name was only recoverable through a reverse id->name lookup over assistant tool_calls still present in the request window, which fails after pruning, truncation or tool_call collapse and produces HTTP 400 "Name cannot be empty" on any follow-up iteration. Add Message.ToolName, set it at every tool-result creation site and preserve it through context pruning, then prefer it when serializing. Fall back to the existing index for history persisted before the field, so old sessions keep working without a reset. When neither source resolves a name, drop the unlabelled tool result instead of emitting an empty one: an empty name is a guaranteed 400 and a synthetic name would match no prior functionCall. Gated behind the existing Gemini detection so other OpenAI-compat hosts, which pair by tool_call_id, are unaffected. Field is JSON-optional, so session history needs no migration and older binaries ignore it on rollback. |
||
|
bilogic
|
dacdddbfa1 | don't react with 💔 on telegram (#1492) | ||
|
bilogic
|
6e61bd1531 | stop responding with "..." (#1491) | ||
|
ntduc
|
4426f76140 | fix(agent): deduplicate automatic media delivery | ||
|
Duc Nguyenandntduc
|
bb7712a9ff |
fix(collaboration): harden delegated task isolation (#1486)
* feat(collaboration): isolate delegated artifacts and child runs Isolate delegated inputs and outputs behind secure artifact exchange lifecycles. Scope Agent Link tasks by tenant and root agent, and enforce delegation spawn-tree boundaries. Add process-wide child-run admission and preserve logical media paths across native, MCP, and sandbox execution. * fix(collaboration): harden delegated task isolation Enforce tenant and root-agent task scope across migrations and stores. Add exactly-once async completion delivery, delegated sandbox boundaries, and confined artifact and media recovery across runtime surfaces. * fix(collaboration): recover interrupted async tasks * fix(collaboration): normalize persisted child-run status --------- Co-authored-by: ntduc <ntduc@cpp.ai.vn> |
||
|
nguyenha935andnguyenha935
|
2c7903e7ea |
fix(agent): make the extractive memory fallback work on non-English sessions (#1484)
The extractive fallback runs when an LLM memory flush fails, so it is the last
thing standing between a session and losing its accumulated decisions. Its cue
patterns were English-only ("decided to", "prefer", "the API is"), which makes it
effectively inert for a session conducted in another language: it still returns a
non-empty result, so nothing looks broken, but the result is noise.
Measured on 195k characters of real Vietnamese session text: 9 decision matches
(all of them stray English fragments), 0 preferences, 0 technical facts — about
97% of what it "saved" was incidental file paths. After adding Vietnamese cues the
same corpus yields 21 decisions and 6 preferences on the largest session.
Adds parallel Vietnamese patterns for the same three categories and merges them
into the existing results with deduplication, so English extraction is unchanged
and mixed-language sessions get both. Tests cover Vietnamese-only,
English-only, and mixed histories, plus a guard that ordinary Vietnamese
narration is not misread as a decision or a preference.
Co-authored-by: nguyenha935 <nguyenha935@users.noreply.github.com>
|
||
|
bilogic
|
e71d6a5519 | in groups, resolve group credentials first, then sender credentials (#1482) | ||
|
Clark Cant
|
5ca433b8ac |
fix: persist mid-loop compaction to stop re-compaction loop and revive episodic
The v3 pipeline compacts session history mid-loop (prune_stage + final-request guard) but only mutates the run's message buffer, never the session store. Each turn reloads full history and re-compacts from scratch: message_tokens climb 129k->156k across turns while every turn compacts back down to ~60k. The lossy compaction differs per run, degrading the agent. The same missing persistence stalls episodic memory: the cumulative compaction count never advances, so the episodic worker's idempotency key (sessionKey:count) is pinned and every cycle after the first is skipped. Observed on live traffic: 8 run.completed since deploy, 0 new episodic. Fixes, all reusing existing machinery (no new store methods, no migrations): - Bug A: emitSessionCompleted reads cumulative GetCompactionCount (matching the legacy v2 path) instead of the per-run counter that resets to 0. - Bug B/anti-loop: finalize passes state.Prune.MidLoopCompacted into maybeSummarize; under pressure it lowers the trigger to a unit-aligned threshold (compactionInputCap - overhead, same MaxRequestShare the guard uses) so the compaction is PERSISTED via the existing TruncateHistory + IncrementCompaction path. Defensive floor prevents over-compaction on pathological config; tool-result-only bloat still skips (history-only). - Bug C: SourceID embeds the count (sessionKey:count) so the eventbus dedup key advances per compaction cycle instead of swallowing rapid same-session turns within the 5m TTL. Tests: episodic compaction, maybe_summarize pressure, request budget. go build (PG + sqliteonly), go vet, go test -race all green. |
||
|
thotam
|
503909d3b6 |
fix(sessions): store last-call context tokens instead of run-cumulative total
SetLastPromptTokens received TotalUsage.PromptTokens — the sum of prompt tokens across every think→act→observe iteration of a run — instead of the final iteration's own usage. A tool-heavy run inflated the stored value by the iteration count, so the sessions list showed e.g. 901K "context used" against a 258K window, and maybeSummarize over-triggered compaction once the inflation exceeded the 40% overhead clamp (observed: 83 compactions on a 4-message session). Track the final iteration's usage separately (ThinkState.LastUsage) and persist its actual context size: prompt tokens + cached segments (Anthropic reports input_tokens excluding cache read/creation) + output tokens (the final reply joins history, so it occupies the next request's prompt). The run-cumulative total still feeds AccumulateTokens for billing. |
||
|
thotam
|
0c1ededc92 |
feat(webhooks): return per-call usage breakdown with provider/model/cost (#1421)
feat(webhooks): per-call usage breakdown with provider/model/cost (#1421) |
||
|
|
db69ccf882 |
tracing: nest tool calls under LLM spans, emit hook spans, fix usage_events FK (#1415)
- Reparent tool_call spans under their producing llm_call span via RunState.CurrentLLMSpanID so a model turn's trace visibly contains the tool calls it triggered. - Wire internal/hooks EmitHookSpan into the dispatcher writeExec so every hook execution (pre/post tool use, etc.) produces a trace span with input, console output, decision, error, and duration for agent troubleshooting. - Fix usage_events_span_id_fkey violations: usage events were inserted synchronously referencing a span flushed ~5s later, silently dropping token/cost data. Route them through the collector so they flush after spans in the same cycle. No schema migration; FK preserved. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ナムandCollective Developer
|
84fab59226 |
fix(memory): guide recency recall tool calls (#1412)
* fix(memory): guide recency recall tool calls * fix(tools): align memory and vault builtin visibility --------- Co-authored-by: Collective Developer <man@collective.dev> |
||
|
|
ba621391c2 |
feat(telegram): trigger-words, channel posts, message edit & topic posting (#1383)
- Fix image_generation nil-pointer crash on codex agents: the sentinel now carries a name-only Function so the many sites reading td.Function.Name never nil-deref; codex_build still branches on Type. - Agent-declared trigger words in IDENTITY.md wake the bot in groups without an @mention (whole-word, Cyrillic-aware; text + caption), cached per-agent 60s. - channel_post support with a synthetic sender + a recover() guard so a malformed update can't crash the gateway. - message tool action=edit (editMessageText + editMessageCaption fallback), targeting the replied-to message via reply_to_message_id. - message tool topic=<name> posts into a named forum topic; topics learned from forum_topic_created into channel_contacts and resolved by name. Co-authored-by: skensel <skensel@MacBook-Pro-skensel.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Bruno ClermontandBruno Clermont
|
d3c04ab3e5 |
feat(bootstrap): skip built-in USER.md for predefined agents with USER_PREDEFINED.md (#1397)
When a predefined agent has an operator-authored USER_PREDEFINED.md, the built-in per-user USER.md template is no longer seeded or injected into the system prompt. This gives operators full control over user-context in the prompt and stops the per-turn name/timezone profile nag. Open agents and predefined agents without USER_PREDEFINED.md are unaffected. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Bruno ClermontandBruno Clermont
|
6bafe54b08 |
fix(agent): filter per-tenant disabled tools from system prompt (#1396)
The system prompt's Tooling section previously listed tools even when they were disabled for a tenant and already stripped from the API tools parameter, confusing the LLM into thinking it could use them. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Duc Nguyen
|
aa15317d5b |
fix: resolve Zalo group by name + notify origin on failed forward (#1395)
* fix: resolve Zalo group name to real chat ID, notify origin on failed forward Two related root causes behind "forward message to group by name" silently failing while the agent reports success: 1. No tool let the agent resolve a group's display name (e.g. "Ban Dieu Hanh") to the chat ID the message tool actually requires. sessions_list only exposes session keys (numeric group IDs), never human-readable names, so the agent had no reliable way to turn a name into a real target and ended up passing the display name itself as `target`. Adds zalo_list_groups, wrapping the already-used (dashboard picker) protocol.FetchGroups behind the same optional-interface pattern as list_group_members/GroupMemberProvider (GroupListProvider on channels.Manager, gated to zalo_personal via RequiredChannelTypes). 2. When the resulting send fails downstream (e.g. Zalo rejects a bad chat_id), dispatchOutbound only ever retried/notified media failures on the same (already-broken) destination, and dropped text-only failures entirely — even though message.go's own postCrossTargetNotice comment states forwards must never announce a fake delivery. Because the bus publish is fire-and-forget, the tool had already returned "sent" and announced success to the origin chat before the real send was even attempted. message.go now tags cross-target forwards with origin channel/chat in OutboundMessage.Metadata; dispatchOutbound uses it to notify the ORIGIN chat with the real failure instead of silently dropping it or retrying against the same invalid target. * test: cover forward-origin metadata tagging and dispatch failure notice Extracts dispatchOutbound's error branch into handleSendFailure so it can be unit tested without driving the consumer loop/goroutine, and adds coverage for: forward failures notifying the origin chat (not the broken destination), pre-existing non-forward media/text-only behavior staying unchanged, message.go tagging cross-target group forwards with origin metadata alongside group_id, and the new zalo_list_groups tool/Manager delegator. |
||
|
nguyenha935
|
f826738ee6 |
feat: add team work classification routing (#1379)
Approved by github-maintain bot. Clean team work classification feature with comprehensive tests and i18n. |
||
|
Bruno ClermontandBruno Clermont
|
cde8940501 |
fix(agent): fix race in TestMaybeSummarize_LogsTriggerDecisionOverThreshold (#1347)
maybeSummarize spawns a background goroutine that logs via slog.Info before calling the LLM provider -- correct, intentional async design, not a bug. The test read the shared log buffer (buf.String(), inside captureSlog) immediately after maybeSummarize returned, i.e. right after the goroutine was merely spawned, not after it finished logging -- an unsynchronized concurrent read/write on bytes.Buffer, caught by go test -race. This was surfaced by an unrelated PR's CI run (nextlevelbuilder/goclaw#1346), confirmed unrelated to that PR's actual changes. Moved the existing done-channel wait inside captureSlog's closure, so the buffer read only happens after the background goroutine has signalled it reached provider.Chat -- which happens strictly after the slog.Info write on this path. Proper happens-before via the channel, no sleep/poll, no production code changed. Verified 10/10 passes under go test -race -count=10, plus full package regression run clean. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
nguyenha935andGoClaw Operator
|
4d3c6bcb05 |
feat: add Telegram manager channel permissions (#1339)
Co-authored-by: GoClaw Operator <operator@goclaw> |
||
|
|
90e7035100 |
fix(mcp): tenant isolation, tool policy engine, and prompt preview fixes (#1333)
* fix(prompt): remove duplicate Team Members section from system prompt The TEAM.md context file already provides the Members section with better formatting. The system prompt section was redundant and inconsistently formatted. - Remove buildTeamMembersSection() call from system prompt building - Remove unused buildTeamMembersSection() function * fix(mcp): wire store to manager for prompt preview tool visibility MCP manager needs database access to query configured servers for tool visibility in prompt preview. - Add SetStore() method to MCP manager - Wire pgStores.MCP to manager after initialization - Add debug logging for MCP initialization flow * fix(systemprompt): hide Tooling section when agent has no tools The Tooling section header and boilerplate were displayed even when the agent had no tools available. Add early return in buildToolingSection() to skip the section entirely when toolNames is empty. This reduces prompt noise for agents with no tool access. * feat(mcp): cache tool descriptions for prompt preview visibility MCP tools now show descriptions in prompt preview without requiring a live server connection. - Add CacheToolDescriptions() method to MCPServerStore (PG + SQLite) - Cache tool descriptions in settings['tool_cache'] when server connects - Use cached descriptions in ListToolsForAgent (fallback: hints → cache → global) - Descriptions are auto-populated from live server manifest on connection - Admins can still override via tool_hints in server settings * test(mcp): add CacheToolDescriptions to MCPServerStore test fakes Commit 5ce410b6 added CacheToolDescriptions() to the MCPServerStore interface but missed updating test mock implementations, breaking go vet across internal/mcp, internal/agent, internal/http, and internal/channels/bitrix24. Add no-op implementations matching each fake's existing style. * fix(providers): enforce per-agent tool policy for Claude CLI provider The Claude CLI provider (stdio+MCP bridge) was not enforcing per-agent tool policy, unlike other providers where the policy-filtered tool list already drives the system prompt's Tooling section. Two gaps closed: 1. --disallowedTools was previously skipped entirely when no MCP config path was resolved, letting the CLI subprocess run with its full native toolset (Bash, Edit, Read, Write, Glob, Grep, WebFetch, WebSearch) regardless of agent policy. It's now unconditional and derived from the agent's actual allowed-tools list (state.Tool.AllowedTools), mapped to Claude CLI's native tool names. 2. The MCP bridge server executed any tool call without checking the calling agent's policy. It now resolves the agent's policy from context (via the existing HMAC-verified agent lookup) and denies calls to tools outside that agent's allowed set, logging security.mcp_bridge_denied on denial. Both gaps were closed using existing plumbing (PolicyEngine.WouldAllow, AgentData.ParseToolsConfig, the bridge context middleware) — no new cross-cutting mechanism was introduced. * feat(web): show MCP/tool schemas in system prompt preview dialog The prompt-preview API response includes a separate `tools` field (the actual JSON schemas sent to the LLM as the tools API parameter) alongside `prompt` (the system prompt text), but the web UI only rendered `prompt`, silently dropping the tools list. Add a collapsible Tools section to both the full-screen System Prompt dialog and the inline agent-detail preview, showing tool count, name, description, and expandable parameter schema per tool. i18n keys added to en/vi/zh locales. * fix(prompt): render pinned skills on bootstrap turns Pinned skills are documented (web UI copy) as "always inlined in the system prompt", but the entire Skills section was gated behind !cfg.IsBootstrap, so pinned skill XML never appeared on bootstrap turns (first message of a session) despite the promise. Separate pinned-skill rendering from bootstrap-suppressed guidance: - Bootstrap + pinned skills present: render pinned XML only, no search/manage guidance (which stays suppressed as before) - Non-bootstrap: unchanged behavior - Minimal/none modes: pinned skills always render regardless of bootstrap state Add regression tests covering all four prompt modes on bootstrap turns, plus a non-bootstrap guard confirming existing behavior is preserved. * fix(skills): resolve managed skills directory per-tenant, not master-only skills.Loader was wired at startup to scan a single fixed directory (the master tenant's managed-skills dir), making any skill belonging to a non-master tenant invisible to both pinned-skills prompt resolution and skill_search/use_skill, regardless of DB visibility settings. - Loader now resolves the calling tenant's managed-skills directory per-call via context (store.TenantIDFromContext), never enumerating other tenants' directories - Skill cache is now tenant-keyed to prevent slug collisions and cross-tenant cache leaks across tenants using the same skill slug - gateway_setup.go passes the root data dir instead of a pre-resolved master-tenant path Write-side tooling (skill_manage, publish_skill) was already correctly tenant-scoped per-operation — no changes needed there. Added TestLoader_ManagedSkills_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's content, including after cache population from a different tenant's lookup. Known follow-up (not in this commit): skill_search's BM25 index is still a single process-global index shared across tenants, which is a related but separate cross-tenant search-result leak requiring its own scoped fix (per-tenant index maps + threading tenant context through ensureIndex/rebuildIndex). * fix(tools): scope skill_search BM25 index per-tenant SkillSearchTool held a single process-global BM25 index built once from whichever tenant's context first triggered ensureIndex, then reused for all subsequent Execute() calls regardless of caller — leaking one tenant's skill search results into another's, the search-path counterpart to the managed-directory bug fixed in 7b4668ad. - index/lastVersion are now keyed per-tenant (map[uuid.UUID]*tenantIndexState) - ensureIndex resolves the calling tenant from context and only builds/reads that tenant's index entry, never touching another tenant's cached state - Builtin/bundled skills remain visible in every tenant's index (Loader already merges those tiers correctly per 7b4668ad) Loader.Version() remains a single global counter — a version bump in one tenant causes unnecessary rebuilds in others but does not cause cross-tenant leakage, an acceptable tradeoff to avoid scope creep. Added TestSkillSearchTool_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's search results, including after cache population from a different tenant. * fix(tools): fix group-spec expansion in tool policy engine PolicyEngine.registry was only ever set via SetRegistry(), which was never called in production (only in one test) — so pe.registry was permanently nil in production. Every group-expansion helper (applyProfile, intersectWithSpec, unionWithSpec, subtractSpec, expandSpec, matchDenySpec, filterByCapability) silently dropped any "group:*" spec entry instead of expanding it when registry was nil. Concretely: "group:mcp" (auto-injected into agentToolPolicy.AlsoAllow for any agent with MCP tools) never resolved to real tool names, so MCP tools connected successfully and appeared in prompt text (which reads the registry directly, bypassing PolicyEngine) but were never included in the actual ChatRequest.Tools payload sent to the LLM — confirmed live via mcp.agent.tools_loaded tools=6 immediately followed by mcp.filtered_tools mcp_defs_count=0 in the same request. This affects any agent relying on group-based grants, not just MCP. PolicyEngine is a shared/global singleton used concurrently across all agents (constructed once at gateway startup), so mutating a registry field per-call would be a data race. Fix instead threads the registry as an explicit parameter from FilterTools down through all internal group-expansion helpers, and adds IsDenied/WouldAllow registry parameters, removing the dead SetRegistry() mechanism entirely. Also fixes group expansion for the per-user-MCP-tools path: FilterTools is sometimes called with a userToolOverlay wrapping a *Registry rather than a *Registry directly; added Unwrap() to userToolOverlay so the new registry-resolution logic works for both cases. Added 4 tests proving group expansion works via the threaded parameter alone (no SetRegistry): plain registry allow, userToolOverlay allow, deny-side group expansion, and the WouldAllow bridge-server path. Blast radius note: this restores intended access for every agent configured with group:* specs (group:mcp, group:vault, group:goclaw, group:coding, etc.) that were silently inert before. Existing agent configs relying on group grants will gain the tool access they were nominally already configured for. * fix(mcp): cache tool descriptions from pool-connected servers too 5ce410b6 added tool-description caching (for prompt-preview visibility) only inside connectServer. connectViaPool — the separate connect path used when MCP connections go through the shared pool — never got the same caching hook, even though it shares the same underlying connectAndDiscover wire handshake. Confirmed live: cloudflare/docker connected via connectViaPool and received real descriptions over the wire, but prompt preview still showed blank descriptions because this path never wrote to the cache. Also removes the temporary mcp.connect.raw_tool debug log added earlier this session for diagnosing the same issue — no longer needed now that the root cause is fixed. * chore(skills): remove temporary pinned-skills diagnostic logging Confirmed live: pinned skills (caveman, infra-ansible-knowledge) now resolve correctly end-to-end for tenant-scoped agents. Debug logging added to trace the resolution chain is no longer needed. * fix(tools): deny always wins over AlsoAllow group grants AlsoAllow's unionWithSpec could reintroduce a tool explicitly listed in Deny, since it added tools back from allTools without re-checking deny specs. Previously masked because AlsoAllow's group-expansion was also broken (fixed in f7af95de this session) — group specs silently expanded to nothing, so this ordering bug never manifested. Now that group expansion works, an admin-denied tool that's also reachable via a group:* AlsoAllow entry (e.g. group:mcp) would silently reappear. Re-apply deny-spec subtraction as a final step after AlsoAllow union, for both global and per-agent policy, so deny always wins regardless of which allow mechanism tries to add a tool back. Added tests proving global and per-agent Deny correctly override an overlapping AlsoAllow group grant, while sibling non-denied tools in the same group remain allowed. * fix(mcp): enumerate cached tools instead of wildcard placeholder in prompt preview ListToolsForAgent (the prompt-preview path) collapsed any server with an empty ToolAllow (unrestricted grant — the common case) into a single "server__*" placeholder entry, even when tool_cache already had every real tool name and description from connect time (5ce410b6, 8ffd67b9). This meant agents with unrestricted MCP server access never saw individual tool names or descriptions in prompt preview, forcing trial-and-error tool usage. When ToolAllow is empty and tool_cache is populated, enumerate every cached tool (skipping any explicitly denied) and emit one MCPToolPreviewInfo per tool, matching the construction logic already used for the ToolAllow-non-empty case. Falls back to the single placeholder only when tool_cache is also empty (server never connected). The live (non-preview) conversation path, buildMCPToolDescs, does not have this bug — it resolves tool identity from the live connected registry, never from ToolAllow, so no placeholder shortcut exists there. Added tests covering both the cache-populated enumeration case and the no-cache placeholder fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(prompt): filter alias re-injection and add MCP schemas to preview ToolDefs Prompt-preview's tools: schema array (PreviewResult.ToolDefs) had two bugs, both preview-only — confirmed live conversations already use correctly-filtered tool payloads via buildToolsPayload/PolicyEngine.FilterTools, unaffected by either bug: 1. Alias re-injection iterated ALL registry aliases globally with no check against the deny-filtered toolNames list, letting denied tools reappear via their alias name (e.g. a denied canonical tool still showing up as its Claude-Code-compat alias like Bash/Edit/Write/Read). Now skips any alias whose canonical tool isn't in the filtered set. 2. MCP tool descriptions (from the store-based, connection-free ListToolsForAgent path) only ever fed the prompt TEXT section, never got converted into ToolDefinition schema objects — so the tools: array never showed MCP tools at all, even when the MCP section of the prompt text correctly listed them. Now appends a ToolDefinition per MCP tool with name+description populated and a placeholder {"type":"object"} Parameters schema, documented as preview-only (real parameter schemas require a live MCP connection, only available during an actual conversation turn). Added tests proving denied-tool-alias exclusion and MCP tool inclusion in preview ToolDefs. * feat(skills): inline full content for pinned skills instead of pointer-only Web UI documented "Pinned skills are always inlined in the system prompt", but BuildPinnedSummary was just BuildSummary with an allowlist filter — same as the general searchable skill list: name/description(truncated)/location pointer only, requiring use_skill+read_file round trips to get actual content. No code path inlined real SKILL.md content for pinned skills specifically. BuildPinnedSummary now reads and inlines full SKILL.md content (frontmatter stripped) per pinned skill inside <skill_instructions> tags. Per-skill (10000 bytes) and total (30000 bytes) size caps fall back to the original pointer-only format with a note when a skill is too large to inline, so oversized skills degrade gracefully instead of blowing the prompt budget. Added tests: full-content inlining, size-cap fallback, and tenant isolation for the new inline path (mirroring the existing managed- skills tenant isolation test). * fix(mcp): real parameter schemas and full policy enforcement in prompt preview Three interconnected fixes to prompt preview, none affecting the live conversation path (which was already correct): 1. Real MCP parameter schemas instead of a useless empty placeholder. tool_cache previously stored only name+description; extended to also capture the real JSON Schema from the MCP server's tools/list response (CachedToolInfo{Description, Parameters}) at connect time, for both direct-connect and pool-connect paths. Preview now shows complete, real input schemas instead of {"type":"object"} with no properties — verified against actual wire data from a live cloudflare MCP server showing genuinely rich schemas (zone_id, type, name, content, ttl, proxied, all typed with descriptions and correct required arrays) that were previously being discarded. Backward-compatible: old-shape cache entries degrade gracefully to description-only rather than crashing, self-healing on next connect. 2. Global tool deny now enforced in preview. BuildPreviewPrompt previously hand-rolled a partial policy reimplementation (per-agent deny only, explicitly skipping the full PolicyEngine "because runtime state isn't available in preview") — but PolicyEngine.WouldAllow already handles this per-tool-name without needing channel context. A tool denied via the global config (not per-agent) would appear in preview despite being correctly denied in every real conversation. Preview now calls WouldAllow per candidate tool, with a graceful per-agent-deny-only fallback when no PolicyEngine is wired (e.g. in tests). 3. MCP tools now also subject to the same policy check — previously the MCP tool supplement (store-based, connection-free tool listing) added MCP tool names unconditionally, bypassing WouldAllow entirely, so a denied MCP tool could still appear in preview. Added tests for all three: real-schema presence, global-deny exclusion for both core and MCP tools, and backward-compat cache handling. * fix(prompt): remove redundant per-tool MCP enumeration from prompt text Now that MCP tool schemas in the tools: API parameter are real and complete (1290d4f1), the ## MCP Tools prompt-text section's per-tool "- mcp_x__y: description" enumeration is pure duplication with zero added value — the model already gets each tool's real schema (including description) via tools:. buildMCPToolsInlineSection now keeps only the behavioral instructions that aren't expressible via JSON schema and thus aren't duplicated: prefer-MCP-over-core-tools guidance, and the optional-parameter guidance (don't guess/fill optional fields). The per-tool name+ description enumeration loop is removed. Section still only appears when the agent has MCP tools (len(cfg.MCPToolDescs) > 0, unchanged gate). Updated tests to assert the enumeration is gone while the behavioral instructions remain; ToolDefs assertions are now the authoritative check for MCP tool allow/deny filtering behavior (prompt text no longer enumerates names at all). * test(agent): update TeamContextInjection test for removed Team Members section TestBuildSystemPrompt_TeamContextInjection asserted the presence of a 'Team Members' prompt-text section that was intentionally removed in 71d33180 (duplicate of the canonical TEAM.md-context-file Members section, which has better formatting). The test was never updated to match, causing it to fail on every run since. Moved the assertion from wantIn to wantNotIn for the 3 affected subtests -- BuildSystemPrompt correctly no longer renders team-member roster info directly; that info now comes exclusively from the TEAM.md context-file mechanism, outside this test's isolated scope. * fix(prompt): resolve real registry for WouldAllow calls in preview BuildPreviewPrompt's two WouldAllow calls hardcoded reg=nil, silently breaking group:* expansion (e.g. group:mcp) needed to resolve the AlsoAllow grant production actually uses to grant MCP tool access (resolver_helpers.go's agentToolPolicyWithMCP injects AlsoAllow: ["group:mcp"]). With reg=nil, WouldAllow could match literal tool names fine (the 18 core/static tools) but could never resolve group-based grants, so every MCP tool silently failed WouldAllow and was excluded from preview -- confirmed live via curl: 18 tools returned, zero mcp_* ones, for an agent with genuinely working MCP access in real conversations. Live conversations were never affected -- internal/mcp/bridge_server.go's WouldAllow call already correctly passes a real registry. Fix resolves a real *tools.Registry from deps.ToolLister via tools.ResolveConcreteRegistry (the same helper used at the live call site), passing it to both WouldAllow calls instead of nil. Falls back to nil gracefully for test mocks that don't implement the full ToolExecutor interface, preserving existing test behavior. Added a test proving an MCP tool granted via the exact production AlsoAllow: ["group:mcp"] pattern is now correctly included in preview ToolDefs, where the old reg=nil bug would have silently excluded it. * fix(prompt): use literal deny check for MCP tools in preview, not group expansion The MCP-tools policy gate added in 1290d4f1 called WouldAllow with a real registry (per 9b5fd2eb), which requires group:mcp expansion against that registry to grant access via the production AlsoAllow: ["group:mcp"] pattern. But MCP tools are only ever registered into ephemeral per-agent registry clones at live connection time (manager_connect.go) -- never into the shared/global registry preview uses. group:mcp always resolved empty in preview's connection-free context, so WouldAllow denied every MCP tool -- confirmed live via tool-name diff: live conversations correctly included all 6 MCP tools, preview included zero. MCP access-granting is already correctly handled by ListToolsForAgent's own per-server tool_allow/tool_deny grant logic (confirmed working correctly earlier this session). The preview gate only needs to catch the narrower case of a literally-denied tool name via global/per-agent policy config -- it never needed group expansion. Replaced WouldAllow with IsDenied(nil, name, agentPolicy), which forces a pure literal-name match with zero registry dependency, matching the existing usage pattern already established elsewhere in policy.go. This class of bug cannot recur: there's no registry-passing code path left in this check to silently reintroduce group-expansion dependence. Added a test proving MCP tool inclusion in preview is independent of group-expansion outcome (no AlsoAllow: group:mcp needed for a non-denied tool to appear). Also reverts the temporary loop.filtered_tool_names/ preview_prompt.filtered_tool_names diagnostic logging used to capture the live-vs-preview tool-name comparison that diagnosed this bug. * fix(http): preserve real MCP parameter schemas through HTTP preview adapter mcpPreviewAdapter.ListToolsForAgent (the HTTP-layer glue converting mcp.MCPToolPreviewInfo to agent.MCPToolPreviewInfo for BuildPreviewPrompt) only copied RegisteredName and Description, silently dropping Parameters -- a bug present since this adapter was introduced (2499d0be/7e250244), unrelated to today's other MCP preview fixes. This was masked until d5fc6344 fixed MCP tools being excluded from preview entirely (a separate bug) -- once MCP tools started appearing again, this pre-existing adapter gap became visible: tools showed up correctly, but always with the bare {"type":"object"} placeholder instead of their real cached schema (confirmed live: update_dns_record missing its 7 real properties). One-line fix: copy Parameters through in the adapter's struct literal. Added a regression test constructing a real *mcp.Manager with populated tool_cache, asserting the adapter's output preserves specific real schema properties (not just non-nil Parameters) -- verified this test fails without the fix and passes with it. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Duc Nguyen
|
747b58d24c |
fix(usage): repair cost analytics and display precision (#1330)
fix(usage): repair cost analytics and display precision - Automatic OpenRouter pricing sync with cost backfill for traces/snapshots/events - Live usage data merging for current-hour dashboard accuracy - 2-decimal API cost formatting across usage/overview pages - Comprehensive test coverage (PG, SQLite, HTTP, UI) Merged by github-maintain automation. |
||
|
Bruno ClermontandBruno Clermont
|
2499d0be78 |
fix(mcp): show configured MCP tools in prompt preview (#1321)
* fix(mcp): show configured MCP tools in prompt preview MCP tools were only visible in prompts if currently loaded in registry (from active sessions). Prompt preview showed empty MCP tools section. Solution: Query MCP store for configured servers and tool lists. Show tools that would be available, not just currently loaded. - internal/mcp/manager.go: add ListToolsForAgent() for store-based tool discovery - internal/agent/preview_prompt.go: supplement live registry with store tools - internal/http/agents.go: add mcpPreviewMgr field and setter - internal/http/agents_prompt_preview.go: create adapter bridging mcp.Manager to preview - cmd/gateway.go: wire up MCPPreviewAdapter during setup * fix(mcp): load MCP servers from database at startup, not config file Single source of truth: MCP servers are now loaded from mcp_servers table at gateway startup, instead of from the config file. This ensures that MCP servers configured via web UI are automatically available without requiring config file updates. - cmd/gateway_setup.go: remove config fallback, add initMCPFromDB() helper - cmd/gateway.go: call initMCPFromDB() after store initialization - internal/mcp/manager.go: add SetConfigs() for runtime configuration Now mcpMgr will be non-nil when MCP servers are in the database, enabling MCP tools in prompt preview. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Bruno ClermontandBruno Clermont
|
7d95e4ae01 |
fix: resolve duplicated tenant-path segment in team shared-workspace resolution (#1322)
Root cause: internal/agent/loop_context.go re-applied tenant scoping (TenantLayer/TenantID) on top of l.dataDir, which was already tenant-scoped upstream in internal/agent/resolver.go, producing paths like /app/workspace/tenants/<slug>/tenants/<slug>/teams/<teamID> instead of the correct /app/workspace/tenants/<slug>/teams/<teamID>. Also consolidates all previously-duplicated tenant-path-joining logic (config.TenantDataDir, config.TenantWorkspace, tools.TenantLayer, workspace.tenantPath) into a single canonical config.TenantScopedDir function in internal/config/tenant_paths.go, with all other implementations now delegating to it. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Zezae Oh
|
606c8e60e1 | feat(usage): preserve cache tokens in usage events (#1311) | ||
|
Zezae Oh
|
968745bd16 | feat(providers): enable Codex prompt cache controls (#1310) | ||
|
nguyenha935andGoClaw Operator
|
969a5879ae |
fix: unify no reply detection on dev (#1303)
Co-authored-by: GoClaw Operator <operator@goclaw> |
||
|
SYNITYandDangTinh311
|
23a4b7407c |
fix(bitrix24): OpenLine greeting and MCP provisioning (#1284)
* fix(bitrix24): skip join greeting in open channels [B24:2794] * wip: rework bitrix24 mcp provisioning + skills prompt section (tam thoi, se lam lai) [B24:2794] --------- Co-authored-by: DangTinh311 <dangtinh31193@gmail.com> |
||
|
nguyenha935andGoClaw Operator
|
64596bf0c1 |
fix: preflight compact v3 context before model calls (#1297)
Run pruning before ThinkStage so oversized histories compact before the provider request is built. Persist V3 last_prompt_tokens with the post-flush persisted message count so post-run compaction calibration has current usage data. Tests: go test ./internal/pipeline -run "TestNewDefaultPipeline_PrunesBeforeThink|TestFinalizeStage_UpdateMetadataReceivesPersistedMessageCount" -count=1; go test ./internal/agent -run "^$" -count=1 Co-authored-by: GoClaw Operator <operator@goclaw> |
||
|
Thanh_Dang
|
669eed6990 |
perf(agent): cache BuildFilteredTools result per run (#1282)
* fix(pipeline): defer non-tool messages until all tool results are emitted Multi-tool responses could interleave synthetic user messages (loop warnings, nudges) between tool results, breaking OpenAI-compatible providers that require contiguous tool_result grouping. Fix both sequential and parallel tool execution paths in tool_stage.go to defer non-tool messages until after all tool results are appended. Fix sanitizeHistory lookahead in loop_history_sanitize.go to reorder interleaved non-tool messages in persisted session history. Closes #1177 * perf(agent): cache BuildFilteredTools result per run All filtering inputs (policy, disabled tools, bootstrap, channel, orchestration mode, user MCP tools) are fixed for the lifetime of a run. Only the final iteration differs (strips all tools). Cache the tool definitions after the first call and reuse for iterations 0..maxIter-1, eliminating N-1 redundant 7-step policy evaluations per agent run. |
||
|
Zezae Oh
|
505b7a9d99 |
fix(agent): recover interrupted runs to failed on gateway startup (#1272)
A run records its terminal run.status (completed/failed/cancelled) only when the agent loop returns. If the gateway process is killed mid-run (restart, crash, or SIGKILL after a slow graceful shutdown), the terminal event is never emitted, so the run's last run.status item stays "started" forever — it shows as perpetually running in the timeline and is never counted as failed. Mirror the cron scheduler's startup reset (recomputeStaleJobs flips stale 'running' jobs to 'interrupted'): add RunTimelineStore.RecoverInterruptedRuns, invoked once on gateway startup. It finds runs that have a started run.status but no terminal sibling and appends a terminal failed run.status item, marked interrupted in metadata so it stays distinguishable from a genuine agent failure. Running only at startup means nothing from before is still executing, so there is no false-positive risk — unlike a periodic sweep (see the deliberately-disabled tracing stale-trace loop). - store: RecoverInterruptedRuns on the interface + PG and SQLite impls - cmd/gateway_managed: invoke once after the timeline recorder is built - tests: SQLite store test — orphan recovered, completed run untouched, idempotent on re-run |
||
|
Thanh_Dang
|
68ad5953d7 |
fix(pipeline): defer non-tool messages until all tool results are emitted (#1270)
Multi-tool responses could interleave synthetic user messages (loop warnings, nudges) between tool results, breaking OpenAI-compatible providers that require contiguous tool_result grouping. Fix both sequential and parallel tool execution paths in tool_stage.go to defer non-tool messages until after all tool results are appended. Fix sanitizeHistory lookahead in loop_history_sanitize.go to reorder interleaved non-tool messages in persisted session history. Closes #1177 |
||
|
Zezae Oh
|
eaa011dae3 |
feat(agent): honor per-agent tools.rate_limit_per_hour (#1263)
The agent config UI (tools-profile-section) and i18n already expose a per-agent "Rate Limit (per hour)" field saved into tools_config, but the backend ignored it — the tool rate limiter only ever used the global tools.rate_limit_per_hour. Wire the field through: ToolPolicySpec gains RateLimitPerHour; the agent loop threads it into the tool-execution context; the registry passes it to the limiter, which uses it in place of the global max when > 0 (0 inherits the global). |
||
|
Zezae OhandClaude Opus 4.8
|
d8cc9542f6 |
fix(agent): also strip bare <invoke> tool-call blocks (no wrapper) (#1261)
Follow-up to #1260. A model (notably the claude-cli proxy under a degraded session) sometimes emits a tool call as a bare <invoke name="...">...</invoke> block with no <function_calls> wrapper. fullToolCallBlockPattern only matched the wrapped form, so the bare block slipped through to the tag-only strip, leaking the inner <parameter> command text into the reply while the tool never ran. Match and remove complete bare <invoke> blocks too (after the wrapped form), add "<invoke name=" as a detection indicator, and log the dropped tool name(s). Partial/unterminated artifacts still fall through to the existing tag strip. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Zezae OhandClaude Opus 4.8
|
97914b16ae |
fix(agent): strip whole text-encoded tool-call blocks, not just tags (#1260)
stripGarbledToolXML removed only the tags of a model-emitted <function_calls> block, leaking the inner <parameter> argument values into the user-facing reply, and the dropped call was completely silent. Detect a complete tool-call block, remove it whole, and log the attempted tool name(s) at WARN so the no-op is diagnosable. Partial artifacts still fall through to the existing tag-level strip. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
nguyenha935andnguyenha935
|
9ec542251d |
fix(agent): configure and instrument compaction timeout (#1259)
Co-authored-by: nguyenha935 <208228297+nguyenha935@users.noreply.github.com> |
||
|
|
bd5adc61c8 |
feat(bitrix24): imbot.v2 migration, 2-way media, openline sender-tag echo, and hardening (#1236)
* refactor(bitrix24): rename "Path B" framing to maintainer-specified naming [B24:2794] Per maintainer hard rule #10 (no generic "Path A/B" framing) from PR #1061 review. The Bitrix24 MCP auto-onboard flow is Bitrix-specific glue ("Bitrix24 OAuth -> existing mcp_user_credentials bridge"), NOT a generic MCP architecture pattern. Naming convention applied consistently: - First mention per file: full "Bitrix24 OAuth -> existing mcp_user_credentials bridge" (matches maintainer comment verbatim). - Subsequent mentions in same file: shortened "mcp_user_credentials bridge". - Test/log context referencing literal endpoint /api/auto-onboard: keep "auto-onboard" reference (it's the actual API endpoint name). Changes are documentation-only: - Rename in code comments + test descriptions + plan docs. - Clarify framing in mcp_client.go + provisioner.go doc comments to emphasize Bitrix-specific glue (not generic MCP infra). - Reuse existing mcp_user_credentials table + MCPServerStore methods (no schema / store / abstraction change). Files: - cmd/gateway.go (factory registration doc) - internal/channels/bitrix24/{channel,factory,mcp_client,provisioner}.go - internal/channels/bitrix24/{mcp_client,provisioner}_test.go - plan/goclaw-mcp-integration.md (21 occurrences) Verified: go build + MCP-related tests pass (TestProvision*, TestInitMCPProvisioner*, TestMCPClient*). Phase 1 of Path C execution per plans/reports/decision-log-260519-1555-bitrix24-pr-fork-decision.md. * fix: confine outbound media paths to agent workspace [B24:2794] Tool MEDIA:<path> output reached channel file-upload sinks (Bitrix imbot.v2.File.upload, Telegram sendDocument, etc.) verbatim via parseMediaResult, with no workspace-boundary check. A malicious or buggy tool emitting MEDIA:/etc/passwd could exfiltrate arbitrary files to chat. Extract the EvalSymlinks+Rel containment from extractMediaFromContent into a shared confineToWorkspace helper and apply it at the parseMediaResult sink in processToolResult. Fixing at the source/egress boundary protects every channel at once rather than per-channel. Paths that escape the workspace are dropped and logged (security.media_path_rejected). Add TestConfineToWorkspace (boundary unit) and TestParseMediaResultConfinedToWorkspace (sink regression for H2). * feat(bitrix24): support inbound + outbound media via imbot.v2 File API [B24:2794] Bitrix24 channel was text-only; attachments were parsed but dropped. - Inbound: download chat files via imbot.v2.File.download (one-time URL), forward to the agent with MIME preserved (internal/channels/bitrix24/download.go). - Outbound: upload agent media to the chat via imbot.v2.File.upload (internal/channels/bitrix24/send_media.go). - Add BaseChannel.HandleMessageMedia to preserve MIME/filename through the bus. - Per-channel media_max_mb cap (default 20) applies to both directions. Tests: 92 pass (internal/channels/bitrix24 + internal/channels), go vet clean (PG + sqliteonly). * refactor(bitrix24): migrate messaging/bot-list/unregister to imbot v2 API [B24:2794] Move outbound REST calls to the imbot v2 family (keeps register on v1): - imbot.message.add -> imbot.v2.Chat.Message.send (fields.message shape, live-verified) - imbot.bot.list (+ legacy imbot.list fallback) -> imbot.v2.Bot.list; add botListRows to normalize the v2 {bots:[...]} envelope, legacy array, and id-keyed map forms - imbot.unregister -> imbot.v2.Bot.unregister Bot registration stays on v1 imbot.register: v2 imbot.v2.Bot.register changes the event-delivery model (per-event handler URLs -> eventMode), which would require rewriting the inbound event parser. No user-facing behavior change. Tests: bitrix24 package green; go vet ./... clean. * feat(bitrix24): route whisper via v1 SKIP_CONNECTOR + add v2 replyId [B24:2794] Bot was leaking HiddenMessage (whisper) replies to the external Zalo connector because every outbound call went through imbot.v2.Chat.Message.send, which has no equivalent of the v1 SKIP_CONNECTOR flag. Branch the outbound path on inbound visibility: whisper → imbot.message.add + SKIP_CONNECTOR=Y (v1, send_v1.go) public → imbot.v2.Chat.Message.send + fields.replyId (v2, send_v2.go) Pipeline: events.go parse data[PARAMS][PARAMS][COMPONENT_ID]=HiddenMessage into EventParams.IsHiddenMessage (form + JSON variants) handle.go set bitrix_visibility on InboundMessage.Metadata consumer forward visibility + message_id into OutboundMessage send.go resolveSendOptions + sendChunk dispatcher + shared callWithRateLimitRetry helper metadata_keys.go single source of truth for the keys + values Defaults preserve pre-refactor behaviour: callers that don't populate bitrix_visibility still go through v2 public, and replyId is omitted unless a numeric bitrix_message_id arrives in metadata. Tests: TestParseEvent_FormURLEncoded_IsHiddenMessage (3 cases) TestParseEvent_JSON_IsHiddenMessage (3 cases) TestResolveSendOptions (8 cases) TestSend_BranchesOnVisibility (4 cases) * feat(bitrix24): openline sender-tag echo on replies [B24:2794] Openline sender-tag echo (this change): - Capture the connector sender tag ("[name #id]:" or "[name] #id:") from inbound openline group messages, strip it from the body the agent sees, and re-prepend the canonical "[name] #id:" form to the reply so the Open Channel connector routes the answer back to the right external user. - New sender_prefix.go helper (+ test) accepts both inbound layouts and emits one canonical form; scoped to messages carrying the tag, so plain chats are unaffected. - metadata_keys.go: MetaKeySenderPrefix; handle.go capture/strip/stash; gateway_consumer_normal.go forwards the key; send.go prepends it on the first chunk before chunking. Bundled bitrix24 channel-core work already on this branch: - handle.go: @mention is the sole trigger for both staff and connector customers; unmentioned traffic is dropped (was: drop all connector msgs). - isGroupMessageType: treat SONET_GROUP "B" as a group. - handle_test.go, mcp_client_test.go: cover the above. * feat(bitrix24): accept colon-less openline sender tag, echo [name] #id [B24:2794] The Open Channel connector dropped the trailing colon from its sender tag: inbound now arrives as "[Name] #id <msg>" (was "[Name] #id: <msg>"). The id-bearing patterns required the colon, so the tag fell through to the name-only branch and the reply echoed "[Name]" — dropping the #id the connector needs to route the answer back. - sender_prefix.go: make the trailing ":" optional on both id layouts ([name #id] / [name] #id, with or without colon) and echo the canonical "[name] #id" (no colon) to match the connector's current format. Bare "[name]" (no id) still echoes "[name]" for Open Channel only. - handle.go: gate the bare name-only layout to Open Channel (isOpenChannel) so ordinary group chats starting with "[x] ..." are left untouched. - sender_prefix_test.go: cover colon/no-colon x id-inside/id-outside, the name-only openline case, and the non-openline no-op. * fix: security and robustness fixes from the bitrix24 channel review [B24:2794] - download.go: block redirect-based SSRF on inbound media. CheckRedirect re-validates each hop (http(s) only, reject private/loopback/link-local hosts, cap hops); the initial portal-domain pin is no longer bypassable via a 3xx to an internal service. Public-host redirects still allowed. - handle.go: extract/echo the openline sender tag only for Open Channel sessions (was: any group chat), removing bogus prefixes in CRM group chats and narrowing the forged-tag misroute surface. - loop_tools.go + loop_media.go: confine result.Media to the agent / team / tenant-allowed roots (new confineToAnyRoot) before a channel uploads it, so a prompt-injected out-of-workspace path (e.g. /etc/passwd) cannot exfiltrate, while legitimate cross-workspace media (team files, delegatee output) still flows. - send_media.go: bounded outbound read via io.LimitReader replaces the os.Stat + os.ReadFile pair, closing the TOCTOU size-cap bypass; cap a single message's outbound attachments at 10 (mirrors inbound). - register.go: paginate imbot.v2.Bot.list (limit/offset + hasNextPage, capped at 40 pages) so verify/lookup see bots past the first 50. - mcp_client.go: redact access_token / refresh_token / client_secret from an echoed MCP error body before it is logged or returned (+ test). * fix(security): validate resolved dial IP on Bitrix media redirects [B24:2794] The inbound media download redirect guard only string-checked the redirect hostname (isPrivateOrLoopback on req.URL.Hostname()), so a redirect to a public hostname that resolves to 127.0.0.1 / 169.254.169.254 / an RFC1918 address — or a DNS-rebinding swap between check and dial — still passed the guard and the client would connect. Reported in PR review. Add security.NewRedirectFollowingSafeClient: it follows redirects but validates the RESOLVED destination IP of every hop at dial time via net.Dialer.Control, reusing the existing blocked-CIDR list. The IP it checks is the IP actually dialed, so both redirect-to-internal and DNS rebinding are refused, while legitimate public CDN redirects still succeed. download.go now uses it instead of the hostname-string guard. Tests: deterministic dial-control table (loopback / link-local / private / multicast / unspecified / public, v4 + v6), malformed/non-IP addr, test bypass, loopback-dial-blocked client wiring, and redirect cap + scheme checks. * feat(bitrix24): per-participant Zalo openline identity from 3-token sender tag [B24:2794] Parse the connector's "[Name] #uid #msgId" sender tag so each external customer in a shared Open Channel group gets its own contact + USER.md instead of collapsing onto the connector proxy id. Identity minting is gated on IS_CONNECTOR=Y to reject operator forged tags. Echo back the msgId only ("#msgId") on replies; keep the legacy single-number and name-only layouts unchanged. Zero DB migration. - sender_prefix.go: parseOpenlineSenderTag() classifies 3-token / legacy / name-only - handle.go: synthetic senderID "openlines:{instance}:{chat}:{uid}" + participant_user_id metadata, gated on FromIsConnector - gateway_consumer_normal.go: deriveGroupUserID() routes participant -> per-person scope, group fallback otherwise - send.go: buildAddressMention numeric-id guard so synthetic ids don't emit invalid [USER=...] BBCode - MetaKeyMessageID kept as Bitrix MESSAGE_ID (drives v2 fields.replyId); connector msgId surfaced only via echo prefix --------- Co-authored-by: DangTinh311 <dangtinh31193@gmail.com> Co-authored-by: Chinh Dang <chinhdang@192.168.68.104> |
||
|
thotam
|
0ae55991bb |
feat(mcp): MCP OAuth 2.1 client for tool servers (#1196)
* feat(mcp): MCP OAuth 2.1 client — full implementation with tests
Implements a complete MCP OAuth 2.1 authorization flow for tool servers that
require user-delegated access, covering all layers from DB to UI.
- discovery.go: RFC 9728 protected-resource → RFC 8414 AS metadata → OIDC
fallback chain with 5-min in-memory cache and InvalidateCache()
- dcr.go: RFC 7591 Dynamic Client Registration with response size guard
- flow.go: PKCE (S256) authorization code flow — StartFlow(), ExchangeCode(),
ClientCredentials(), auto-cleanup of expired flows; carries AS issuer through
PendingFlow for status display
- refresher.go: OAuthTokenProvider with in-memory token cache, automatic refresh
on expiry, per-user vs global slot isolation, InvalidateCache/InvalidateServer
- migrations/000074 + SQLite schema: mcp_oauth_tokens with AES-256-GCM encrypted
access/refresh tokens, partial unique index for global vs per-user rows,
ON DELETE CASCADE from mcp_servers
- store.MCPOAuthTokenStore: Upsert, Get/GetUser, Delete/DeleteUser, and
DeleteServerOAuthTokens (purge all rows for a server)
- PostgreSQL + SQLite implementations
- POST /v1/mcp/oauth/start — discovery + optional DCR + PKCE redirect URL;
client_credentials completes server-side (no redirect) and returns completed=true
- GET /v1/mcp/oauth/callback — exchange code, persist token, publish WS event;
payload built via json.Marshal (no reflected XSS via error_description)
- GET /v1/mcp/oauth/status/{id}, DELETE /v1/mcp/oauth/token/{id} — admin-gated
- POST /v1/mcp/oauth/discover/{id} — on-demand discovery probe
- All outbound calls go through the SSRF-safe client with pinned IPs
- pkg/protocol/mcp_events.go: EventMCPOAuthComplete routed only to the initiating
user (admins in-tenant included); fail-closed across tenants
- getUserMCPTools() injects Authorization: Bearer from OAuthTokenProvider; on a
401 for OAuth servers it purges the cached token so the next turn re-resolves
- handleUpdateServer purges all OAuth tokens (global + per-user), drops the
refresher cache, and evicts the pool when a server's URL or OAuth config
(client_id / endpoints / grant_type / scope / auth_type) changes — so the
status UI and agent never use a token minted for the old resource/AS
- MCPOAuthDialog (WS-driven), unified user-credentials dialog, OAuth settings
fields; handles the no-redirect client_credentials completion
- internal/mcp/oauth/*_test.go: discovery cache, PKCE, DCR, refresher
- internal/http/mcp_oauth_test.go + mcp_update_oauth_purge_test.go: routes, auth
gating, WS event, purge-on-URL/OAuth-config-change
- tests/integration: store + encryption + tenant isolation, E2E start→callback,
DeleteServerOAuthTokens
- internal/gateway/event_filter_test.go, internal/agent/loop_mcp_user_test.go
* fix(mcp): return 400 on OAuth callback with code but missing state
The callback handler rendered a 200 HTML page whenever code or state was
absent. An auth code WITH a missing state is a malformed / CSRF-risk
callback (state is the CSRF token), so reject that case with HTTP 400.
A bare hit with neither code nor state (user opening the URL directly),
provider errors, and exchange failures keep their 200 HTML popup page.
Adds a status code parameter to writeCallbackHTML. Fixes the
TestOAuthCallbackMissingState integration regression while keeping
TestHandleCallbackMissingCodeAndState (no params -> 200) green.
* fix(mcp): scope-based OAuth auth + honor manual OAuth endpoints
Addresses the two MCP/OAuth security-review findings.
Finding 1 — authorization. mcp_oauth_tokens is tenant-scoped, but
start/status/revoke were gated only by requireAuth(RoleAdmin), an RBAC
role check, not tenant membership, so a RoleAdmin caller could act on a
tenant they don't administer. A blanket requireTenantAdmin would have
broken per-user self-service, which the UI exposes (the per-user
MCPUserCredentialsDialog shows an "Authorize" button to regular users for
their own credentials). Instead mirror the existing per-user MCP
credentials model (resolveTargetUserID in mcp_user_credentials.go):
- start/status/revoke accept any authenticated user; each handler calls
authorizeOAuthScope.
- a caller may manage their OWN per-user token (self-service); the
global/server token (user_id="") and other users' tokens require
tenant-admin (owner bypass), so a RoleAdmin that is not a tenant admin
is rejected.
- discover stays admin-only (it only previews AS metadata for a server).
Add a TenantStore dependency. Tests cover self-service, on-behalf-of-
another (403), and global-by-non-tenant-admin (403).
Finding 2 — honor manual OAuth config end-to-end. The UI sent use_dcr /
auth_endpoint / token_endpoint and the update path fingerprinted them for
purge, but handleStart always discovered + DCR'd and ignored them. Now:
- use_dcr=false (a *bool, so legacy/absent stays discover+DCR) skips
discovery/registration and uses the operator endpoints, SSRF-validated.
- token_endpoint is always required; auth_endpoint only for auth-code
grants — client_credentials needs no authorization URL, matching the UI
which hides that field for that grant.
- the refresher already refreshes against the stored token_endpoint and
the callback persists it, so manual-mode tokens refresh correctly.
- oauthFingerprint includes use_dcr (nil normalized to true) so toggling
DCR mode purges stale tokens.
- the web form only serializes manual endpoints when use_dcr is off.
Audited all MCP dialogs (form, global OAuth, per-user credentials, grants,
tools): OAuth dialogs handle completed/auth_url identically and read
config from stored server settings; runtime connect uses the stored token
via the refresher (no re-discovery).
Tests: manual auth-code + client_credentials endpoints, missing/SSRF
endpoints, and the full self/global/on-behalf authorization matrix.
|
||
|
thotam
|
389640ae51 |
fix(agent): check nil td.Function to prevent panic on native tools (#1192)
* fix(agent): check nil td.Function to prevent panic on native tools Fix nil pointer dereference panic when agent processes native tools (such as image_generation) which do not contain function schema wrappers. Key changes: - loop_pipeline_callbacks.go & think_stage.go: add checks for td.Function != nil before accessing td.Function.Name. - loop_tool_filter.go & loop_history_toolnames.go: add safety checks when filtering and extracting tool names. - anthropic_request.go & codex_build.go: verify t.Function != nil when formatting tools for provider API requests. * test: add regression tests for native tools nil function panic Add targeted regression tests to verify that: - ThinkStage builds AllowedTools without panicking when mixed with native tools - Loop's buildFilteredTools filters native tools correctly without panicking across all filters - Anthropic request builder skips native tools safely - Codex request builder handles native tools with nil Function fields gracefully |
||
|
SYNITYandDangTinh311
|
8eef0509ce |
fix(agent): surface per-user MCP tools to the LLM tool list [B24:2794] (#1235)
require_user_credentials MCP servers loaded and connected, but their tool defs were never sent to the model, so the agent could never call the per-user mcp_<prefix>__* tools (it only ever saw the shared MCP server). The per-user tool objects are intentionally kept out of the shared registry to prevent cross-user credential leaks, so FilterTools could not emit them. Surface them through a request-scoped overlay registry passed to the policy engine: per-user tools are now evaluated AND emitted under the same allow/deny rules as registry tools, then resolved per-actor at execution time via executeToolForActor. The overlay is discarded after filtering, so the shared registry and other users stay unaffected. - add tools.NewUserToolOverlay (request-scoped, discarded post-filter) - buildFilteredTools: wrap registry in overlay when per-user tools present; no-policy path appends their defs directly - fix stale getUserMCPTools docstring (claimed shared-registry registration) - tests: overlay policy allow/deny + per-user surfacing / strip / dedup Co-authored-by: DangTinh311 <dangtinh31193@gmail.com> |
||
|
Goon
|
591d809779 |
Merge remote-tracking branch 'upstream/dev' into dev
# Conflicts: # internal/cron/service.go |
||
|
Goon
|
06877365c4 | merge: sync dev into usage analytics branch | ||
|
Goon
|
4002bb4c13 | feat(usage): add event analytics dashboard | ||
|
Goon
|
9cd57a920c | feat: add multi-attachment delivery batching | ||
|
Duy /zuey/
|
515596c44c |
Merge pull request #167 from digitopvn/codex/issue-161-skill-self-evolution
feat(skills): add skill self-evolution metrics |
||
|
Goon
|
a76f443411 | feat: add persona context to delivery messages | ||
|
Goon
|
f45bfa860c | feat(skills): add skill self-evolution metrics |