mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-12 14:15:00 +00:00
dev
380
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chandan Kumar
|
65dbbcfd59 |
fix(vault): search document bodies and match any query word (#1554)
vault_search only indexed title + path + the auto-summary, and the summary is written from the first 3000 runes of the file. Anything the summary left out (room names, prices, codes) could never be found. On top of that the FTS query used plainto_tsquery, which ANDs every word, so a normal question like "what is the price of the Deluxe Ocean Suite?" matched nothing even when the key words were indexed. - Add vault_document_chunks (migration 98): the file body split into chunks with their own tsvector and embedding. body_indexed_hash on vault_documents records which content_hash the chunks came from, so unchanged files are not re-chunked or re-embedded. - The enrich worker rebuilds chunks when a file changes. This needs no LLM, so it runs even when no provider is configured. - Rescan backfills chunks for docs indexed before this change, since they already have a summary and never go back through the worker. - FTS matches any query word; ts_rank still ranks docs with more matching words first. Both FTS and vector search look at the doc and its chunks and score each doc by its best hit. SQLite is unchanged: its vault search is LIKE on title/path only. |
||
|
fcy
|
6063d975e8 |
fix(http): restore ACP executable-path validation, bypass SSRF URL check (#1481)
ACP providers use api_base to store an executable command/path, not a URL. A previous SSRF-hardening refactor incorrectly classified ACP as a local-URL provider type, causing create/update to reject valid ACP configs with "provider URL must use http or https scheme". Changes: - Remove ProviderACP from localURLProviderTypes. - Add validateACPExecutablePath mirroring the runtime registration logic in cmd/gateway_providers.go (allows empty, claude/codex/gemini, or absolute path; rejects URLs and relative paths). - Route ACP through the new validator inside validateProviderURL. - Add unit tests for validateACPExecutablePath. - Update existing local-type tests that assumed ACP was URL-based. - Add create/update HTTP handler regression tests for ACP binary, empty binary, and URL rejection. Bug-first verification: without the fix, ACP api_base values like "gemini" or absolute paths fail URL validation, while URLs are incorrectly accepted; with the fix the behavior is reversed to match the intended executable-path model. Fixes nextlevelbuilder/goclaw#1481. |
||
|
Serply
|
407e564b91 |
feat(tools): add Serply as a web_search provider (#1567)
* feat(tools): add Serply as a web_search provider Serply is a SERP API returning Google web results on a tenant-supplied key. It joins the existing chain the same way Exa, Tavily and Brave do: keyed from config_secrets, skipped silently when no key is stored, and selectable through the provider parameter for cross-engine corroboration. Freshness maps onto Google's tbs recency filter, which Serply forwards upstream. Only the pd/pw/pm/py shortcuts are sent: upstream ignores the equivalent cdr date range and answers with unfiltered results, so a range leaves tbs unset rather than naming a filter the results do not honour. The settings form gained a fifth sortable provider, so the locked DuckDuckGo card now derives its position number instead of hardcoding it, and a stored provider_order is reconciled against the full provider list rather than against Parallel alone. * test(tools): use a locally named round trip stub in the serply test The serply cancellation test borrowed parallelRoundTripFunc from web_search_parallel_test.go. Define serplyRoundTripFunc alongside the test that uses it so the file does not depend on another provider's test helper. |
||
|
Thibault Jaigu
|
32eb2328f3 |
feat(provider): add Requesty OpenAI-compatible provider (#1579)
Register Requesty (https://router.requesty.ai/v1) next to OpenRouter on the existing OpenAI-compatible transport: config and env vars (GOCLAW_REQUESTY_API_KEY, GOCLAW_REQUESTY_BASE_URL), secret masking, DB and in-memory registration, CLI setup, doctor, placeholder provider, OpenAPI enum, Web/Desktop provider lists and docs. The models list merges Requesty managed policies (GET /models/managed) with the key's catalog from GET /models. |
||
|
Clark Cant
|
2066342a71 |
Merge pull request #1568 from yatul/feat/permanent-pairing
feat(pairing): let an operator make a device pairing permanent |
||
|
yatul
|
4f80824177 |
feat(agents): let create opt out of background summoning with "summon": false (#1570)
Creating a predefined agent with a description kicks off summoning in a
background goroutine. It finishes 10-20 seconds after the create response and
writes SOUL.md, IDENTITY.md, CAPABILITIES.md and the agent frontmatter.
For a client that manages agents as code that is data loss. Such a client
brings its own context files and writes them through agents.files.set as soon
as create returns; summoning lands afterwards and replaces them. Nothing
fails: no error from the API, nothing in the logs, and the agent quietly runs
on generated text instead of the committed role. The only way to avoid it
today is to poll the agent status until it leaves "summoning" — which means
every automation has to know that summoning exists at all.
POST /v1/agents now accepts an optional "summon" field. Absent or true keeps
today's behaviour, so existing clients are unaffected. False creates the agent
active with the seeded template files and starts no LLM call; summoning is
still available afterwards via POST /v1/agents/{id}/resummon.
The status fallback also now resets an explicitly requested "summoning" status
when nothing will summon the agent: that status is only ever cleared by the
summoner, so such an agent would otherwise stay stuck in it forever.
Surface parity: CLI and WS RPC (agents.create) create agents without a
description and never summoned, so they need no flag; the web UI create flow
wants summoning and keeps the default.
|
||
|
yatulandClaude Opus 5
|
5379cc1163 |
feat(pairing): let an operator make a device pairing permanent
Every approved pairing expires 30 days after approval (pairedDeviceTTL),
and nothing lets an operator change that: owners of a personal setup have
to re-pair their own Telegram account and browser every month. The schema
already treats a NULL paired_devices.expires_at as "never expires" (IsPaired
and the prune both check for it), only no code path ever writes NULL.
- store: SetPairingPermanent(senderID, channel, permanent) on PairingStore,
PG and SQLite. permanent=true clears expires_at, false restarts the
default TTL from now. An already expired pairing is not revived.
PairedDeviceData gains expires_at (Unix ms, null = never), returned by
ApprovePairing and ListPaired.
- gateway: device.pair.approve accepts `permanent`; new admin RPC
device.pair.update {senderId, channel, permanent} for existing pairings,
classified in permissions/policy.go next to the other pairing methods.
- mcp: pairing approve tool accepts `permanent`.
- web UI (Nodes): "Never expires" switch in the approve dialog, an
Expires column, and a Make permanent / Set expiry action per device.
ConfirmDialog takes optional children for the switch.
The default stays 30 days; permanence is an explicit operator choice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
DennyHo0917
|
eeee2daa5e | merge: resolve API Route provider conflicts with dev | ||
|
DennyHo0917
|
d67e1b0909 | feat(provider): add API Route OpenAI-compatible provider | ||
|
HaiDuongandmor-phongdt
|
b39f0decb9 |
fix(skills): five defects in slash activation, grants, subagent tool policy, preview and import (#1534)
* fix(agent): filter skill slash commands by the agent's grants The inline <available_skills> block was already filtered by visibility and agent grants, but slash activation read the loader's full skill list. A /<slug> command could therefore activate a skill the agent was never granted, and the not-found suggestions disclosed that such skills existed. Resolve slash commands against the agent's allow list via FilterSkills. The list is filtered before matching, so /list-skills, /help and the near-match suggestions are all covered by the same change. The allow-list convention is unchanged: nil means every skill (the fallback when the access store errors), an empty slice means none, and a populated slice is an explicit set of slugs. * fix(agent): split slash commands on any whitespace The tokenizer separated the skill name from the rest of the message with strings.Cut(after, " ") — a literal space. A user who typed the command and pressed Enter before the rest of the message sent "/ck:git\nreview the diff", which parsed to the target "ck:git\nreview" and matched no skill. The reply was "skill not found" followed by near-matches that included the skill they had just named, because the similarity fallback searched the mangled string and still landed beside it. Multi-line messages are ordinary in Slack and Telegram, so this was reachable in normal use. Introduce cutFirstField, which splits on the first run of whitespace, and use it everywhere the literal-space split appeared. The match loop had the same assumption in strings.HasPrefix(raw, value+" "); hasFieldPrefix replaces it and decodes the following rune properly, so a multi-byte skill name is not truncated. That also stops a plain string prefix from matching: a command for "frontend-design-extra" no longer resolves to "frontend-design". * fix(tools): inherit the parent agent's tool policy in spawned subagents buildSubagentToolsRegistry clones the parent registry, then overwrites exec, read_file, write_file and list_files with freshly constructed tools. A fresh tool carries none of the hardening the gateway applies to the parent's instances at startup: exec path denials and their exemptions, the shell deny-group toggles, the command keyword allowlist, and the read/write/list deny prefixes covering config.json, the internal databases and delegate/. Spawning a subagent therefore widened what an agent could reach — the parent was blocked from the data dir, the subagent was not. Verified by disabling the new call: the subagent's exec read config.json out of the denied data dir, read_file did the same, and write_file overwrote it. Copy the policy from the live parent instances rather than re-deriving it, so there is one source of truth and a deny path reloaded later through config pub/sub reaches subagents without a second wiring site that can drift. Allow-prefixes are inherited alongside the denials: they are what make the denied roots usable at all, since the skills store sits under the denied data dir. Copying denials without them would leave a subagent unable to read the skills it is told to use. * fix(agent): use one inline-vs-search decision for prompt and preview The system-prompt preview exists to show the prompt an agent actually gets, but it decided between inline skills and search mode on its own terms: it counted tokens with the fallback counter over the fully rendered XML — tags and <location> paths included, at roughly runes/2 — while the prompt builder estimated name+description at chars/4. On the same eighteen skills that read 3087 against 944, so the preview reported search mode for an agent that was running inline. Extract shouldInlineSkills and call it from both paths. The estimate keeps the prompt builder's rule, mirroring BuildSummary's 200-rune description truncation, so the decision still costs no rendering. The preview's loader interface gains FilterSkills to feed it. Its behaviour on an access-store error is unchanged: an empty allow list yields no skills rather than falling back to showing every skill. * fix(http): keep reference files when importing skills Export archives the whole skill directory, but import recognised only metadata.json, SKILL.md and grants.jsonl. The switch had no default, so every other file was discarded without a log line. A skill whose SKILL.md cites references/*.md arrived without them, the import reported success, and the skill failed at the first read. Collect the remaining entries and write them under the skill directory with their structure intact. Archive entry names are attacker-controlled, so each path goes through sanitizeRelPath: a traversal attempt collapses to a relative path and the write stays inside the skill directory. * test(agent): make the inline-decision table exercise both gates The "100 skills, 200-char descriptions" case claimed to cover the token ceiling, but 100 exceeds the count ceiling so the count check short-circuited and the token branch never ran — deleting that branch would not have failed the test. Split the table so each case isolates one gate: tiny descriptions for the count rows, a count safely under the ceiling for the token rows. Removing the token comparison now fails the over-the-ceiling case and nothing else. * fix(agent): gate slash commands on the managed tier only The allow list comes from a query over the `skills` table, so filesystem-tier skills — the workspace, .agents and ~/.agents directories of the five-tier loader — have no row in it. Filtering every skill against the list made those four tiers unreachable by slash while skill_search still found them unfiltered, which turned a security fix into a functional regression. Apply the list to managed skills only. Builtins are seeded with is_system and returned unconditionally, so they stay reachable either way. This closes slash activation of ungranted managed skills. It does not close skill_search and use_skill, which still read the loader's full list; that is a wider change and wants its own review. * fix(http): stop imported skill files from replacing guarded ones Two ways the auxiliary write could go wrong. GuardSkillContent scans SKILL.md before anything reaches disk, but the switch that routes archive entries compares the raw path. An entry named "./SKILL.md" does not match it, so it fell through to the auxiliary set — where sanitizeRelPath collapses it back to "SKILL.md" and the write replaced the file that had just been scanned. Reject any auxiliary path that resolves to a name the import handles by itself, and log the attempt. Tar directory entries carry a trailing separator and no content. Written as files they took the name a real directory needed, so everything beneath was dropped — and whether that happened depended on map iteration order, making it intermittent. Skip them; MkdirAll creates what is needed. Also cap the number of auxiliary files per skill and log when the cap trims an archive, so a truncated import is visible rather than silent. --------- Co-authored-by: mor-phongdt <phong.dangtuan@mor.com.vn> |
||
|
Excelius
|
b75753b787 | fix: handle SQLite provider settings updates (#1531) | ||
|
minhdang03
|
1d68a21f65 |
fix(channels): archive group messages before the pending buffer drops them (#1514)
channel_pending_messages is a buffer, not an archive. Rows were deleted outright on two paths: the bot being mentioned hands the buffer to the agent and clears the key, and LLM compaction replaces old rows with a summary. For group capture that buffer held the only copy of the raw text, so a single mention or compaction pass destroyed days of messages nothing had read yet. Copy every row into channel_message_archive inside the same transaction as the delete, tagged with the reason (consumed, compacted, stale). Archived rows keep their original id, so a replayed delete is a no-op. Add ListArchivedByKey so consumers that need full history read the archive instead of the buffer. |
||
|
bilogic
|
72c2c301a5 | get refreshable tokens | ||
|
mdsandClaude Code
|
94d2799788 |
feat(http): support persistent sessions via X-GoClaw-User-Id header (#713)
When X-GoClaw-User-Id header is provided, build a stable session key
using BuildSessionKey() — same canonical format as Telegram, Discord,
Slack channels. This allows HTTP API clients to maintain conversation
history across multiple requests.
Without the header, behavior is unchanged (random session per request).
Session key format:
With userId: agent:{id}:http:direct:{userId} (persistent)
Without: agent:{id}:http-{random} (stateless)
Co-authored-by: Claude Code <claude@anthropic.com>
|
||
|
Clark Cant
|
d47cc51aa0 |
Merge pull request #1451 from binyangzhu000-sudo/codex/add-atlascloud-provider-goclaw
Add Atlas Cloud provider defaults |
||
|
Floze
|
276b4bbab7 | test(mcp): lock test and save SSRF parity (#1450) | ||
|
binyangzhu000-sudo
|
76d2c290c2 | Add Atlas Cloud provider defaults | ||
|
|
293aa2a5e1 |
(feat) MCP Server (#1365)
* feat(hooks/observe): add structured output validation hook (ObserveHook) - hooks/types.go: Add ObserveHook interface + BuiltInHookType enum - hooks/dispatcher.go: HookDispatcher emits ObserveHook with PhaseResult payload - pipeline/observe_stage.go: ObserveStage emits hook after ObserveResult built - pipeline/substates.go: ObserveStageResult carries hook results + validation errors - hooks/config.go: BuiltInHookTypeObserve added to BuiltInHookType enum PhaseResult carries structured output, token usage, tool calls + validation errors emitted post-ObserveStage. ObserveHook implementations can validate structured output against schemas, detect tool-call loops, enforce token budgets, etc. Hook fires after ObserveStage produces ObserveResult, before results propagate to next stage. ValidationError returned by hook halts pipeline and propagates error to caller without further stage execution. Co-Authored-By: Claude <noreply@anthropic.com> * ui(hooks): add post_model_response event to web UI - Add event to Zod schema, filter dropdown, and form dialog - Implement conditional test panel UI for model response payload - Add translations (en/zh/vi) for new test panel fields - Updated beta description to reference the new event Co-Authored-By: Claude <noreply@anthropic.com> * feat(mcp): add MCP CRUD server exposing goclaw resources at /api/mcp/ with Bearer token auth and X-GoClaw-Tenant-Id header, default to master tenant * feat(mcp): add goclaw_skills_write_file tool to edit skill files on disk The CRUD MCP server's goclaw_skills_update only touched skill DB metadata, with no way to edit a skill's SKILL.md/file content on the filesystem. Extract the versioned write logic from the web UI's skill file editor (SkillsHandler.handleWriteFile) into skills.WriteVersionedFile so both surfaces share identical validation and versioning, and expose it as a new MCP tool. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Matt Van HornandMatt Van Horn
|
e5b5d03e3d |
fix: persist explicit None TTS provider instead of dropping it (#1424)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> |
||
|
thotam
|
0c1ededc92 |
feat(webhooks): return per-call usage breakdown with provider/model/cost (#1421)
feat(webhooks): per-call usage breakdown with provider/model/cost (#1421) |
||
|
ナムandCollective Developer
|
6b738924d5 |
fix(discord): qualify group titles across surfaces (#1420)
Co-authored-by: Collective Developer <man@collective.dev> |
||
|
Zezae Oh
|
9be468edad | feat(providers): add GPT-5.6 Codex models (#1416) | ||
|
Bruno ClermontandBruno Clermont
|
d856a58218 |
feat(providers): make provider request timeout configurable (default 30s) (#1400)
The provider verify call and the provider models-list call used hardcoded timeouts (30s and 15s) that were too low for slow models. Introduce a tenant-scoped, UI-editable setting `providers.request_timeout_sec` (default 30) stored in system_configs, consumed by both handlers, and overlaid from config.json like tts.timeout_ms. Adds a field to the System Settings modal (all 4 locales) so operators can raise it without a redeploy. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Bruno ClermontandBruno Clermont
|
6bafe54b08 |
fix(agent): filter per-tenant disabled tools from system prompt (#1396)
The system prompt's Tooling section previously listed tools even when they were disabled for a tenant and already stripped from the API tools parameter, confusing the LLM into thinking it could use them. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
ナムandCollective Developer
|
fc7d183961 |
Enhance passive memory extraction tuning (#1393)
* feat(channelmemory): add passive extraction tuning * feat(ui): expose passive memory tuning controls * docs(memory): document passive extraction tuning --------- Co-authored-by: Collective Developer <man@collective.dev> |
||
|
SYNITYandDangTinh311
|
3b8bc1bc4d |
feat(bitrix24): replace two MCP text inputs with a filtered dropdown (#1392)
* feat(bitrix24): replace two MCP text inputs with a filtered dropdown [B24:2794]
Bitrix24 channel creation used to demand two hand-typed strings —
mcp_server_name and mcp_base_url — plus zero indication of which MCP
servers can actually auto-onboard. Typos silently disabled provisioning
and admin had to know which servers implement /api/auto-onboard.
Ship a single dropdown backed by mcp_servers.require_user_credentials,
plus the machinery to make it work end-to-end.
Phase 1 — DB & store
* Promote require_user_credentials from settings JSONB to a top-level
column on mcp_servers (PG migration 000089, SQLite migration v54).
* Backfill from existing settings blobs so no admin needs to re-tick.
* Add MCPServerData.RequireUserCredentials to the Go store layer, plumb
through Create / Get / GetByName / List / Update on both stores,
extend the export DTO, and add require_user_credentials to the HTTP
allowlist.
* Bump RequiredSchemaVersion 87 -> 89 (jumping 88, which was on disk
but not wired) and SchemaVersion 53 -> 54 with an
idempotentColumnMigration guard.
Phase 2 — Bitrix24 channel factory
* Add MCPServerID (UUID string) to bitrix24 InstanceConfig, keep
MCPServerName + MCPBaseURL as legacy fallback with a "deprecated"
doc comment.
* Factory validation accepts either mcp_server_id alone or the legacy
pair; half-config still fails fast.
* initMCPProvisioner prefers GetServer(id) and sources the base URL
from mcp_servers.url when the id path is used. Legacy name path
unchanged so pre-migration configs keep working.
* Log line now carries mcp_server_id + require_user_credentials so
operators can eyeball the wiring.
Phase 3 — Frontend types & MCP form
* Add optional top-level require_user_credentials to MCPServerData /
MCPServerInput in both ui/web and ui/desktop/frontend types.
* mcp-form-dialog reads the top-level flag first and falls back to
settings.require_user_credentials so cached responses from
pre-upgrade backends still render correctly.
* On submit send both the top-level flag AND the legacy settings
entry so mid-rollout backends stay consistent.
Phase 4 — Bitrix24 channel form dropdown
* New mcp-select field type + MCPServerSelect component. Uses the
shared useMCP() react-query cache and filters client-side to
servers whose require_user_credentials is true (OR settings
JSONB during the migration window).
* Explicit "None (disable MCP provisioning)" option so admins can
clear the binding without editing config JSON.
* Legacy mcp_server_name / mcp_base_url text inputs kept in the
Advanced panel, relabelled "(legacy)" with pointer help text.
Phase 5 — channel_instances.config backfill
* PG migration 000090 and SQLite migration v55 rewrite existing
bitrix24 channel_instances.config to add mcp_server_id by
resolving mcp_server_name against mcp_servers, tenant-scoped
via agents.tenant_id (channel_instances doesn't carry tenant_id
directly).
* Idempotent — only touches rows already carrying
mcp_server_name that lack mcp_server_id. Legacy keys are left
in place so provisioner.go can still fall back for unmigrated
or future-created legacy configs.
* down.sql drops the mcp_server_id key. Provisioner immediately
reverts to the legacy fallback path.
Tests
* provisioner_test.go: three new cases exercise the mcp_server_id
path (invalid UUID string, valid UUID with missing row, valid
UUID with a per-user row). fakeMCPStore gains a serversByID
map and a real GetServer implementation.
* Existing legacy-config tests unchanged and still green.
Verification
* go build ./... && go build -tags sqliteonly ./...
* go vet ./internal/mcp/... ./internal/channels/bitrix24/...
./internal/store/... ./internal/http/...
* go test ./internal/mcp/... ./internal/channels/bitrix24/... -> ok
* Live-tested against a local docker image on the goclaw-deploy
postgres. Migrations 89 + 90 applied cleanly. Three existing
bitrix24 channels (bitrix-sales / nguyen-dao-openline / tieu-vi)
had their configs backfilled with the b24-syn-mcp UUID and the
provisioner boots with require_user_credentials=true. UI dropdown
correctly shows only b24-syn-mcp (the only server with the flag
ticked) alongside a "None" clearer option.
Surface parity
* Gateway server: store + factory + provisioner + HTTP allowlist.
* API contract: adds require_user_credentials + mcp_server_id
as optional fields on existing routes. No new endpoints.
* Web UI: MCP form + Bitrix24 channel form + shared types.
* CLI/runtime package: N/A because no CLI subcommand reads the
mcp_server_id field.
* fix(bitrix24): derive auto-onboard base URL from mcp_servers.url origin [B24:2794]
The Phase 2 refactor swapped provisioner base-URL sourcing from the
legacy per-channel MCPBaseURL config field (which historically stored the
MCP server's ORIGIN, e.g. https://mcp.example.com) to mcp_servers.url,
which stores the JSON-RPC ENDPOINT the agent loop dials (e.g.
https://mcp.example.com/mcp). The two are semantically different but
share a single column.
mcp_client.newMCPClient then appends "/api/auto-onboard" to whatever
baseURL it receives, so the id-path started POSTing to
".../mcp/api/auto-onboard" — 404 for every per-user credential mint and
refresh. Existing users kept working only until their cached access
tokens expired.
Fix: derive the origin (scheme://host[:port]) from server.URL before
handing it to the auto-onboard client. The legacy path is untouched
because MCPBaseURL from channel config is already the origin.
https://b24-mcp-dev.synity.so/mcp -> https://b24-mcp-dev.synity.so
https://mcp.example.com/mcp/ -> https://mcp.example.com
https://mcp.example.com -> https://mcp.example.com
Table-driven test covers six shapes plus four error cases (empty,
whitespace-only, no scheme, no host). Updated the existing
TestInitMCPProvisioner_MCPServerID fixture to seed a URL with the /mcp
subpath so it regression-guards the same code path.
Verified live: user 614 sent a message that triggered the expired-cred
refresh branch; goclaw logged "self-refreshed user credentials
created=false" and the agent immediately reported
"mcp.user_tools_loaded user=614 tools=2". Before this fix the same event
logged 'auto-onboard failed: mcp auto-onboard: 404 Not Found'.
Surface parity:
- Gateway server: provisioner + one new helper (deriveAutoOnboardBaseURL).
- API contract: N/A because the wire shape hasn't changed.
- Web UI: N/A because the UI still writes mcp_server_id verbatim.
- CLI/runtime: N/A because no CLI reads the derived base URL.
---------
Co-authored-by: DangTinh311 <dangtinh31193@gmail.com>
|
||
|
ナムandCollective Developer
|
a9c7feb99c |
fix(channelmemory): harden passive extraction reliability (#1386)
Co-authored-by: Collective Developer <man@collective.dev> |
||
|
nguyenha935andnguyenha935
|
086532484d |
feat: add configurable appearance settings (#1384)
Co-authored-by: nguyenha935 <208228297+nguyenha935@users.noreply.github.com> |
||
|
daotriandTrí Đào Nguyễn Minh
|
dc4cfddd31 |
fix(mcp): refresh per-user credentials without restart (#1375)
Co-authored-by: Trí Đào Nguyễn Minh <tri.dao@buymed.com> |
||
|
ナムandCollective Developer
|
f31214f5ce |
feat(channel-memory): enhance passive extraction for Discord (#1361)
* fix(channels): persist pending history under instance names * feat(channel-memory): add run-all extraction workflow * feat(discord): resolve passive memory channel titles * feat(web): enhance passive memory controls * docs(channel-memory): document extraction endpoints --------- Co-authored-by: Collective Developer <man@collective.dev> |
||
|
|
a3fe5f7858 |
fix(mcp): resolve tool descriptions/schemas consistently across all MCP tool-listing paths (#1369)
* fix(mcp): return bare tool names with descriptions for already-connected MCP servers, fixing empty descriptions/schemas in agent prompts * fix(mcp): resolve live tool descriptions/schemas in ListToolsForAgent for legacy prefixed tool_allow entries The previous fix (d612da8e) only corrected the admin "browse/select tools" endpoint. The actual runtime path building the LLM system prompt's tool list, Manager.ListToolsForAgent, still looked up tool_cache/registry entries keyed by the raw stored tool_allow name. Agents with grants captured before that fix have tool_allow persisted as registered (prefixed) names instead of bare MCP tool names, so every lookup missed and every tool in the prompt preview came back with an empty description and schema. Add bareMCPToolName to normalize both legacy-prefixed and current bare tool_allow/tool_deny entries before matching, and resolveMCPToolInfo to prefer the live registry's current description/schema (falling back to the settings tool_cache) instead of trusting whatever was persisted verbatim. Both the explicit-allow and cache-enumeration branches now share this single resolution path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Bruno ClermontandBruno Clermont
|
db8ac12b16 |
feat(providers): allow LAN Ollama hosts via GOCLAW_OLLAMA_ALLOWED_HOSTS (#1367)
Adds an operator opt-in env var (comma-separated hostnames/IPs) to whitelist additional local hosts for ollama/acp provider-type URL validation, on top of the existing hardcoded localhost allowlist. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
Bruno ClermontandBruno Clermont
|
374defe04e |
fix(skills): allow editing SKILL.md content for database-managed skills (#1363)
* feat(skills): allow editing SKILL.md content for database-managed skills
Adds the ability to edit a skill's actual SKILL.md content directly
from the web UI, restricted to database/tenant-managed skills only --
bundled/system skills remain read-only.
Backend: new PUT /v1/skills/{id}/files/{path...} endpoint
(handleWriteFile in skills_versions.go), writing to the current
version directory of a non-system skill only (403 for is_system
skills), with the same path-traversal/symlink/ownership guards as the
existing read endpoint. Bumps skill store version and emits
cache-invalidate + audit event on success.
Frontend: Content tab in the skill detail dialog gets an "Edit
content" button, disabled with a tooltip for system skills. Editing
fetches the RAW file content via the existing read endpoint (which
does not strip YAML frontmatter, unlike the stripped preview shown in
the Content tab by default) to avoid silently destroying frontmatter
on save. Save/cancel follow the established catch-and-toast error
pattern (no uncaught promise rejections, matching PR #1346's config
save button fix).
i18n keys added to all 4 locales (en, ko, vi, zh).
* fix(skills): bump version on content edit, fix scroll regression in content view
handleWriteFile previously overwrote the current skill version in place
instead of creating a new immutable version, inconsistent with the
skill_manage tool's patch action. Now creates a new version directory
and updates the DB pointer, matching that convention.
Also fixes a scroll regression in the skill detail dialog's Content
tab introduced by the edit-mode textarea -- both the read-only preview
and edit textarea are now properly scrollable within the dialog.
* fix(skills): refresh version display after save, fix dialog height constraint for scrolling
Version bump now correctly reflects in the UI immediately after save
without requiring a page reload. Fixed the actual root cause of the
scroll issue: the dialog's height wasn't bounded, so overflow-y-auto
on inner content had no effect since nothing constrained the dialog's
total height in the first place.
---------
Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com>
|
||
|
thotam
|
dcf000ab6a |
fix(http): allow Operator to read providers and OAuth status (#1075) (#1359)
Browser pairing authenticates users as RoleOperator, but every provider and OAuth route was wrapped in requireAuth(RoleAdmin). The /setup flow reads GET /v1/providers, so paired browsers got 403 and hung on the setup page. Split provider/OAuth route auth by operation: - providers.go: GET list/get/models/status/embedding/codex-activity now use a readAuth wrapper (method-derived min → Viewer); POST/PUT/DELETE and reconnect/verify stay Admin. Responses already mask API keys and queries stay tenant-scoped, so no secrets are exposed. - oauth.go: GET status/quota use readAuth (they return only connection state and quota, never tokens); start/callback/logout stay Admin (preserving the #450 admin-only-management decision for mutations). Updates the two auth tests to assert the new contract: an Operator can read providers/OAuth status (200) but still cannot mutate them (403). |
||
|
Bruno ClermontandBruno Clermont
|
8000a1d0f5 |
feat(providers): add provider-level thinking/reasoning toggle for Ollama (#1355)
Ollama has two separate request-building code paths: OllamaProvider (native /api/chat, used for num_ctx control) and OpenAIProvider (OpenAI-compat /v1/chat/completions). An earlier fix disabled thinking mode by hardcoding think=false, but only in the OpenAI-compat path -- OllamaProvider.buildRequest() never set the think field at all, so reasoning-capable models (qwq, deepseek-r1) defaulted to visible chain-of-thought reasoning regardless of that fix. Confirmed live via a docker-engineer agent streaming full reasoning traces despite the existing disable. Replaced the hardcoded always-off behavior with a provider-level tri-state setting (llm_providers.settings.thinking_enabled: unset = default off, explicit true/false overrides), configurable via the provider's Advanced settings dialog. Both OllamaProvider.buildRequest() and OpenAIProvider.buildRequestBody() now read and respect this same setting, so the toggle works regardless of which Ollama code path a given deployment routes through. Added tests for setting parsing (unset/true/false/malformed) and both provider request-builders' handling of the override. Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
thotam
|
f87cdaf616 |
feat(webhooks): split webhook usage tokens by cache vs non-cache (#1345)
Surface cache-read and cache-creation input tokens (plus the prompt_tokens_include_cached_segments flag) in webhook usage responses, so callers can distinguish cached from non-cached input tokens. The data already flowed end-to-end via providers.Usage and agent.RunResult.Usage; the two webhook envelope structs (webhookLLMUsage for the sync LLM webhook, callbackUsage for the async callback payload) copied only 3 of the fields, dropping the cache data. Add the 3 cache fields (mirroring providers.Usage JSON tags exactly, all omitempty) and populate them. The async callbackUsage mapping is extracted into a pure newCallbackUsage helper for unit testing. Additive and non-breaking: omitempty keeps responses without caching byte-identical. No schema change. |
||
|
Dmitry
|
2810ad96f0 |
feat(providers): add AI/ML API preset (#1336)
Approved by github-maintain. Clean implementation of AI/ML API as branded OpenAI-compatible provider. All CI green. Closes #1325. |
||
|
|
90e7035100 |
fix(mcp): tenant isolation, tool policy engine, and prompt preview fixes (#1333)
* fix(prompt): remove duplicate Team Members section from system prompt The TEAM.md context file already provides the Members section with better formatting. The system prompt section was redundant and inconsistently formatted. - Remove buildTeamMembersSection() call from system prompt building - Remove unused buildTeamMembersSection() function * fix(mcp): wire store to manager for prompt preview tool visibility MCP manager needs database access to query configured servers for tool visibility in prompt preview. - Add SetStore() method to MCP manager - Wire pgStores.MCP to manager after initialization - Add debug logging for MCP initialization flow * fix(systemprompt): hide Tooling section when agent has no tools The Tooling section header and boilerplate were displayed even when the agent had no tools available. Add early return in buildToolingSection() to skip the section entirely when toolNames is empty. This reduces prompt noise for agents with no tool access. * feat(mcp): cache tool descriptions for prompt preview visibility MCP tools now show descriptions in prompt preview without requiring a live server connection. - Add CacheToolDescriptions() method to MCPServerStore (PG + SQLite) - Cache tool descriptions in settings['tool_cache'] when server connects - Use cached descriptions in ListToolsForAgent (fallback: hints → cache → global) - Descriptions are auto-populated from live server manifest on connection - Admins can still override via tool_hints in server settings * test(mcp): add CacheToolDescriptions to MCPServerStore test fakes Commit 5ce410b6 added CacheToolDescriptions() to the MCPServerStore interface but missed updating test mock implementations, breaking go vet across internal/mcp, internal/agent, internal/http, and internal/channels/bitrix24. Add no-op implementations matching each fake's existing style. * fix(providers): enforce per-agent tool policy for Claude CLI provider The Claude CLI provider (stdio+MCP bridge) was not enforcing per-agent tool policy, unlike other providers where the policy-filtered tool list already drives the system prompt's Tooling section. Two gaps closed: 1. --disallowedTools was previously skipped entirely when no MCP config path was resolved, letting the CLI subprocess run with its full native toolset (Bash, Edit, Read, Write, Glob, Grep, WebFetch, WebSearch) regardless of agent policy. It's now unconditional and derived from the agent's actual allowed-tools list (state.Tool.AllowedTools), mapped to Claude CLI's native tool names. 2. The MCP bridge server executed any tool call without checking the calling agent's policy. It now resolves the agent's policy from context (via the existing HMAC-verified agent lookup) and denies calls to tools outside that agent's allowed set, logging security.mcp_bridge_denied on denial. Both gaps were closed using existing plumbing (PolicyEngine.WouldAllow, AgentData.ParseToolsConfig, the bridge context middleware) — no new cross-cutting mechanism was introduced. * feat(web): show MCP/tool schemas in system prompt preview dialog The prompt-preview API response includes a separate `tools` field (the actual JSON schemas sent to the LLM as the tools API parameter) alongside `prompt` (the system prompt text), but the web UI only rendered `prompt`, silently dropping the tools list. Add a collapsible Tools section to both the full-screen System Prompt dialog and the inline agent-detail preview, showing tool count, name, description, and expandable parameter schema per tool. i18n keys added to en/vi/zh locales. * fix(prompt): render pinned skills on bootstrap turns Pinned skills are documented (web UI copy) as "always inlined in the system prompt", but the entire Skills section was gated behind !cfg.IsBootstrap, so pinned skill XML never appeared on bootstrap turns (first message of a session) despite the promise. Separate pinned-skill rendering from bootstrap-suppressed guidance: - Bootstrap + pinned skills present: render pinned XML only, no search/manage guidance (which stays suppressed as before) - Non-bootstrap: unchanged behavior - Minimal/none modes: pinned skills always render regardless of bootstrap state Add regression tests covering all four prompt modes on bootstrap turns, plus a non-bootstrap guard confirming existing behavior is preserved. * fix(skills): resolve managed skills directory per-tenant, not master-only skills.Loader was wired at startup to scan a single fixed directory (the master tenant's managed-skills dir), making any skill belonging to a non-master tenant invisible to both pinned-skills prompt resolution and skill_search/use_skill, regardless of DB visibility settings. - Loader now resolves the calling tenant's managed-skills directory per-call via context (store.TenantIDFromContext), never enumerating other tenants' directories - Skill cache is now tenant-keyed to prevent slug collisions and cross-tenant cache leaks across tenants using the same skill slug - gateway_setup.go passes the root data dir instead of a pre-resolved master-tenant path Write-side tooling (skill_manage, publish_skill) was already correctly tenant-scoped per-operation — no changes needed there. Added TestLoader_ManagedSkills_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's content, including after cache population from a different tenant's lookup. Known follow-up (not in this commit): skill_search's BM25 index is still a single process-global index shared across tenants, which is a related but separate cross-tenant search-result leak requiring its own scoped fix (per-tenant index maps + threading tenant context through ensureIndex/rebuildIndex). * fix(tools): scope skill_search BM25 index per-tenant SkillSearchTool held a single process-global BM25 index built once from whichever tenant's context first triggered ensureIndex, then reused for all subsequent Execute() calls regardless of caller — leaking one tenant's skill search results into another's, the search-path counterpart to the managed-directory bug fixed in 7b4668ad. - index/lastVersion are now keyed per-tenant (map[uuid.UUID]*tenantIndexState) - ensureIndex resolves the calling tenant from context and only builds/reads that tenant's index entry, never touching another tenant's cached state - Builtin/bundled skills remain visible in every tenant's index (Loader already merges those tiers correctly per 7b4668ad) Loader.Version() remains a single global counter — a version bump in one tenant causes unnecessary rebuilds in others but does not cause cross-tenant leakage, an acceptable tradeoff to avoid scope creep. Added TestSkillSearchTool_TenantIsolation proving two tenants with same-slug/different-content skills never see each other's search results, including after cache population from a different tenant. * fix(tools): fix group-spec expansion in tool policy engine PolicyEngine.registry was only ever set via SetRegistry(), which was never called in production (only in one test) — so pe.registry was permanently nil in production. Every group-expansion helper (applyProfile, intersectWithSpec, unionWithSpec, subtractSpec, expandSpec, matchDenySpec, filterByCapability) silently dropped any "group:*" spec entry instead of expanding it when registry was nil. Concretely: "group:mcp" (auto-injected into agentToolPolicy.AlsoAllow for any agent with MCP tools) never resolved to real tool names, so MCP tools connected successfully and appeared in prompt text (which reads the registry directly, bypassing PolicyEngine) but were never included in the actual ChatRequest.Tools payload sent to the LLM — confirmed live via mcp.agent.tools_loaded tools=6 immediately followed by mcp.filtered_tools mcp_defs_count=0 in the same request. This affects any agent relying on group-based grants, not just MCP. PolicyEngine is a shared/global singleton used concurrently across all agents (constructed once at gateway startup), so mutating a registry field per-call would be a data race. Fix instead threads the registry as an explicit parameter from FilterTools down through all internal group-expansion helpers, and adds IsDenied/WouldAllow registry parameters, removing the dead SetRegistry() mechanism entirely. Also fixes group expansion for the per-user-MCP-tools path: FilterTools is sometimes called with a userToolOverlay wrapping a *Registry rather than a *Registry directly; added Unwrap() to userToolOverlay so the new registry-resolution logic works for both cases. Added 4 tests proving group expansion works via the threaded parameter alone (no SetRegistry): plain registry allow, userToolOverlay allow, deny-side group expansion, and the WouldAllow bridge-server path. Blast radius note: this restores intended access for every agent configured with group:* specs (group:mcp, group:vault, group:goclaw, group:coding, etc.) that were silently inert before. Existing agent configs relying on group grants will gain the tool access they were nominally already configured for. * fix(mcp): cache tool descriptions from pool-connected servers too 5ce410b6 added tool-description caching (for prompt-preview visibility) only inside connectServer. connectViaPool — the separate connect path used when MCP connections go through the shared pool — never got the same caching hook, even though it shares the same underlying connectAndDiscover wire handshake. Confirmed live: cloudflare/docker connected via connectViaPool and received real descriptions over the wire, but prompt preview still showed blank descriptions because this path never wrote to the cache. Also removes the temporary mcp.connect.raw_tool debug log added earlier this session for diagnosing the same issue — no longer needed now that the root cause is fixed. * chore(skills): remove temporary pinned-skills diagnostic logging Confirmed live: pinned skills (caveman, infra-ansible-knowledge) now resolve correctly end-to-end for tenant-scoped agents. Debug logging added to trace the resolution chain is no longer needed. * fix(tools): deny always wins over AlsoAllow group grants AlsoAllow's unionWithSpec could reintroduce a tool explicitly listed in Deny, since it added tools back from allTools without re-checking deny specs. Previously masked because AlsoAllow's group-expansion was also broken (fixed in f7af95de this session) — group specs silently expanded to nothing, so this ordering bug never manifested. Now that group expansion works, an admin-denied tool that's also reachable via a group:* AlsoAllow entry (e.g. group:mcp) would silently reappear. Re-apply deny-spec subtraction as a final step after AlsoAllow union, for both global and per-agent policy, so deny always wins regardless of which allow mechanism tries to add a tool back. Added tests proving global and per-agent Deny correctly override an overlapping AlsoAllow group grant, while sibling non-denied tools in the same group remain allowed. * fix(mcp): enumerate cached tools instead of wildcard placeholder in prompt preview ListToolsForAgent (the prompt-preview path) collapsed any server with an empty ToolAllow (unrestricted grant — the common case) into a single "server__*" placeholder entry, even when tool_cache already had every real tool name and description from connect time (5ce410b6, 8ffd67b9). This meant agents with unrestricted MCP server access never saw individual tool names or descriptions in prompt preview, forcing trial-and-error tool usage. When ToolAllow is empty and tool_cache is populated, enumerate every cached tool (skipping any explicitly denied) and emit one MCPToolPreviewInfo per tool, matching the construction logic already used for the ToolAllow-non-empty case. Falls back to the single placeholder only when tool_cache is also empty (server never connected). The live (non-preview) conversation path, buildMCPToolDescs, does not have this bug — it resolves tool identity from the live connected registry, never from ToolAllow, so no placeholder shortcut exists there. Added tests covering both the cache-populated enumeration case and the no-cache placeholder fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(prompt): filter alias re-injection and add MCP schemas to preview ToolDefs Prompt-preview's tools: schema array (PreviewResult.ToolDefs) had two bugs, both preview-only — confirmed live conversations already use correctly-filtered tool payloads via buildToolsPayload/PolicyEngine.FilterTools, unaffected by either bug: 1. Alias re-injection iterated ALL registry aliases globally with no check against the deny-filtered toolNames list, letting denied tools reappear via their alias name (e.g. a denied canonical tool still showing up as its Claude-Code-compat alias like Bash/Edit/Write/Read). Now skips any alias whose canonical tool isn't in the filtered set. 2. MCP tool descriptions (from the store-based, connection-free ListToolsForAgent path) only ever fed the prompt TEXT section, never got converted into ToolDefinition schema objects — so the tools: array never showed MCP tools at all, even when the MCP section of the prompt text correctly listed them. Now appends a ToolDefinition per MCP tool with name+description populated and a placeholder {"type":"object"} Parameters schema, documented as preview-only (real parameter schemas require a live MCP connection, only available during an actual conversation turn). Added tests proving denied-tool-alias exclusion and MCP tool inclusion in preview ToolDefs. * feat(skills): inline full content for pinned skills instead of pointer-only Web UI documented "Pinned skills are always inlined in the system prompt", but BuildPinnedSummary was just BuildSummary with an allowlist filter — same as the general searchable skill list: name/description(truncated)/location pointer only, requiring use_skill+read_file round trips to get actual content. No code path inlined real SKILL.md content for pinned skills specifically. BuildPinnedSummary now reads and inlines full SKILL.md content (frontmatter stripped) per pinned skill inside <skill_instructions> tags. Per-skill (10000 bytes) and total (30000 bytes) size caps fall back to the original pointer-only format with a note when a skill is too large to inline, so oversized skills degrade gracefully instead of blowing the prompt budget. Added tests: full-content inlining, size-cap fallback, and tenant isolation for the new inline path (mirroring the existing managed- skills tenant isolation test). * fix(mcp): real parameter schemas and full policy enforcement in prompt preview Three interconnected fixes to prompt preview, none affecting the live conversation path (which was already correct): 1. Real MCP parameter schemas instead of a useless empty placeholder. tool_cache previously stored only name+description; extended to also capture the real JSON Schema from the MCP server's tools/list response (CachedToolInfo{Description, Parameters}) at connect time, for both direct-connect and pool-connect paths. Preview now shows complete, real input schemas instead of {"type":"object"} with no properties — verified against actual wire data from a live cloudflare MCP server showing genuinely rich schemas (zone_id, type, name, content, ttl, proxied, all typed with descriptions and correct required arrays) that were previously being discarded. Backward-compatible: old-shape cache entries degrade gracefully to description-only rather than crashing, self-healing on next connect. 2. Global tool deny now enforced in preview. BuildPreviewPrompt previously hand-rolled a partial policy reimplementation (per-agent deny only, explicitly skipping the full PolicyEngine "because runtime state isn't available in preview") — but PolicyEngine.WouldAllow already handles this per-tool-name without needing channel context. A tool denied via the global config (not per-agent) would appear in preview despite being correctly denied in every real conversation. Preview now calls WouldAllow per candidate tool, with a graceful per-agent-deny-only fallback when no PolicyEngine is wired (e.g. in tests). 3. MCP tools now also subject to the same policy check — previously the MCP tool supplement (store-based, connection-free tool listing) added MCP tool names unconditionally, bypassing WouldAllow entirely, so a denied MCP tool could still appear in preview. Added tests for all three: real-schema presence, global-deny exclusion for both core and MCP tools, and backward-compat cache handling. * fix(prompt): remove redundant per-tool MCP enumeration from prompt text Now that MCP tool schemas in the tools: API parameter are real and complete (1290d4f1), the ## MCP Tools prompt-text section's per-tool "- mcp_x__y: description" enumeration is pure duplication with zero added value — the model already gets each tool's real schema (including description) via tools:. buildMCPToolsInlineSection now keeps only the behavioral instructions that aren't expressible via JSON schema and thus aren't duplicated: prefer-MCP-over-core-tools guidance, and the optional-parameter guidance (don't guess/fill optional fields). The per-tool name+ description enumeration loop is removed. Section still only appears when the agent has MCP tools (len(cfg.MCPToolDescs) > 0, unchanged gate). Updated tests to assert the enumeration is gone while the behavioral instructions remain; ToolDefs assertions are now the authoritative check for MCP tool allow/deny filtering behavior (prompt text no longer enumerates names at all). * test(agent): update TeamContextInjection test for removed Team Members section TestBuildSystemPrompt_TeamContextInjection asserted the presence of a 'Team Members' prompt-text section that was intentionally removed in 71d33180 (duplicate of the canonical TEAM.md-context-file Members section, which has better formatting). The test was never updated to match, causing it to fail on every run since. Moved the assertion from wantIn to wantNotIn for the 3 affected subtests -- BuildSystemPrompt correctly no longer renders team-member roster info directly; that info now comes exclusively from the TEAM.md context-file mechanism, outside this test's isolated scope. * fix(prompt): resolve real registry for WouldAllow calls in preview BuildPreviewPrompt's two WouldAllow calls hardcoded reg=nil, silently breaking group:* expansion (e.g. group:mcp) needed to resolve the AlsoAllow grant production actually uses to grant MCP tool access (resolver_helpers.go's agentToolPolicyWithMCP injects AlsoAllow: ["group:mcp"]). With reg=nil, WouldAllow could match literal tool names fine (the 18 core/static tools) but could never resolve group-based grants, so every MCP tool silently failed WouldAllow and was excluded from preview -- confirmed live via curl: 18 tools returned, zero mcp_* ones, for an agent with genuinely working MCP access in real conversations. Live conversations were never affected -- internal/mcp/bridge_server.go's WouldAllow call already correctly passes a real registry. Fix resolves a real *tools.Registry from deps.ToolLister via tools.ResolveConcreteRegistry (the same helper used at the live call site), passing it to both WouldAllow calls instead of nil. Falls back to nil gracefully for test mocks that don't implement the full ToolExecutor interface, preserving existing test behavior. Added a test proving an MCP tool granted via the exact production AlsoAllow: ["group:mcp"] pattern is now correctly included in preview ToolDefs, where the old reg=nil bug would have silently excluded it. * fix(prompt): use literal deny check for MCP tools in preview, not group expansion The MCP-tools policy gate added in 1290d4f1 called WouldAllow with a real registry (per 9b5fd2eb), which requires group:mcp expansion against that registry to grant access via the production AlsoAllow: ["group:mcp"] pattern. But MCP tools are only ever registered into ephemeral per-agent registry clones at live connection time (manager_connect.go) -- never into the shared/global registry preview uses. group:mcp always resolved empty in preview's connection-free context, so WouldAllow denied every MCP tool -- confirmed live via tool-name diff: live conversations correctly included all 6 MCP tools, preview included zero. MCP access-granting is already correctly handled by ListToolsForAgent's own per-server tool_allow/tool_deny grant logic (confirmed working correctly earlier this session). The preview gate only needs to catch the narrower case of a literally-denied tool name via global/per-agent policy config -- it never needed group expansion. Replaced WouldAllow with IsDenied(nil, name, agentPolicy), which forces a pure literal-name match with zero registry dependency, matching the existing usage pattern already established elsewhere in policy.go. This class of bug cannot recur: there's no registry-passing code path left in this check to silently reintroduce group-expansion dependence. Added a test proving MCP tool inclusion in preview is independent of group-expansion outcome (no AlsoAllow: group:mcp needed for a non-denied tool to appear). Also reverts the temporary loop.filtered_tool_names/ preview_prompt.filtered_tool_names diagnostic logging used to capture the live-vs-preview tool-name comparison that diagnosed this bug. * fix(http): preserve real MCP parameter schemas through HTTP preview adapter mcpPreviewAdapter.ListToolsForAgent (the HTTP-layer glue converting mcp.MCPToolPreviewInfo to agent.MCPToolPreviewInfo for BuildPreviewPrompt) only copied RegisteredName and Description, silently dropping Parameters -- a bug present since this adapter was introduced (2499d0be/7e250244), unrelated to today's other MCP preview fixes. This was masked until d5fc6344 fixed MCP tools being excluded from preview entirely (a separate bug) -- once MCP tools started appearing again, this pre-existing adapter gap became visible: tools showed up correctly, but always with the bare {"type":"object"} placeholder instead of their real cached schema (confirmed live: update_dns_record missing its 7 real properties). One-line fix: copy Parameters through in the adapter's struct literal. Added a regression test constructing a real *mcp.Manager with populated tool_cache, asserting the adapter's output preserves specific real schema properties (not just non-nil Parameters) -- verified this test fails without the fix and passes with it. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Duc Nguyen
|
747b58d24c |
fix(usage): repair cost analytics and display precision (#1330)
fix(usage): repair cost analytics and display precision - Automatic OpenRouter pricing sync with cost backfill for traces/snapshots/events - Live usage data merging for current-hour dashboard accuracy - 2-decimal API cost formatting across usage/overview pages - Comprehensive test coverage (PG, SQLite, HTTP, UI) Merged by github-maintain automation. |
||
|
Bruno ClermontandBruno Clermont
|
2499d0be78 |
fix(mcp): show configured MCP tools in prompt preview (#1321)
* fix(mcp): show configured MCP tools in prompt preview MCP tools were only visible in prompts if currently loaded in registry (from active sessions). Prompt preview showed empty MCP tools section. Solution: Query MCP store for configured servers and tool lists. Show tools that would be available, not just currently loaded. - internal/mcp/manager.go: add ListToolsForAgent() for store-based tool discovery - internal/agent/preview_prompt.go: supplement live registry with store tools - internal/http/agents.go: add mcpPreviewMgr field and setter - internal/http/agents_prompt_preview.go: create adapter bridging mcp.Manager to preview - cmd/gateway.go: wire up MCPPreviewAdapter during setup * fix(mcp): load MCP servers from database at startup, not config file Single source of truth: MCP servers are now loaded from mcp_servers table at gateway startup, instead of from the config file. This ensures that MCP servers configured via web UI are automatically available without requiring config file updates. - cmd/gateway_setup.go: remove config fallback, add initMCPFromDB() helper - cmd/gateway.go: call initMCPFromDB() after store initialization - internal/mcp/manager.go: add SetConfigs() for runtime configuration Now mcpMgr will be non-nil when MCP servers are in the database, enabling MCP tools in prompt preview. --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> |
||
|
|
8963ec3177 |
fix(ollama): always send num_ctx to prevent context-window errors (#1307)
* feat(ollama): auto-detect context window, allow user override Query Ollama API (/api/show) to fetch model's actual context_length on provider startup/hot-registration. Use queried value by default, but allow user to override via num_ctx field in provider settings. - backend: query API, store in provider config, use in requests - frontend: add num_ctx input field to Ollama provider dialog (4 langs) - fallback: 131072 tokens (128K) if API unreachable Fixes context-window errors when using large-context models. * fix(ollama): route to /api/chat endpoint to honor num_ctx Ollama's /v1/chat/completions (OpenAI-compat shim) ignores options.num_ctx. Route to /api/chat (native endpoint) which respects num_ctx. Request body is identical, only endpoint URL changes: - Before: /v1/chat/completions (ignores options.num_ctx) - After: /api/chat (honors options.num_ctx) Fixes context-window errors when using models with large context windows. * refactor(providers): replace OpenAI-wrapped Ollama with native SDK Implement dedicated OllamaProvider using github.com/ollama/ollama/api official client library. Replaces the broken approach of wrapping Ollama behind OpenAI-compatible parsing. Benefits: - Proper response parsing (Ollama native format, not OpenAI) - Automatic /v1 suffix stripping - Built-in num_ctx support (injected into every request) - Tool call support via Ollama's native ToolCall schema - Token usage mapping from Ollama Metrics - Clean streaming implementation Fixes context-window errors by using Ollama native API directly. Removes dependency on OpenAI response format hacks. - new: internal/providers/ollama.go (OllamaProvider) - updated: cmd/gateway_providers.go, internal/http/providers.go - updated: go.mod (added github.com/ollama/ollama) * fix(ollama): fetch num_ctx for config-registered providers Config-file-registered Ollama providers (cfg.Providers.Ollama.Host) were hardcoding numCtx=nil, so num_ctx was never sent to Ollama. Add FetchOllamaModelContext call for config providers, same as DB-registered providers. Now config Ollama providers also auto-detect and send the model's context window to the Ollama API. Fixes context-window errors when using config-registered Ollama. * fix(ollama): always send num_ctx to prevent context window errors OllamaProvider.buildRequest only injected options.num_ctx when an explicit override was configured. When resolveOllamaNumCtx saw that /api/show returned the OllamaDefaultNumCtx value (131072), it returned nil, causing the provider to omit num_ctx entirely. Ollama then fell back to the model's baked-in default (4096 tokens), causing requests with long system prompts to fail with "request exceeds context size". Fix both layers: - resolveOllamaNumCtx now always returns &numCtx instead of nil when /api/show returns the default, so the explicit value is always passed. - OllamaProvider.buildRequest falls back to OllamaDefaultNumCtx (131072) when p.numCtx is nil, ensuring every request carries a large context window even if the startup query was skipped. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
978a0f255e |
Fix/skill violation display (#1304)
* feat(skills): display structured security violations in upload error When skill upload fails due to security violations, show each violation as a clear list with line number and reason in the error display. Changes: - Add ApiError.getViolations() method to extract violations from response - Update HTTP upload client to capture violations in error details - Add violations field to SkillEntry type - Display violations as bulleted list in SkillSubtitle when present - Add i18n keys for security violations display (en, vi, zh, ko) New UI behavior: Instead of a generic "Upload failed" message, users now see structured violation details: Security violations: - Line 5: SQL DROP TABLE - Line 12: curl pipe to shell Backward compatible - falls back to generic error message if violations array is absent from response. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(skills): return structured JSON violations array in upload error response Replace flat string formatting with a structured violations array (line, reason) so clients can display per-line security rule details instead of a generic error. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Quoc-Anh Nguyen
|
ce4535f305 |
feat(auth): inherit tenant_users.role for paired sessions (#1066)
Both WS pairing (router.go Path 3a) and HTTP X-GoClaw-Sender-Id auth were pinned to RoleOperator. Now they map tenant_users.role -> permissions.Role (owner/admin/operator|member/viewer) so a tenant admin who logs in via pairing actually gets admin in the dashboard. Falls back to RoleOperator when the user has no membership row (backward compatible). Tenant is inferred from a single membership when no hint is provided. Signed-off-by: imcvampire <anhnq0@gmail.com> |
||
|
|
e0f40a3878 |
fix(skills): normalize backslash in ZIP entry names for Windows compatibility (#1103)
* fix(skills): normalize backslash in ZIP entry names for Windows compatibility ZIP archives created on Windows may use backslash (\) as path separator instead of forward slash (/). When extracted on Linux, this causes files like 'scripts\search.py' to be created as a single flat file instead of the proper 'scripts/search.py' directory structure. The ZIP specification (PKZIP APPNOTE) requires forward slashes, but many Windows tools produce non-conforming archives. This adds a strings.ReplaceAll backslash normalization step during skill ZIP extraction to handle this edge case. * test(skills): cover backslash zip extraction --------- Co-authored-by: buidackim <221586880+buidackim@users.noreply.github.com> Co-authored-by: Mỹ Ngân <goclaw-maintain@users.noreply.github.com> |
||
|
Aaron VuandAaron Vu
|
c1ea0c0090 |
docs(openapi): cover the full HTTP API surface (#1171)
* docs(openapi): add vault endpoints + schemas
The committed OpenAPI spec only documented 41 of ~286 actual routes. This
adds the 19 Vault endpoints — 16 under /v1/vault/* and 3 under
/v1/agents/{agentID}/vault/* — plus their request/response schemas.
Routes covered (matches internal/http/vault_handler_*.go and
vault_graph_handler.go registrations):
- documents: list, create, get, update, delete (per-agent and cross-agent)
- documents/{docID}/links: outlinks + backlinks + name lookup
- links: create, delete, batch get outlinks
- upload: multipart, allowed exts whitelist
- rescan: per-tenant workspace rescan w/ concurrency guard (409)
- tree: lazy sidebar listing
- search: hybrid FTS + vector
- enrichment: status + stop
- graph: lightweight nodes/edges for visualization
Schemas: VaultDocument, VaultDocumentInput, VaultDocumentUpdateInput,
VaultLink, VaultBacklink, VaultSearchResult, VaultTreeEntry,
VaultGraphNode, VaultGraphEdge. Tags: Vault, Vault Graph.
Pure spec addition — no handler changes. The insertion is surgical
(no reformatting of existing entries), so the diff is additive only.
* docs(openapi): cover the remaining HTTP endpoints
Continues the gap-fill started for Vault. Adds 150 more path operations
spanning every other HTTP-route file under internal/http/, with 19 new
component schemas and 8 new tags. After this commit the spec covers 210
paths vs 60 previously committed (the actual handler surface is ~258).
Coverage by area (file → routes added):
knowledge_graph_handlers.go 9 (Knowledge Graph: extract, graph, stats,
dedup scan/dismiss, merge, single entity)
vault_graph_handler.go 1 (kg/graph/compact)
memory_handlers.go 6 (documents CRUD with {path...}, chunks,
index, index-all)
oauth.go 8 (chatgpt + openai: status, start,
callback, logout × 2)
storage.go 4 (files {path...} read/delete, size, move;
POST upload merged into the existing
/v1/storage/files block so the path key
stays unique and YAML-strict parsers like
Swagger UI's loader accept the spec)
agents.go 22 (sync-workspace, shares CRUD, regenerate /
resummon / cancel-summon, system-prompt-
preview, export {preview,download/token},
import {preview,merge}, teams export/import,
instances + metadata, orchestration,
v3-flags)
evolution_handlers.go 3 (metrics, suggestions, suggestion review)
episodic_handlers.go 2 (list, search)
skills.go 18 (versions, files, files/{path...}, upload,
grants agent/user × revoke, install-deps
trio, runtimes, toggle, tenant-config,
export {preview}, import)
mcp.go + mcp_user_credentials.go 16 (test, reconnect, grants + per-user creds,
requests CRUD + review, export {preview},
import, agent-grants list)
providers.go 4 (verify-embedding, codex-pool-activity,
claude-cli auth-status, embedding/status)
tenants.go + users 7 (CRUD + tenant users + tenant-users list,
users/search)
secure_cli.go + agent_grants 17 (cli-credentials full CRUD + presets +
check-binary + test + user-credentials +
agent-grants)
system_configs.go 4 (list, get, put, delete)
backup_handler.go + s3 + tenant 12 (system backup preflight/download/s3 set,
tenant backup + restore)
packages.go 6 (list, install/uninstall, runtimes,
github-releases, shell-deny-groups)
builtin_tools.go 3 (tenant-config get/set/delete)
tts.go + voices.go + caps 6 (config, synthesize, test-connection,
capabilities, voices, voices/refresh)
channel_instances.go +contacts +pending 10 (channel writers, writer groups,
contacts resolve / merge / unmerge / merged,
pending-messages)
files.go + workspace_upload 3 (files {path...} read, files/sign,
team workspace upload + move, team events,
team task attachment download)
edition.go + openapi.go 2 (edition info, openapi.json)
api-keys + traces + usage 3 (revoke, traces/{traceID}/export,
usage/breakdown)
New schemas: KGEntity, KGRelation, KGDedupCandidate, KGStats, TraversalResult,
MemoryDocument, MemoryChunk, MemorySearchHit, OAuthStatus, OAuthStartResponse,
Skill, McpServer, Tenant, TenantUser, CLICredential, Package, EvolutionMetric,
EvolutionSuggestion, EpisodicSummary.
New tags: Tenants, CLI Credentials, Packages, Contacts, Teams, TTS, Evolution,
Episodic Memory.
Purely additive — no existing operation, schema, or tag is modified. The one
overlap (`/v1/storage/files`, previously documented as GET-only) is handled
by inserting the new POST method inside the existing path block in place
rather than re-declaring the key, so each path key appears exactly once and
the spec parses cleanly under both JSON and strict YAML loaders.
Verified by parsing HEAD and the new file with python json.load + yaml.safe_load
and diffing entry-by-entry. Under `git diff --histogram`/`--patience` the diff
shows zero deletions; the default myers algorithm reports cosmetic deletion
lines but they are diff-algorithm noise, not real changes.
Where a body field or response shape was clear from the handler, the spec
captures it; where the handler accepts free-form / additive JSON payloads
the request body is documented as a generic object (additionalProperties:
true) rather than a fabricated rigid schema. Better to be honest about
flexibility than to lock callers into a wrong type.
* docs(openapi): drop nonexistent DELETE /v1/api-keys/{id}
The spec carried a DELETE /v1/api-keys/{id} operation, but
internal/http/api_keys.go registers no such route — only
GET /v1/api-keys, POST /v1/api-keys, and
POST /v1/api-keys/{id}/revoke. The DELETE entry predates this PR
(it was already on dev), so the semantic-preservation check left
it untouched; this removes the stale op so the spec matches the
registered routes. The correct POST /v1/api-keys/{id}/revoke
operation is unchanged.
Verified: json.load parses, go build ./internal/http/ and
go vet ./internal/http/ both clean.
---------
Co-authored-by: Aaron Vu <aaron-tsar@users.noreply.github.com>
|
||
|
thotam
|
cd5ad845f5 |
feat(webhooks): lease heartbeat to prevent duplicate async processing (#1275)
The async webhook worker double-processed long-running agent calls: the 90s stale-running sweep reclaimed rows whose agent ran longer than 90s (no heartbeat), re-running the agent and duplicating MCP side-effects. Add a last_heartbeat_at column + a Heartbeat lease-renewal method (CAS on lease_token). While an agent runs, the worker renews the lease every 30s; ReclaimStale now keys off last_heartbeat_at instead of started_at, so a live run is never reclaimed while a dead worker is still recovered within the stale window. If a run loses its lease (reclaimed), the heartbeat cancels the run context so the agent stops immediately and writes no further side-effects. - migrations/000085 (PG) + SQLite schema v52 + RequiredSchemaVersion 85 - WebhookCallStore.Heartbeat (PG + SQLite impls); ClaimNext/ReclaimStale switched to last_heartbeat_at - worker heartbeat goroutine + cancel-on-lease-loss; invokeAgent honors ctx - store + worker tests |
||
|
Zezae Oh
|
505b7a9d99 |
fix(agent): recover interrupted runs to failed on gateway startup (#1272)
A run records its terminal run.status (completed/failed/cancelled) only when the agent loop returns. If the gateway process is killed mid-run (restart, crash, or SIGKILL after a slow graceful shutdown), the terminal event is never emitted, so the run's last run.status item stays "started" forever — it shows as perpetually running in the timeline and is never counted as failed. Mirror the cron scheduler's startup reset (recomputeStaleJobs flips stale 'running' jobs to 'interrupted'): add RunTimelineStore.RecoverInterruptedRuns, invoked once on gateway startup. It finds runs that have a started run.status but no terminal sibling and appends a terminal failed run.status item, marked interrupted in metadata so it stays distinguishable from a genuine agent failure. Running only at startup means nothing from before is still executing, so there is no false-positive risk — unlike a periodic sweep (see the deliberately-disabled tracing stale-trace loop). - store: RecoverInterruptedRuns on the interface + PG and SQLite impls - cmd/gateway_managed: invoke once after the timeline recorder is built - tests: SQLite store test — orphan recovered, completed run untouched, idempotent on re-run |
||
|
thotam
|
e2ec3370dd |
feat(webhooks): stream provider responses for server-side runs to enable prompt caching (#1273)
Server-side webhook agent runs (sync, async, and admin test) used Stream:false, so OpenAI-compatible routers that only cache streaming requests never populated or served their prompt cache. Webhook runs paid full input-token price on every turn even with a stable session and an identical multi-turn prefix, while WS chat (Stream:true) got cache hits from the 3rd turn. Add gateway.webhook_stream (default true, env GOCLAW_WEBHOOK_STREAM) and apply it to all three run sites. ChatStream returns the fully assembled response, so the payload returned to the caller is unchanged. Set to false to restore non-streaming. |
||
|
thotam
|
06ef06ca37 |
feat(webhooks): paginate list + call-history endpoints (server + web UI) (#1268)
Add total-count pagination to the webhook admin endpoints and the web UI.
Store:
- WebhookStore/WebhookCallStore gain Count; WebhookListFilter gains
IncludeRevoked + Query (PG + SQLite, parameterized, tenant-scoped)
API:
- GET /v1/webhooks and GET /v1/webhooks/{id}/calls return
{items, total, limit, offset} with server-side search + revoked filtering
Web UI:
- server-driven list pager + search/revoked filter; call-history dialog uses
the real total (fixes the full-page "has more" boundary bug)
- i18n pager labels (en/vi/zh)
Tests: store pagination integration test; mock stores implement Count.
No schema migration (read-only COUNT).
|
||
|
thotam
|
a10464290d |
feat(webhooks): configurable agent run timeout (default 600s) (#1267)
Replace the hardcoded 30s webhook agent-run deadline with a configurable
timeout (default 600s, cap 3600s) for both the async worker and the
sync/test HTTP handler. Legitimate multi-step runs were dying at 30s
mid-tool-call; the same request over WebSocket completed fine.
- internal/webhooks/timeout.go: ResolveTimeoutSec helper (<=0 → 600s, cap 3600s)
- WorkerConfig.AsyncAgentTimeout (async worker) + WebhookLLMHandler.syncTimeout
(sync + admin test), wired from config
- config keys gateway.webhook_{async,sync}_timeout_sec, also settable via
GOCLAW_WEBHOOK_{ASYNC,SYNC}_TIMEOUT_SEC env (env overrides config)
- tests: timeout bounds + config file/env override
|