First real builtin under Phase 04 infrastructure. Registers two DB rows
(user_prompt_submit + pre_tool_use) on startup, runs the embedded JS with
source-tier gate + dispatcher dotted-path allowlist.
- internal/hooks/builtin/pii-redactor.js (ES5.1, goja-compatible):
handle(event) returns updatedInput only when email or E.164-ish phone
actually matched. Redacts rawInput + toolInput.{command,query,content}.
- builtins.yaml: pii-redactor entry (version 1, priority 900, timeout 2s,
on_timeout=allow — missing redaction < blocking conversation).
- Drop _placeholder.js; pii-redactor.js is now the first real //go:embed match.
- Export builtin.Source(name) so handler tests load embedded JS without
going through Load()/cache state.
- Tests:
* loader_test: registry contains pii-redactor after Load().
* script_pii_redactor_test (handler-level, real goja runtime): email
masked in rawInput; +E164 phone masked; toolInput.command redacted,
toolInput.path left alone (not in mutable_fields); no PII → no
mutation; redact(redact(s)) == redact(s).
* dispatcher_test: 3 dotted-path walker cases (toolInput.command only
lets that key through; rawInput-only doesn't let forged toolInput
override original event keys; toolInput wildcard merges all keys).
Bench: ~242µs/op on 1 KiB Apple M4 Max, well under <1ms target.
Two pre-existing issues kept the PG hook tests from running under the
documented flow (TEST_DATABASE_URL="postgres://postgres:test@localhost:5433/
goclaw_test?sslmode=disable"):
1. migrate.New path had one extra `../` — resolved above the repo root.
Fixed from `file://../../../../migrations` to `file://../../../migrations`.
2. `seedTenantAndAgent` derived slug + agent_key from `uuid[:8]`, and two
UUIDv7 calls in the same millisecond share their 8-char prefix. The
second tenant INSERT got silently swallowed by ON CONFLICT DO NOTHING on
the unique slug constraint, so the follow-up agent INSERT failed the
`agents_tenant_id_fkey` FK. Fixed by using the full UUID in the slug /
agent_key suffix.
Verified: `TestPGHookStore_*` (6 tests) now green under the integration
DB flow; `TestHooks*` under `-tags integration` still green.
Wave 1 Phase 03. Wires the Phase 02 Goja handler into the dispatcher, widens
the DB schema for script + builtin sources, and refactors Dispatcher.Fire to
return a FireResult so builtin-source hooks can mutate event input.
Dispatcher:
- FireResult{Decision, UpdatedToolInput, UpdatedRawInput} replaces the plain
Decision return. stdDispatcher.runSync keeps a local evMut copy so a
builtin-source hook's updatedInput flows to downstream hooks in the chain
and to the caller; non-builtin (source=ui) script mutations are dropped +
WARN-logged (defense-in-depth; source tier enforced at the dispatcher, not
only at the handler).
- applyBuiltinMutation + placeholder builtinAllowlistFor (rawInput, toolInput)
— Phase 04 overrides the allowlist from builtins.yaml.
- noopDispatcher returns FireResult{DecisionAllow}.
- Script mutation tests: TestDispatcher_ScriptMutation_BuiltinSourceApplies,
TestDispatcher_ScriptMutation_UISourceDenied.
Pipeline + callers (14 Fire sites refactored):
- FireHook wrapper returns FireResult; context_stage.go:52 applies
UpdatedRawInput → state.Input.Message; tool_stage.go:52 applies
UpdatedToolInput → tc.Arguments before ExecuteToolCall.
- delegate_bridge + delegate_tool keep the Decision branch; Updated* ignored.
- dispatcher_test / delegate_bridge_test / delegate_tool_hooks_test /
integration test helpers updated for the new shape.
Validation:
- edition_gate allows HandlerScript on every edition (sandboxed, no shell
escape surface).
- config.validateHandler rejects empty source, source > 32 KiB, goja compile
error; validateTimeout rejects on_timeout=ask|defer (reserved).
- Six new config tests cover the script path.
Migrations:
- PG migration 000053 relaxes handler_type + source CHECKs and drops
uq_hooks_{global,tenant,agent} — scripts routinely want many small hooks
per event. RequiredSchemaVersion → 53.
- SQLite SchemaVersion → 21; patch 20 is a SELECT 1 placeholder because
SQLite can't ALTER a CHECK — rebuildAgentHooksV21 runs outside the
migration tx (parallel to backfillV16) to rename/recreate the table with
widened CHECKs. schema.sql fresh-DB path matches.
- H9 fix: PGHookStore.Create + SqliteHookStore.Create honor caller-provided
cfg.ID (uuid.Nil falls back to UUIDv7), unblocking Phase 04 idempotent
UUIDv5 seed. TestCreateHonorsFixedID on both stores.
Config:
- config.HooksConfig{ScriptConcurrency, ScriptPerTenantConcurrency,
ScriptCacheSize, BuiltinDisable}; buildHookHandlers wires the script
handler with the caps from appCfg.Hooks.
Gates green: go build ./... + sqliteonly, go test -race -count=1 across
hooks/pipeline/sqlitestore.
Wave 1 Phase 01 — UI gates before script-handler work begins.
Backend:
- WS connect response adds `is_master_scope` (store.IsMasterScope with
role+tenant scoped ctx) and `edition` (edition.Current().Name).
Web:
- Auth store persists both values; ws-provider pipes them from ws-client
on connect and resets on disconnect.
- Hook form: removes `command` radio + legacy tooltip; scope options
hide `global` when caller is not master scope.
- Legacy rows with `handler_type="command"` coerce to `http` on edit
(phase 07 auto-disables them). Schema enum narrowed to http|prompt.
- i18n: drops 4 obsolete keys (form.command, form.allowedEnvVars,
form.cwd, form.commandDisabledTooltip) from en/vi/zh.
Security invariant: is_master_scope is an advisory UI hint only —
backend still enforces master scope on write paths.
Adds /hooks page with tenant-scoped list, create/edit dialog (per-handler
config forms for command/http/prompt), history table with diff viewer,
and dry-run test panel that calls hooks.test RPC. Sidebar entry gated by
edition/scope through existing auth store.
i18n: new hooks.json namespace + sidebar.json additions for en/vi/zh.
Routes + query-keys wired; no new runtime deps added.
Adds config.defaults WS method so UI can render pruning-knob placeholders
that mirror the server's internal fallback values. Exports
agent.DefaultPruningValues() as the single source of truth used by both
the resolver and this RPC — keeps UI in sync without a parallel table.
Exposes agent hooks over WebSocket (hooks.list/create/update/delete/
toggle/test/history) behind existing auth layer. Additions:
- Gateway wiring: buildHookHandlers() factory reused by dispatcher and
hooks.test runner, so UI test panel exercises production code paths
- GetByID tenant-scope guard: non-master callers only see own tenant +
global rows (matches List() behavior on both PG and SQLite)
- i18n keys + catalogs (en/vi/zh) for hooks error messages
- Protocol method constants for hooks.* and config.defaults
Adds PromptHandler that runs an LLM over matched hook events (user prompts,
tool calls) to produce decisions (allow/block/modify). Features:
- Budget integration: per-tenant token spend enforced via budget.Store
- Per-turn counter seeded in ContextStage and propagated through state.Ctx
so PreToolUse fires under the same cap as UserPromptSubmit
- prompt_template required (non-empty) at validate time to prevent
misconfigured hooks from silently no-oping
- Injection-hardened: tool_input/raw_input are quoted into system prompt
rather than concatenated
- Resolver abstraction (RegistryResolver) picks model from provider registry
with SystemConfigs fallback
Atomic budget store for prompt-handler LLM spend, with PG and SQLite
implementations. Enforces per-hook token caps to prevent runaway cost
from matcher-bypass scenarios.
Fixes#922 - Zalo OA bot was silently dropping all incoming messages due to JSON field mapping mismatches.
- Align zaloBotInfo/zaloFrom/zaloChat JSON tags with live API (display_name, chat_type)
- Change getUpdates to decode single-object response (API returns one update, not an array)
- Add dedicated pollClient with Timeout=0 + per-call context timeout for long-polling
- Update tests accordingly
Opt-in feature (features.auto_react: true) that automatically likes
a user's Facebook comment via Pancake /likes endpoint when the comment
webhook is received, as an engagement signal.
- Add AutoReact bool to Features config struct
- Add ReactComment() to APIClient (multipart POST to /likes, auth via api_key)
- Add reactCommentAsync() to comment handler; fires before keyword filter,
independent of comment_reply feature
- Bound with 10-slot semaphore + 5s stopCtx timeout per goroutine
- Guard: only fires on platform=facebook with non-empty convID + msgID
- Startup warning when auto_react=true without webhook_secret (F9)
- Update feature gate: exit only when BOTH auto_react AND comment_reply disabled
- Add 8 tests (ReactComment contract, error body, invalid IDs, handler variants)
SearchByEmbedding query uses COALESCE(description, '') without an AS
alias, causing PostgreSQL to name the column 'coalesce'. The sqlx scan
struct expects db:"description", so the scan fails with:
missing destination name coalesce in *[]pg.skillEmbeddingSearchRow
The BackfillEmbeddings query on line 84 already has the correct
AS description alias — this fix makes the search query consistent.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- webhook_handler.go: demote per-event "page_id resolution" log from Info to
Debug. Firing on every webhook was the exact anti-pattern called out in the
routing-metadata refactor review.
- pancake.go: move reply_to_comment_id guard above rememberOutboundEcho so a
missing-metadata error does not stamp phantom echoes that would pollute
inbound echo dedup.
- docs/journals: drop three Pancake journals (not needed in repo).
* feat(channels): make Pancake platform a required select with 11 options
- Add mandatory platform select field (11 options) to Pancake channel schema
- Add config required validation on channel instance form (create-only)
- Add i18n fieldOptions/fieldConfig for platform in en/vi/zh locales
- Add TDD tests for Pancake config schema (channel-schemas.test.ts)
- Add backend test TestFactoryExplicitPlatformPreserved
- Add slog.Debug for auto-detect path in pancake.go
- Update Platform field comment in types.go for clarity
- Update plan status to completed; add changelog entry
* feat(channels): hide comment_reply for non-social Pancake platforms
Extend showWhen to accept string | string[] values. Apply it to
features.comment_reply so the field only appears for platforms that
support public posts/comments (facebook, instagram, threads, tiktok,
youtube). E-commerce platforms (shopee, lazada, tokopedia) and
messaging-only platforms (line, google, chat_plugin) no longer show
the irrelevant Comment Reply toggle.
Also guard depValue before String() coercion to avoid the "undefined"
literal matching hazard.
* feat(channels): Pancake comment reply fix + platform select + UI improvements
- Fix comment reply: pass message_id (reply_to_comment_id) to Pancake API
ReplyComment now requires messageID param; guard added for empty ID
- Add SendMessageRequest.MessageID field (omitempty) for reply_comment action
- Platform select: required field with 11 options, showWhen gate for comment_reply
- Webhook Page ID moved to Advanced collapsible section in channel form
(auto-expands when existing value is configured)
- routing_metadata.go: centralize routing metadata key constants
- config-flatten.ts: flatten/unflatten nested config for form state
Closes: Pancake comment reply returns 'Missing required field: message_id'
Phase 2 (a587231b) advertised SSRF hardening for the http hook handler
but the supporting `internal/security` package was never created, so the
production HTTPHandler fell back to a bare http.Client and admin-config
webhooks could probe loopback / link-local / RFC1918 / cloud-metadata.
- internal/security/ssrf.go: Validate(rawURL) parses + resolves once,
rejects loopback/link-local/private/multicast/unspecified + 169.254.169.254;
NewSafeClient(timeout) returns an http.Client whose DialContext pins the
resolved IP from context (defense-in-depth re-checks the dialed IP) and
refuses redirects (CheckRedirect = ErrUseLastResponse)
- internal/hooks/handlers/http_handler.go: call Validate before each request,
attach pinned IP via security.WithPinnedIP(ctx, ip)
- cmd/gateway_managed.go: construct HTTPHandler with
Client: security.NewSafeClient(10*time.Second)
- tests: 13 new ssrf_test.go cases (every block category + redirect refused
+ dial pinned); existing http_test.go retrofitted with
security.SetAllowLoopbackForTest helper for httptest.NewServer
- plan: phase-02 Step 2a marked done with reference to this commit
Refs: GitHub Issue #875
The root cause is missing default initialization of JSON field in update flow, leading to NOT NULL violations in both SQLite and Postgres.
Fix ensures consistent behavior across Lite and Standard deployments.
Co-authored-by: Viet Tran <viettranx@gmail.com>
Co-authored-by: Chunning Ha <cn@ha.td>
Rework of #911 against dev with correctness fixes.
Public-facing channels (Facebook, Telegram, Discord, Feishu, WhatsApp,
Zalo OA, Zalo Personal, Pancake, Slack) no longer receive raw internal
error text when an agent run fails — an empty outbound is published
instead so channels still clean up placeholders / typing indicators.
Errors continue to be logged server-side at Error level.
Applied on both hot paths:
- cmd/gateway_consumer_normal.go (agent run failure)
- cmd/gateway_subagent_announce_queue.go (announce lead run failure)
Fixes vs #911:
- Whitelist uses channels.Type* constants (compile-time safe) instead
of string literals. The original PR's literals ("zalo", "line") did
not match any real channel type, so Zalo OA/Personal and Pancake
leaked errors while "line" was a dead branch.
- Added TypePancake and TypeSlack which were missing.
- Empty Content in Facebook.Send and Pancake.Send now short-circuits
(matching Telegram/Discord/Slack). Previously, suppressed errors
flowed to Graph/Pancake APIs as empty payloads, triggering 400s and
channel health degradation.
- Added table-driven test for isExternalChannel covering all real
channel types + empty + unknown + legacy "zalo" short form.
Closes#911
A single hung ch.Start() (e.g. Telegram GetMe stalled on DNS/TCP) blocks
InstanceLoader.mu and wedges every subsequent Reload(), leaving new
channels stuck in "checking" forever.
Wrap Start() in a goroutine bounded by reloadStartTimeout (90s) so Reload
can move on to the next instance. On timeout, Stop the channel in a
bounded window, record a typed failure for the UI, and asynchronously
drain the late-returning Start to detect channels that ignore context
cancellation.
Critically, pass the caller's ctx — not a timeout-wrapped ctx — into
ch.Start. Channels routinely derive long-running goroutines from the
ctx they receive (Telegram's pollCtx is WithCancel(ctx)). A timeout-
wrapped ctx would silently kill polling 90s after a successful Reload.
Add regression tests covering both the hang-then-timeout path and the
caller-context preservation invariant.
Closes#914
Gemini 2.5/3 default to high thinking via OpenAI-compat, consuming the
entire max_tokens budget and truncating titles to a single word (and
adding latency). Title generation is a trivial task that does not
benefit from reasoning, so set OptThinkingLevel="off" and bump
max_tokens from 50 to 256 for headroom across reasoning-capable
providers.
Also update mapGeminiReasoningEffort to forward "off" as "low" (the
minimum effort all Gemini models accept via OpenAI-compat), since not
forwarding causes the server to fall back to "high".
Red-team feedback: the new `name` field on role=tool messages was
being sent to ALL OpenAI-compat hosts including Together, Groq, vLLM.
Those hosts typically reject unknown fields with HTTP 400 — Gemini
is the only backend that requires `name` (via FunctionResponse.name
in Google's OpenAI-compat shim). Scope the field to Gemini-only via
supportsThoughtSignature, matching how we already gate
thought_signature forwarding.
Trace: 019d8f33-2de1-7ab2-9a32-9df92cd610dd
The previous MANDATORY phrasing ("MUST ALSO call write_file for
USER.md and BOOTSTRAP.md before your response ends") was making
small models emit write_file({}) when they had no real user info
to persist, tripping HTTP 400 on Google's Gemini OpenAI-compat shim.
Rewrite the predefined+BOOTSTRAP.md branch with softer "get to know
the user" copy plus explicit hard rules forbidding empty/placeholder
arguments and session-identifier content in USER.md. Safety net is
preserved: the USER PROFILE INCOMPLETE branch nudges the model every
subsequent turn until USER.md is populated.
Sync BOOTSTRAP_PREDEFINED.md template to remove the contradictory
MANDATORY block and replace with matching hard rules.
Trace: 019d8f33-2de1-7ab2-9a32-9df92cd610dd
Gemini returns finish_reason="tool_calls" (not "length") even when the
thinking budget exhausted max_tokens before tool arguments could be
emitted. Extend the existing truncation retry condition to detect this
case via an allowlist of mutating tools (write_file, edit, exec,
create_image, read_file) that virtually never legitimately call with
empty args. Nullary tools (datetime, heartbeat) pass through unchanged.
Trace: 019d8f33-2de1-7ab2-9a32-9df92cd610dd
Gemini 3 defaults to "high" thinking budget internally when no
reasoning_effort is set, consuming the full max_tokens budget and
leaving no tokens for tool call arguments on small models. Map
OptThinkingLevel to reasoning_effort on Gemini routes (native endpoint
or proxy-by-model), with "medium" downgraded to "high" per Gemini 3
Preview constraint. Non-Gemini OpenAI-compat hosts unchanged.
Trace: 019d8f33-2de1-7ab2-9a32-9df92cd610dd
Google Gemini's OpenAI-compat shim maps role=tool messages to native
FunctionResponse{name, response}; an empty name trips HTTP 400
("Name cannot be empty"). Build raw-ID → tool-name index at serialize
time and populate the name field on role=tool wire messages.
Trace: 019d8f33-2de1-7ab2-9a32-9df92cd610dd
Port goclaw context pruning to match upstream TS design in
openclaw/src/agents/pi-hooks/context-pruning/:
- Opt-in default: prune only when mode="cache-ttl" (was opt-out)
- Remove Pass 0 per-result 30% guard (duplicated Pass 1 with different
suffix, caused wobble)
- Dedupe double prune call per iteration: PruneStage owns the single
entry point; loop_history only runs limitHistoryTurns + sanitizeHistory
- Add cache-TTL gate for Anthropic prompt cache: skip prune while cache
is live, scoped per-session via sync.Map
- Add context.pruned event emission for observability
- Configurable TTL as Go duration string ("5m", "30s")
BREAKING CHANGE: context pruning now opt-in. Add
contextPruning.mode: "cache-ttl" to config.agents.defaults to restore.
Migration 51 / SQLite v19 backfills mode="cache-ttl" for agents with
existing custom context_pruning config missing the mode field, so
previously-configured agents keep pruning after the opt-in flip.
NULL configs stay NULL (new opt-in default applies).
Web UI adds Cache TTL input + toggle wiring mode to cache-ttl/off.
Add tts namespace to desktop i18n (en/vi/zh). Extend tools.json with STT form keys including WhatsApp privacy banner. Mirrors web ui/web locale structure.
Port web voice picker and STT provider form to desktop frontend. Singleton audio preview (hide when preview_url null), whatsapp_enabled toggle with privacy banner. Reuses desktop's custom Combobox primitive. Voice id persists via other_config.tts_voice_id merge in AgentDetailPanel. ToolSettingsDialog routes stt tool to new SttProviderForm.
Add stt-provider-form.tsx React component with form fields for model, language, and API key configuration. Mirror TTS provider form structure for consistency. Integrate into builtin-tool-settings-dialog.tsx. Includes comprehensive unit tests for form validation, submission, and error handling.
viettranx