mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-11 03:13:24 +00:00
dev
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Kai (Tam Nhu) Tranandviettranx
|
abb10976f7 |
feat(codex-pool,create_image): collapse primary_first + route pools through create_image chain (#1006)
* refactor(codex-pool): remove redundant primary_first strategy * test(tools): update tool schema fixtures to pointer form * feat(create_image): route Codex pools through chain with member failover Codex pool chain entries now iterate pool members per the pool's own strategy (round_robin or priority_order) and fail over internally before the outer chain advances to the next entry. - ChatGPTOAuthRouter.GenerateImage implements NativeImageProvider: iterates orderedProviders, tries each member, advances round-robin state only on success, aggregates errors on pool exhaustion. - media_provider_chain.wrapPoolProvider wraps a *CodexProvider in a router when RoutingDefaults has extras or a non-primary-first strategy. Solo Codex (no extras) stays unwrapped. - Router exposes ProviderType() so chain telemetry records "chatgpt_oauth". Closes #1008 * feat(ui/builtin-tools): pool badge on create_image chain entries Chain entry card shows a read-only "Pool · <strategy>" badge when the entry's provider is a Codex pool base. Strategy label translates in en/vi/zh. No new toggle or selector — pool config lives on the provider itself. #1008 * refactor(providers): move Codex test helpers to providertest subpackage Addresses code-review High finding: NewTestCodexProviderFast and its staticTestTokenSource were exported from a non-_test.go file, compiling into the production binary. Relocating to internal/providers/providertest/ keeps the helper importable from other packages' tests without leaking test-only symbols into production. Also tightens wrapPoolProvider — pools with zero extra members no longer get wrapped in a router (KISS: nothing to rotate between). - Added CodexProvider.WithRetryConfig as a legitimate fluent option (the test helper now uses the public API). - Dropped the internal-package test helper file. #1008 * docs(pr-1006): add pool badge UI evidence Force-UI style capture of the read-only "Pool · Round-robin" badge on the create_image chain entry card when the selected provider carries settings.codex_pool with extras. Captured against staging gateway. * refactor(ui/builtin-tools): unify pool UX with Create Agent pattern The create_image chain Provider dropdown was listing pool members alongside pool owners, letting users accidentally bypass pool semantics by picking a member directly. Matches the pattern already used by Create Agent: hide pool members, show an inline "Pool" chip on owners. - Filter pool members from the chain Provider dropdown via getChatGPTOAuthPoolOwnership.ownerByMember. - Inline "Pool" chip on owner options, reusing the existing providers:list.poolBadge i18n key. - Drop the separate card-level "Pool · <strategy>" badge — the dropdown chip alone conveys the information without duplication. - Remove the now-unused isPoolProvider / poolStrategyOf helpers and the builtin.mediaChain.poolBadge* i18n keys we briefly introduced. #1008 * docs(pr-1006): add pool-filtered dropdown screenshot * fix(ui/builtin-tools): migrate stale pool-member chain entries on load Red-team blocker: a chain saved before the pool-aware UI landed may reference a pool member by name (e.g. openai-codex-2). After the dropdown started hiding members, such an entry produced: - empty Select trigger (no SelectItem matches the stored value) - conflicting Row 1 label still showing the member name - silent runtime misroute (bare solo call, no pool wrap) parseInitialEntries now consults getChatGPTOAuthPoolOwnership and rewrites any chain entry whose provider is a pool member to the owner's name + id. The next save persists the migrated value. Safe no-op for non-pool entries and for entries already pointing at an owner. Also memoize enabledProviders in the parent form so the card's useMemo boundaries actually hold (minor perf ding flagged by same review). * docs(pr-1006): refresh HTML evidence to match filter-based UX * fix(ui/agent-codex-pool): traffic policy reflects effective strategy under inherit On the agent's OpenAI Account Pool page, when Agent routing mode is Use Provider Defaults, the Traffic Policy buttons painted the draft's placeholder strategy ("priority_order") as selected — contradicting the top-of-page chip which already correctly shows the provider's effective strategy. The removal of primary_first in this PR unmasked the latent bug: the placeholder used to be a deprecated value that didn't match any live button, so nothing appeared selected. Derive selectedStrategy from defaultRouting when mode === "inherit": the button highlight now mirrors what actually runs. Buttons remain disabled in inherit mode (unchanged), but the displayed selection no longer misleads the user. * docs(pr-1006): add screenshot of inherit-mode Traffic Policy fix * fix(permissions): remove duplicate MethodSessionsCompact entry The writeExact slice listed MethodSessionsCompact twice. slices.Contains still returned correct results so runtime behavior is unchanged, but the duplicate entry was dead code. Spotted during review of PR #1006. --------- Co-authored-by: viettranx <edu@200lab.io> |
||
|
Kai (Tam Nhu) Tranandviettranx
|
4b02c27307 |
feat: native image_generation for Codex + OpenAI-compat providers (#1002)
* refactor(providers): migrate ToolDefinition.Function to pointer + add image response fields
ToolDefinition.Function becomes *ToolFunctionSchema with omitempty so native tool types (image_generation, web_search, etc.) can be declared without a function body. All 9 internal construction sites updated. CleanToolSchemas refactored — function-shape cleaning extracted into cleanFunctionSchema helper, outer pass-through handles native tools.
Added image response fields needed by the next commits: ChatResponse.Images, StreamChunk.Images, ImageContent.Partial (distinguishes partial frames from final images).
* feat(providers): native image_generation for Codex + OpenAI-compat tracks
Codex native (POST /codex/responses): emit image_generation tool object in request tools[] (type, action, model, output_format, partial_images). Handle SSE events response.image_generation_call.partial_image + response.output_item.done (type image_generation_call) + response.completed output[] walk for non-stream. Dedup per item_id. Extend codexSSEEvent/codexItem with output_format, result, partial_image_b64, partial_image_index.
OpenAI-compat (/v1/chat/completions): serialize ToolDefinition{Type:'image_generation'} as {type:'image_generation'} pass-through. Parse choices[0].message.images[] + delta.images[] (data URLs) via new parseDataURL helper; append to ChatResponse.Images.
ProviderCapabilities.ImageGeneration flag; Codex provider + adapter set true. Other providers default false.
* feat(agent,http,store): persist assistant images + tri-level image_generation gate
Agent loop tri-level gate: (provider capability) AND (AgentConfig.AllowImageGeneration, default true, stored in other_config.allow_image_generation) AND (request lacks x-goclaw-no-image-gen header). Gate in loop_tool_filter.go appends ToolDefinition{Type:'image_generation'} only when all three pass. Per-request opt-out parsed in chat_completions.go and propagated via RunRequest.NoImageGen.
Media persistence: persistAssistantImages writes final images (Partial:false) to {workspace}/media/{sha256}.{ext}, returns MediaRef entries, clears inline Images[] from the message. Idempotent on hash, traversal-safe, symlink-guarded. Invoked from pipeline.FinalizeStage via new Deps.PersistAssistantImages callback — covers both stream-final and non-stream paths.
Agent store reads AllowImageGeneration from other_config JSONB with absent/nil = true default (matches V3Flags pattern). No DB migration — code-only default.
* feat(ui/web): image_generation toggle + streaming placeholder + download filename
Composer chip 'Images' visible only when active agent's provider has ImageGeneration capability. Per-agent localStorage persistence via useImageGenToggle hook. When off, sends noImageGen:true to WS chat method (maps to x-goclaw-no-image-gen on upstream HTTP call path).
ActiveRunZone renders a skeleton placeholder while streaming partial_image frames arrive. MediaGallery assigns generated-YYYYMMDD-HHmmss.png as the download filename for hex/UUID PNGs.
i18n keys added to en/vi/zh chat.json: imageGenToggle, imageGenGenerating, imageGenDownloadName. 8 vitest tests for the toggle hook.
* docs: add Image Generation section to codebase-summary + changelog entry
Documents the new native image_generation pipeline across providers layer (Codex + OpenAI-compat), agent gate, media persistence, and web UI surface.
* fix(ui/web): match Codex-routed providers for image_generation toggle
Image-gen toggle visibility was hard-coded to provider id 'chatgpt_oauth' but real Codex-routed agents in production use provider ids like 'cliproxy-codex'. The toggle never rendered.
Replace the Set-has check with a small helper that accepts the literal ids plus any provider string containing 'codex' (case-insensitive). Same logic applied in both chat-input.tsx (composer chip) and chat-page.tsx (streaming placeholder gate).
Verified against a live Codex-routed agent: toggle now renders, noImageGen:true propagates on toggle-off.
* docs(pr-1002): targeted-mode UX evidence report
Captures the UI integration surface for native image_generation against a live Codex-routed agent on the remote dev backend.
Includes: composer toggle chip (rendered), streaming skeleton placeholder, and honest failure-path capture showing the legacy create_image builtin fallback. Self-contained HTML report in .github/pr-assets/1002/index.html.
* fix(permissions): classify sessions.compact as write method
CI RBAC-drift test (TestMethodRole_DriftCoverage_AllProtocolMethodsClassified) was failing because the new sessions.compact method added upstream was not classified in any of isPublicMethod / isAdminMethod / isWriteMethod / isReadMethod.
Sessions compaction mutates session history (compacts messages into summaries), so it belongs with the other sessions.* write methods.
* fix(tests): remove duplicate contains() in integration package
Both tts_gemini_live_test.go and mcp_grant_revoke_test.go declared a file-local func contains(s, substr string) bool with identical bodies, causing 'contains redeclared in this block' at compile time in the integration job.
Replace all call sites with strings.Contains (same semantics, stdlib) and drop the duplicates. No behavior change.
* feat(providers): NativeImageProvider interface + Codex implementation
Defines a provider-level contract (NativeImageProvider.GenerateImage) that OAuth-backed providers can implement to serve image generation without exposing static API credentials. Re-uses the PR's Track A native wire format (POST /codex/responses with image_generation tool, item.result decoding, SSE fallback).
CodexProvider + CodexAdapter implement it. Also adds MediaRef.Prompt field so downstream layers can propagate the generating prompt alongside the asset.
* feat(tools): route create_image through NativeImageProvider for OAuth providers
create_image.callProvider now checks for a NativeImageProvider implementation before the credentialProvider interface. When the provider chain points at a Codex-family provider (no static API key), the tool delegates to the provider's GenerateImage which executes the native ChatGPT Responses API image_generation flow.
Resolves 'provider X does not expose API credentials required for image generation' errors for openai-codex / cliproxy-codex chains. On success the tool embeds the user's prompt as a PNG tEXt 'Description' chunk (file-local pngEmbedPrompt helper to avoid tools→agent import cycle), writes the image to /tmp, and threads the prompt through result.MediaPrompts for downstream MediaRef propagation.
* feat(agent,pipeline): propagate image prompt through MediaRef + PNG tEXt embed helper
Adds EmbedPNGPrompt public helper in internal/agent/png_metadata.go that inserts a tEXt 'Description' chunk (plus 'Software: goclaw') into PNG byte streams before the IEND chunk. Non-PNG inputs are passed through unchanged — the helper never errors on unknown formats.
FinalizeStage wires MediaResult.Prompt (from create_image tool output) onto MediaRef.Prompt so the UI can render the generating prompt alongside the image. Per-image prompt list threaded via pipeline RunState.
* feat(ui/web): show generating prompt as caption under each image in MediaGallery
When a MediaRef carries a prompt, MediaGallery renders it as a muted italic caption (line-clamp-2) beneath the image with the full text in the title tooltip. Caption is hidden when the prompt is absent so non-assistant images (user uploads, legacy data) look unchanged.
MediaItem + session media_refs types extended with an optional prompt field; the chat-message adapter threads ref.prompt through when converting WS payloads to UI state.
* fix(providers/codex): stream:true + instructions for native image_generation
The ChatGPT Responses API on /codex/responses rejects two things hard:
- stream:false → HTTP 400 "Stream must be set to true"
- missing instructions → HTTP 400 "Instructions are required"
buildNativeImageRequestBody now sets stream:true and a purpose-specific instructions string ("Generate an image matching the user's description using the image_generation tool. Return only the image; do not describe it in text."). The existing parseNativeImageSSE path was already in place for stream parsing; routing changed from the non-stream branch to the SSE branch.
Regression assertions added to TestCodexGenerateImage_BuildsNativeRequest so these two fields can't silently regress.
* feat(providers,tools,ui): image_model whitelist (gpt-image-2 default, gpt-image-1.5 legacy)
Replaces the hardcoded "gpt-image-2" literal in buildNativeImageRequestBody with a user-configurable field threaded through NativeImageRequest.ImageModel. The whitelist is enforced by ValidateImageModel which rejects anything outside {gpt-image-2, gpt-image-1.5} with a clear error — prevents silent upstream 400s from model names the Responses API would reject.
create_image.callProvider reads params.image_model from the chain entry and threads it through. Empty / absent falls back to DefaultImageModel (gpt-image-2).
UI: added an 'Image model' select inside the existing openai-codex Settings panel on the Create Image Provider Chain dialog. Options: Default · gpt-image-2 (recommended) and Legacy · gpt-image-1.5. i18n keys in en/vi/zh tools.json under builtin.mediaChain.
Tests: TestCodexGenerateImage covers default/legacy/rejected model cases; TestCreateImageTool_ThreadsImageModel covers params→request threading with empty/legacy/explicit sub-cases.
* fix(tools): raise media chain default timeout to 600s/1 retry for image gen
gpt-image-2 on complex prompts (dense Vietnamese text, infographic layouts) legitimately takes 4–8 minutes to complete. The previous default of 120s × 2 retries routinely died mid-generation with 'context deadline exceeded' — the upstream run was still producing bytes when our ctx cancelled.
Default is now Timeout: 600 / MaxRetries: 1. Retries dropped to 1 because image generation is stateful per upstream run: a mid-flight timeout leaves orphan server work, and retrying a fresh generation doubles cost for no gain. Surface the failure fast so operators can widen the timeout instead.
Operators can still set a tighter value explicitly via the Chain dialog.
* refactor: remove user-facing Images toggle, keep admin-level AllowImageGeneration
The per-request opt-out toggle (composer chip + streaming placeholder + noImageGen header plumbing) was a support footgun — users toggle OFF, forget, then can't generate images and think it's broken. Removed in full.
Kept: AgentConfig.AllowImageGeneration (admin kill-switch, stored in other_config.allow_image_generation, default true). Tri-level gate simplifies to two tiers: provider capability AND agent config allows.
Removed: useImageGenToggle hook, IMAGE_GEN_PROVIDER_IDS set in chat-input, supportsImageGenProvider helper, agentProvider/agentKey props on ChatInput, showImageGenPlaceholder prop on MessageBubble/ActiveRunZone/ChatThread, noImageGen param on use-chat-send, parseNoImageGen in chat_completions.go, NoImageGen on RunRequest, no_image_gen_header_test.go, imageGenToggle/imageGenGenerating i18n keys. Kept imageGenDownloadName — used by MediaGallery for generated-\*.png filename resolution.
* docs(pr-1002): refreshed UX trace + updated codebase notes
Replaces the earlier stealth-state evidence with a clean three-capture trace from a real successful run: inline image + prompt caption, MediaGallery lightbox expansion, Chain dialog with the new Image model dropdown open. Skill-routing rows scrubbed from the capture — they reflect per-agent skill setup, not anything this PR introduces.
codebase-summary + changelog: reflect final state (toggle removed, image_model selector, 600s default chain timeout, gpt-image-2 as quality baseline).
* fix(pipeline): preserve mid-loop image_generation output across iterations
FinalizeStage previously read state.Think.LastResponse.Images, which holds
only the last iteration's response. If the LLM emitted image_generation_call
in iteration N alongside a function_call, then responded text-only in N+1,
the image from N was silently dropped on finalize.
Accumulate final (non-partial) images into state.Observe.AssistantImages
across every iteration via ObserveStage, and source FinalizeStage from the
accumulator instead of LastResponse. Partial streaming frames are filtered
defensively; response.Images is cleared on drain to prevent double-counting
on re-exec.
* test(pipeline): regression coverage for mid-loop image accumulation
Six ObserveStage cases covering image accumulation semantics:
- single-iter image-only, image+tool_call same iter, mid-loop image
surviving text-only final iter, multiple images across iters, partial
frame filtering, nil-response safety.
Two FinalizeStage cases verifying accumulator is the source of truth:
- PersistsFromObserveAccumulator: image in Observe + empty LastResponse
must still be persisted via PersistAssistantImages.
- NoPersistWhenAccumulatorEmpty: no call when no images were emitted.
---------
Co-authored-by: viettranx <viettranx@gmail.com>
|
||
|
viettranx
|
7701058de6 |
chore: remove obsolete PR assets
Clean up old PR screenshot/preview files that are no longer needed. |
||
|
Kai (Tam Nhu) Tran
|
cbbdbc992f |
feat(skills): multi-skill ZIP upload with hash-based idempotency (#846)
feat(skills): multi-skill ZIP upload with hash-based idempotency - Multi-skill ZIP detection: upload single ZIP containing multiple skill directories - Hash-based idempotency: SHA-256 of SKILL.md content deduplicates re-uploads - Grouped upload UI: ZIP group headers with per-skill badges (NEW/UNCHANGED/ERROR) - Safety: 50-skill per-ZIP limit, TOCTOU-safe hash check under lock - Performance: cached ZIP parsing O(N) vs O(N*M) - Tests: 17 frontend + 9 backend tests Closes #845 |
||
|
Kai (Tam Nhu) Tran
|
8730a90acb |
fix(exec): allow uploaded files in active workspaces (#748)
Shell-aware command parsing, dynamic workspace exemptions, and symlink canonicalization for exec path denial. Fixes #739. |
||
|
Kai (Tam Nhu) Tran
|
e22a870cba |
fix(chat): load message history on first conversation click (#730)
* fix(chat): load message history when selecting existing conversation from clean state The skipNextHistoryRef was unconditionally set when sessionKey transitioned from empty to non-empty. This prevented loadHistory() from running when clicking an existing conversation from the initial /chat page. The skip was only intended for the new-chat send flow where the optimistic message is already displayed. Guard the skip with expectingRunRef so it only activates when a message send is in flight. Closes #729 * docs: add UI diff evidence for PR #730 Before/after screenshots and HTML comparison report showing first conversation click behavior fix. |
||
|
|
e9733e08c4 |
fix(pool): improve pool management UX — clickable affordance, stale ref handling, managed-by banner (#671)
* fix(secure-cli): resolve ambiguous column in LookupByBinary JOIN query (#641) LookupByBinary uses LEFT JOIN with secure_cli_user_credentials but SELECT columns lacked table alias prefix, causing PostgreSQL error: "column reference 'id' is ambiguous (SQLSTATE 42702)" This silently broke ALL credentialed CLI exec — commands fell through to regular shell exec without injected env vars. Fix: use b.-prefixed column names for JOIN queries. Also add diagnostic logging to lookupCredentialedBinary for future debugging. * fix(agent): defer warning messages after parallel tool results (#644) When parallel tool calls trigger loop detection warnings, the warning messages (role="user") were inserted between tool result messages (role="tool"). This breaks the Anthropic API when routed through OpenAI-compatible proxies (e.g. LiteLLM): the proxy groups consecutive tool messages into a single user message with tool_result blocks, but an intervening user warning splits the group, causing orphaned tool_results and HTTP 400 "tool_use ids without tool_result blocks". Fix: accumulate warning messages during parallel result processing and append them after all tool results, preserving the consecutive grouping. Closes #642 * fix(docker): resolve @rollup/rollup-linux-arm64-musl missing on Alpine (#647) Added ui/web/.npmrc with supportedArchitectures for musl+glibc/arm64+x64. Updated Dockerfile to use --no-frozen-lockfile so pnpm fetches native rollup binding compatible with Alpine's musl libc. Lockfile still pinned by copy order. * docs(README): add history stars (#462) * fix(pool): skip stale pool member references during validation Unknown pool member references (deleted or disabled providers) now continue instead of returning an error. Prevents stale data from blocking provider saves. Closes #670 * fix(ui): redesign pool member selector and add managed-by banner Pool member selector: - Replace invisible outline button with custom element using dashed primary border, + icon badge, and "Click to add" hint text - Visible in both light and dark themes; hover transitions to solid border with shadow; active press scales down for tactile feedback Managed-by banner: - Show "Pool Defaults" section on pool members with info banner explaining which provider owns the pool, plus a Link navigation - Previously this section was completely hidden with no explanation i18n: add poolManagedByDescription and clickToAdd keys (en/vi/zh) * docs: add before/after UI evidence for PR #671 Annotated screenshots with red callout borders marking review areas. Self-contained HTML comparison report with dark/light theme toggle. * feat(ui): add pool discovery badges and setup wizard Replace verbose info banner with per-card "Pool available" badge on unpooled ChatGPT OAuth providers. Clicking the badge opens a new pool setup wizard dialog where users select owner, members, and strategy in one step. * docs: update UI evidence with pool discovery before/after * fix(ui): hide pool members from provider selector in agent forms Pool member providers are managed via the pool owner's routing config. Showing them as standalone options in the agent Provider dropdown is confusing — users may select a member directly instead of the owner, bypassing pool routing entirely. Filter out providers that exist in ownerByMember from the enabled providers list in ProviderModelSelect. * fix(ui): hide pool members from provider selector and add Pool badge Pool member providers are filtered out of the agent Provider dropdown in both the Create Agent dialog and the shared ProviderModelSelect component. Pool owners display a "Pool" badge so users know the provider routes to multiple accounts automatically. * docs: add provider selector before/after evidence * fix: revert stale merge in secure_cli.go and fix hardcoded i18n strings - Revert secureCLISelectColsAliased: b.agent_id → b.is_global (agent_id was dropped in migration 36, stale merge conflict artifact) - Replace hardcoded "Pool" badge text with t("providers:list.poolBadge") in provider-model-select and agent-identity-and-model-fields - Replace hardcoded "Disabled" with t("common:disabled") in pool wizard - Add list.poolBadge key to en/vi/zh locale files --------- Co-authored-by: Viet Tran <viettranx@gmail.com> Co-authored-by: Plateau Nguyen <nguyennlt.ncc@gmail.com> Co-authored-by: DNT <ducconit@gmail.com> |
||
|
Kai (Tam Nhu) Tran
|
4cd7b0a8d4 |
feat: add actionable channel health diagnostics and remediation (#634)
* feat: add channel health diagnostics * fix(channels): harden startup failure reporting - sanitize operator-facing failure details instead of exposing raw upstream errors - keep registered channel instances failed during reload/startup errors and clean up pre-registration failure entries - stop Telegram warning states from reporting running=true after polling or startup failures * feat(channels-ui): surface actionable health diagnostics - expand channel and overview views with richer health summaries, remediation hints, and failure history - refine the channel dashboard layout so diagnostics are easier to scan instead of reading a wall of text - keep shared web and desktop channel types aligned with the new health payload * fix(web): harden channel health diagnostics states - treat enabled channels without backend status as checking instead of stopped\n- align fallback remediation and overview attention synthesis with channel reality\n- localize diagnostics copy and reject invalid channel detail tabs\n\nRefs #628\nRefs #633 * docs(pr-assets): add channel diagnostics visual review pack |
||
|
Kai (Tam Nhu) Tran
|
343b530480 |
fix: cap tool call IDs to 40 chars via hash-based uniquification (#590)
Closes #532 - Replace prefix truncation with SHA-256 hash-based shortening for oversized tool call IDs (40-char OpenAI/Azure limit) - Normalize provider-prefixed model IDs (e.g. openai/o3-mini) before capability checks for temperature and max_completion_tokens - Add regression tests for ID collision, correlation, and prefixed model routing |
||
|
Kai (Tam Nhu) Tranandviettranx
|
287946bb9a |
fix(ui): restore provider-owned Codex pool inherit state (#585)
* fix(ui): restore provider-owned codex pool inherit state * docs(pr): add before-after UI evidence * refactor: remove dead hasProviderDefaults param and harden pool rendering - Remove unused _hasProviderDefaults parameter from buildDraftRouting and routingDraftSignature, clean up all call sites and useMemo deps - Filter deleted providers from selectedPoolProviderNames to prevent ghost entries when a saved pool member no longer exists - Add symmetric backend test for inherit + non-nil provider defaults --------- Co-authored-by: viettranx <viettranx@gmail.com> |