diff --git a/README.md b/README.md index 3bc3f421..a75d3d6f 100644 --- a/README.md +++ b/README.md @@ -147,6 +147,9 @@ The dashboard provides visual management for all account types: **Ollama Integration**: Run local open-source models (qwen3-coder, gpt-oss:20b) with full privacy. Use `ccs api create --preset ollama` - requires [Ollama v0.14.0+](https://ollama.com) installed. For cloud models, use `ccs api create --preset ollama-cloud`. +> **Third-party WebSearch steering:** Claude-backed third-party launches keep Anthropic's native `WebSearch` disabled, provision `ccs-websearch.WebSearch` when the managed runtime is available, and append a short system hint so Claude prefers that managed tool over ad hoc Bash or `curl` lookups whenever current web information is needed. +> Setting `websearch.enabled: false` disables the managed local runtime, but CCS still suppresses Anthropic's native `WebSearch` on third-party backends because those providers cannot execute it correctly. + > **Copilot config behavior:** Opening the dashboard or other read-only Copilot endpoints does not rewrite `~/.ccs/copilot.settings.json`. If CCS detects deprecated Copilot model IDs such as `raptor-mini`, it shows warnings immediately and only persists replacements when you explicitly save the Copilot configuration. **llama.cpp Integration**: Run a local llama.cpp OpenAI-compatible server and create a profile with `ccs api create --preset llamacpp`. CCS defaults to `http://127.0.0.1:8080`, matching the standard llama.cpp server port. @@ -630,18 +633,18 @@ Without Developer Mode, CCS falls back to copying directories. ## WebSearch -Third-party profiles (Gemini, Codex, GLM, etc.) cannot use Anthropic's native WebSearch. CCS intercepts those requests and resolves them through real local search backends instead of depending on another model CLI to do the search. +Third-party profiles (Gemini, Codex, GLM, etc.) cannot use Anthropic's native WebSearch. CCS now provisions a first-class local `ccs-websearch` MCP tool when the managed runtime is available, disables native `WebSearch` on third-party launches, and steers Claude toward real local providers instead of surfacing a denied native-tool call. ### How It Works | Profile Type | WebSearch Method | |--------------|------------------| | Claude (native) | Anthropic WebSearch API | -| Third-party profiles | Local Search Backend Chain | +| Third-party profiles | CCS local MCP `WebSearch` tool when available; otherwise Bash/network fallback | ### Local Search Backend Chain -CCS intercepts WebSearch requests and routes them through deterministic search providers: +For third-party profiles, CCS steers Claude toward the managed `ccs-websearch.WebSearch` MCP tool when it is available. The tool is intentionally named to match the native `WebSearch` concept, which helps Claude prefer it over ad hoc Bash or `curl` fetches, but Bash/network fallback can still happen if the tool is unavailable or ignored. When the tool is used, CCS routes that request through deterministic search providers in this order: | Priority | Provider | Setup | Notes | |----------|----------|-------|-------| @@ -673,6 +676,10 @@ websearch: > [!TIP] > **DuckDuckGo** still works out of the box. Add **Exa**, **Tavily**, or **Brave Search** if you want API-backed results, then keep Gemini/OpenCode/Grok only if you explicitly want legacy fallback behavior. +> CCS manages the user-scope MCP entry in `~/.claude.json` and syncs it into isolated account configs when needed. + +> [!NOTE] +> Set `CCS_WEBSEARCH_TRACE=1` to write correlated launch, MCP, provider, and headless summary records to `~/.ccs/logs/websearch-trace.jsonl`. That trace is designed to answer whether CCS exposed the managed tool, whether Claude called it, which provider won, and when a headless run likely bypassed it via `Bash` or `WebFetch`. See [docs/websearch.md](./docs/websearch.md) for detailed configuration and troubleshooting. diff --git a/docs/project-overview-pdr.md b/docs/project-overview-pdr.md index 744de589..9df5d38a 100644 --- a/docs/project-overview-pdr.md +++ b/docs/project-overview-pdr.md @@ -35,7 +35,7 @@ CCS provides: 3. **AI Providers**: Dedicated CLIProxy dashboard for Gemini, Codex, Claude, Vertex, and OpenAI-compatible API-key families 4. **API Profiles**: GLM, Kimi, OpenRouter, any Anthropic-compatible API 5. **Visual Dashboard**: React SPA for configuration management -6. **Automatic WebSearch**: Real backend fallback chain for third-party providers +6. **Automatic WebSearch**: First-class local WebSearch tool with deterministic provider chain for third-party providers 7. **Usage Analytics**: Token tracking, cost analysis, model breakdown 8. **Official Claude Channels**: Runtime auto-enable plus dashboard token/config flow for Telegram, Discord, and macOS-only iMessage @@ -93,7 +93,8 @@ CCS provides: - Validate symlinks and permissions ### FR-007: WebSearch Fallback -- Intercept WebSearch for third-party profiles that cannot reach Anthropic's native tool +- Expose a CCS-managed local WebSearch tool for third-party profiles that cannot reach Anthropic's native tool +- Suppress native `WebSearch` on third-party launches and steer Claude toward the CCS-owned path when it is available - Support Exa, Tavily, Brave, and DuckDuckGo real search backends - Keep Gemini CLI, OpenCode, and Grok as optional legacy fallback - Graceful fallback chain diff --git a/docs/project-roadmap.md b/docs/project-roadmap.md index eee18ec2..5ff51ac3 100644 --- a/docs/project-roadmap.md +++ b/docs/project-roadmap.md @@ -1,6 +1,6 @@ # CCS Project Roadmap -Last Updated: 2026-03-28 +Last Updated: 2026-03-30 Forward-looking roadmap documenting current priorities, GitHub issues, and future feature plans. @@ -41,6 +41,7 @@ All major modularization work is complete. The codebase evolved from monolithic ### Recent Fixes +- **2026-03-30**: **#862** Third-party WebSearch now uses a first-class CCS-managed MCP tool path instead of relying on a denied native Anthropic `WebSearch` call as the normal UX. CCS provisions `ccs-websearch` into `~/.claude.json`, syncs it into isolated account configs when needed, suppresses native `WebSearch` on third-party launches, preserves the provider order `Exa -> Tavily -> Brave -> DuckDuckGo -> legacy CLI fallback`, and keeps the old hook runtime only as shared provider plumbing plus compatibility fallback. Uninstall cleanup now also removes the managed WebSearch MCP runtime. - **2026-03-28**: **#773** CCS now ships a dedicated `Compatible -> Codex CLI` dashboard route with a real split-view control center. The page detects the local Codex binary, keeps overview/docs guidance, and adds guided editors for the user-owned `~/.codex/config.toml` layer: top-level runtime defaults, project trust, profiles, model providers, MCP servers, and supported feature flags. Structured saves intentionally normalize TOML formatting and drop comments, so the raw editor remains the fidelity escape hatch. Follow-up fixes added immediate raw snapshot refresh, refresh/discard recovery for stale raw drafts, dirty raw-editor guarding for structured controls, project-trust path validation, read-only handling for unreadable config files, preservation of unsupported upstream values such as granular `approval_policy`, and feature reset-to-default support. CCS still warns that transient runtime overrides such as `codex -c key=value` and `CCS_CODEX_API_KEY` may change effective behavior without persisting into the file. - **2026-03-27**: WebSearch dashboard cards now manage Exa, Tavily, and Brave API keys inline instead of relying on a separate manual env step. CCS stores those secrets through `global_env`, reflects masked key state in `/api/websearch`, and counts dashboard-managed keys as ready in the WebSearch status flow. - **2026-03-27**: **#812** CCS now includes a first-class `ccs docker` command suite for self-hosting the integrated Dashboard + CLIProxy stack. The CLI can stage bundled Docker assets locally or to a remote `--host` over SSH, report compose/supervisor status, stream CCS or CLIProxy logs, and run in-container update flows without relying on ad-hoc deployment scripts. diff --git a/docs/websearch.md b/docs/websearch.md index 914200b1..5421d0eb 100644 --- a/docs/websearch.md +++ b/docs/websearch.md @@ -1,6 +1,6 @@ # WebSearch Configuration Guide -Last Updated: 2026-03-27 +Last Updated: 2026-03-30 CCS provides automatic web search for third-party profiles that cannot access Anthropic's native WebSearch API. @@ -12,7 +12,7 @@ Native Claude subscription accounts still use Anthropic's server-side WebSearch ### Third-Party Profiles -Third-party profiles cannot execute Anthropic's server-side WebSearch because the tool never reaches their backend. CCS now solves that by intercepting WebSearch and running real local search providers directly. +Third-party profiles cannot execute Anthropic's server-side WebSearch because the tool never reaches their backend. CCS now handles that by provisioning a first-class local MCP tool when the managed runtime is available, suppressing native `WebSearch` for those launches, appending a short launch-time steering hint, and running real local search providers directly. ## Architecture @@ -20,30 +20,45 @@ Third-party profiles cannot execute Anthropic's server-side WebSearch because th ┌──────────────────────────────────────────────────────────────┐ │ Claude Code CLI │ │ │ -│ WebSearch Tool Request │ +│ Search Request │ │ │ │ │ ├── Native Claude Account? → Anthropic WebSearch API │ │ │ │ -│ └── Third-party Profile? → PreToolUse Hook │ +│ └── Third-party Profile? → native WebSearch disabled │ │ │ │ -│ ├── 1. Exa Search API │ -│ ├── 2. Tavily Search API │ -│ ├── 3. Brave Search API │ -│ ├── 4. DuckDuckGo HTML │ -│ └── 5. Legacy CLI fallback │ -│ (Gemini/OpenCode/Grok) │ +│ ├── CCS MCP tool when ready│ +│ │ ccs-websearch.WebSearch│ +│ │ │ │ +│ │ ├── 1. Exa │ +│ │ ├── 2. Tavily│ +│ │ ├── 3. Brave │ +│ │ ├── 4. DuckDuckGo│ +│ │ └── 5. Legacy CLI│ +│ │ fallback │ +│ │ (Gemini/ │ +│ │ OpenCode/ │ +│ │ Grok) │ +│ └── Bash/network fallback │ └──────────────────────────────────────────────────────────────┘ ``` ## Why This Changed -The previous design asked another model CLI to perform web search and summarize the answer. That was brittle: +The previous design asked another model CLI to perform web search and summarize the answer. A later compatibility path also depended on a denied native-tool hook. Both were brittle: - CLI syntax changed upstream - auth state varied per tool - prompt/tool behavior drifted across releases +- hook-shaped denial output produced awkward host UX -The new flow matches the `goclaw` model more closely: web search is treated as a first-class deterministic capability, not an LLM-to-LLM workaround. +The new flow matches the `goclaw` model more closely: web search is treated as a first-class deterministic capability, not an LLM-to-LLM workaround or a denied native tool call. + +When provisioned, the managed MCP tool is exposed as `ccs-websearch.WebSearch`, not a generic `search` helper. That naming is deliberate: it gives Claude a tool that matches the native `WebSearch` concept more directly, which should reduce cases where the model reaches for ad hoc Bash or `curl` fetches instead. + +CCS also appends a third-party-only `--append-system-prompt` hint telling Claude to prefer that managed `WebSearch` tool for web lookups and current-information requests. This is soft steering only: if the user explicitly asks for shell commands, or the tool is unavailable, Claude can still fall back to Bash/network tools. +That shared launch helper applies to normal third-party settings profiles, CLIProxy/Copilot-backed Claude launches, and CCS headless/delegation runs that execute through a settings profile. + +`websearch.enabled: false` disables the managed local runtime, but CCS still suppresses Anthropic's native `WebSearch` on third-party profiles. That native tool cannot be satisfied by Exa, Tavily, Brave, DuckDuckGo, or other non-Anthropic backends, so CCS avoids sending a broken native-tool request and lets Claude fall back to normal shell/network tools instead. ## Providers @@ -101,6 +116,8 @@ websearch: timeout: 55 ``` +Note: `enabled: false` stops provisioning the managed local `ccs-websearch.WebSearch` runtime. It does not re-enable Anthropic's native `WebSearch` for third-party backends. + ## Environment Variables | Variable | Description | @@ -109,8 +126,16 @@ websearch: | `TAVILY_API_KEY` | Enables Tavily when `providers.tavily.enabled: true` | | `BRAVE_API_KEY` | Enables Brave Search when `providers.brave.enabled: true` | | `GROK_API_KEY` | Required only for legacy Grok CLI fallback | -| `CCS_WEBSEARCH_SKIP` | Skip hook entirely | -| `CCS_DEBUG` | Verbose hook logging | +| `CCS_WEBSEARCH_SKIP` | Disable the CCS local WebSearch runtime for the current process; third-party launches still keep native Anthropic `WebSearch` disabled | +| `CCS_DEBUG` | Verbose WebSearch runtime logging | +| `CCS_WEBSEARCH_TRACE` | Write opt-in JSONL trace records under `~/.ccs/logs/websearch-trace.jsonl` | +| `CCS_WEBSEARCH_TRACE_FILE` | Override the trace file path (must stay inside `~/.ccs/`, your system temp directory, or `/var/log`) | + +## Managed Runtime Files + +- `~/.claude.json` → CCS manages `mcpServers.ccs-websearch` +- `~/.ccs/mcp/ccs-websearch-server.cjs` → local MCP server binary +- `~/.ccs/hooks/websearch-transformer.cjs` → shared provider runtime plus legacy compatibility fallback ## Troubleshooting @@ -135,12 +160,23 @@ If the dashboard says the key is stored but still not ready, check whether `Sett Those providers remain supported, but they are no longer the primary path. Enable them explicitly in `config.yaml` if you want them as last-resort fallback. +### I need to see whether CCS exposed WebSearch or the model bypassed it + +Run the launch with `CCS_WEBSEARCH_TRACE=1` (or `CCS_DEBUG=1`). CCS writes a JSONL trace to `~/.ccs/logs/websearch-trace.jsonl` with: + +1. source-side launch records from CCS (`ccs_websearch_launch`) +2. MCP exposure and call records (`mcp_initialize`, `mcp_tools_list`, `mcp_tool_call_*`) +3. provider attempt and winner records (`websearch_provider_attempt`, `websearch_provider_success`) +4. session summaries (`mcp_session_summary`, and headless `headless_websearch_summary` when applicable) + +Queries are fingerprinted (`queryHash`, `queryLength`) instead of logged raw by default. For headless/delegation runs, `headless_websearch_summary.likelyBypassed=true` means the MCP tool was exposed, no WebSearch call occurred, and Claude fell back to `Bash` or `WebFetch`. + ### WebSearch returns no results 1. Check `websearch.enabled: true` 2. Keep DuckDuckGo enabled unless you have a strong reason to disable it 3. If using Exa, Tavily, or Brave, verify the matching API key -4. Run with `CCS_DEBUG=1` for hook logs +4. Run with `CCS_DEBUG=1` for runtime logs, or `CCS_WEBSEARCH_TRACE=1` for correlated launch/MCP/provider traces ## Security Considerations diff --git a/lib/hooks/websearch-transformer.cjs b/lib/hooks/websearch-transformer.cjs index 95311502..1165bc36 100644 --- a/lib/hooks/websearch-transformer.cjs +++ b/lib/hooks/websearch-transformer.cjs @@ -15,6 +15,10 @@ */ const { spawnSync } = require('child_process'); +const { createHash } = require('crypto'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); const isWindows = process.platform === 'win32'; const DEFAULT_TIMEOUT_SEC = 55; @@ -26,6 +30,13 @@ const DDG_URL = 'https://html.duckduckgo.com/html/'; const BRAVE_URL = 'https://api.search.brave.com/res/v1/web/search'; const USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'; +const PROVIDER_STATE_FILE = 'websearch-provider-state.json'; +const SHORT_RETRY_AFTER_MAX_SEC = 3; +const TRANSIENT_RETRY_DELAY_MS = 750; +const TRANSIENT_RETRY_ATTEMPTS = 1; +const DEFAULT_RATE_LIMIT_COOLDOWN_SEC = 120; +const DEFAULT_QUOTA_COOLDOWN_SEC = 900; +const MAX_PROVIDER_COOLDOWN_SEC = 60 * 60; const SHARED_INSTRUCTIONS = `Instructions: 1. Search the web for current, up-to-date information @@ -64,12 +75,246 @@ function debug(message) { } } -function shouldSkipHook() { - if (process.env.CCS_WEBSEARCH_SKIP === '1') return true; +function getCcsDirPath() { + if ((process.env.CCS_DIR || '').trim()) { + return path.resolve(process.env.CCS_DIR.trim()); + } + + if ((process.env.CCS_HOME || '').trim()) { + return path.join(path.resolve(process.env.CCS_HOME.trim()), '.ccs'); + } + + const home = (process.env.HOME || process.env.USERPROFILE || '').trim(); + if (home) { + return path.join(home, '.ccs'); + } + + return path.join(process.cwd(), '.ccs'); +} + +function isTraceEnabled() { + return process.env.CCS_WEBSEARCH_TRACE === '1' || process.env.CCS_DEBUG === '1'; +} + +function normalizeSafePrefix(inputPath) { + return `${path.resolve(inputPath)}${path.sep}`; +} + +function getSafeTracePrefixes() { + return [ + normalizeSafePrefix(path.join(getCcsDirPath(), 'logs')), + normalizeSafePrefix(os.tmpdir()), + normalizeSafePrefix('/var/log'), + ]; +} + +function getProviderStatePath() { + return path.join(getCcsDirPath(), 'cache', PROVIDER_STATE_FILE); +} + +function readProviderState() { + try { + const statePath = getProviderStatePath(); + if (!fs.existsSync(statePath)) { + return { cooldowns: {} }; + } + + const parsed = JSON.parse(fs.readFileSync(statePath, 'utf8')); + const cooldowns = + parsed && typeof parsed === 'object' && parsed.cooldowns && typeof parsed.cooldowns === 'object' + ? parsed.cooldowns + : {}; + return { cooldowns }; + } catch { + return { cooldowns: {} }; + } +} + +function writeProviderState(state) { + try { + const statePath = getProviderStatePath(); + fs.mkdirSync(path.dirname(statePath), { recursive: true }); + const tempPath = `${statePath}.${process.pid}.${Date.now()}.tmp`; + fs.writeFileSync(tempPath, JSON.stringify(state, null, 2) + '\n', 'utf8'); + fs.renameSync(tempPath, statePath); + } catch { + // Best-effort only. + } +} + +function sanitizeProviderState(state) { + const now = Date.now(); + const nextCooldowns = {}; + let changed = false; + + for (const [providerId, entry] of Object.entries(state.cooldowns || {})) { + if (!entry || typeof entry !== 'object') { + changed = true; + continue; + } + + const until = Number.parseInt(String(entry.until || ''), 10); + if (!Number.isFinite(until) || until <= now) { + changed = true; + continue; + } + + nextCooldowns[providerId] = { + until, + reason: typeof entry.reason === 'string' ? entry.reason : 'rate_limited', + updatedAt: Number.parseInt(String(entry.updatedAt || ''), 10) || now, + sourceError: typeof entry.sourceError === 'string' ? entry.sourceError : '', + }; + } + + return { + state: { cooldowns: nextCooldowns }, + changed, + }; +} + +function getProviderCooldown(providerId) { + const { state, changed } = sanitizeProviderState(readProviderState()); + if (changed) { + writeProviderState(state); + } + + return state.cooldowns[providerId] || null; +} + +function clearProviderCooldown(providerId) { + const { state } = sanitizeProviderState(readProviderState()); + if (!(providerId in state.cooldowns)) { + return; + } + + delete state.cooldowns[providerId]; + writeProviderState(state); +} + +function applyProviderCooldown(providerId, cooldownSec, reason, sourceError) { + const clampedCooldownSec = Math.max( + 1, + Math.min(MAX_PROVIDER_COOLDOWN_SEC, Math.floor(cooldownSec)) + ); + const { state } = sanitizeProviderState(readProviderState()); + const until = Date.now() + clampedCooldownSec * 1000; + state.cooldowns[providerId] = { + until, + reason, + updatedAt: Date.now(), + sourceError: sourceError || '', + }; + writeProviderState(state); + return until; +} + +function sleep(ms) { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +function getAllowedTraceFileOverride() { + const configured = (process.env.CCS_WEBSEARCH_TRACE_FILE || '').trim(); + if (!configured) { + return null; + } + + const resolved = path.resolve(configured); + if (getSafeTracePrefixes().some((prefix) => resolved.startsWith(prefix))) { + return resolved; + } + + return null; +} + +function getTraceFilePath() { + const fallback = path.join(getCcsDirPath(), 'logs', 'websearch-trace.jsonl'); + return getAllowedTraceFileOverride() || fallback; +} + +function traceWebSearchEvent(event, payload = {}) { + if (!isTraceEnabled()) { + return; + } + + try { + const traceFilePath = getTraceFilePath(); + fs.mkdirSync(path.dirname(traceFilePath), { recursive: true }); + fs.appendFileSync( + traceFilePath, + JSON.stringify({ + at: new Date().toISOString(), + event, + launchId: process.env.CCS_WEBSEARCH_TRACE_LAUNCH_ID || null, + launcher: process.env.CCS_WEBSEARCH_TRACE_LAUNCHER || null, + profileType: process.env.CCS_PROFILE_TYPE || null, + pid: process.pid, + ...payload, + }) + '\n', + 'utf8' + ); + } catch { + // Best-effort only. + } +} + +function readHeaderValue(headers, headerName) { + if (!headers) { + return ''; + } + + if (typeof headers.get === 'function') { + return headers.get(headerName) || ''; + } + + const direct = headers[headerName] ?? headers[String(headerName).toLowerCase()]; + if (Array.isArray(direct)) { + return direct[0] || ''; + } + return typeof direct === 'string' ? direct : ''; +} + +function parseRetryAfterSeconds(rawValue) { + const value = String(rawValue || '').trim(); + if (!value) { + return null; + } + + const asSeconds = Number.parseInt(value, 10); + if (Number.isFinite(asSeconds) && asSeconds > 0) { + return asSeconds; + } + + const asDate = Date.parse(value); + if (Number.isFinite(asDate)) { + const deltaSec = Math.ceil((asDate - Date.now()) / 1000); + return deltaSec > 0 ? deltaSec : null; + } + + return null; +} + +function getQueryFingerprint(query) { + const normalizedQuery = typeof query === 'string' ? query.trim() : ''; + return { + queryHash: normalizedQuery + ? createHash('sha256').update(normalizedQuery).digest('hex').slice(0, 16) + : null, + queryLength: normalizedQuery.length, + }; +} + +function getSkipReason() { + if (process.env.CCS_WEBSEARCH_SKIP === '1') return 'skip_flag'; const profileType = process.env.CCS_PROFILE_TYPE; - if (profileType === 'account' || profileType === 'default') return true; - if (process.env.CCS_WEBSEARCH_ENABLED === '0') return true; - return false; + if (profileType === 'account') return 'native_account_profile'; + if (profileType === 'default') return 'native_default_profile'; + if (process.env.CCS_WEBSEARCH_ENABLED === '0') return 'disabled'; + return null; +} + +function shouldSkipHook() { + return getSkipReason() !== null; } function isCliAvailable(cmd) { @@ -274,6 +519,8 @@ async function tryBraveSearch(query, timeoutSec = DEFAULT_TIMEOUT_SEC) { return { success: false, error: `Brave Search returned ${response.status}: ${body.slice(0, 160)}`, + statusCode: response.status, + retryAfterSec: parseRetryAfterSeconds(readHeaderValue(response.headers, 'retry-after')), }; } @@ -325,7 +572,12 @@ async function tryExaSearch(query, timeoutSec = DEFAULT_TIMEOUT_SEC) { if (!response.ok) { const body = await response.text(); - return { success: false, error: `Exa returned ${response.status}: ${body.slice(0, 160)}` }; + return { + success: false, + error: `Exa returned ${response.status}: ${body.slice(0, 160)}`, + statusCode: response.status, + retryAfterSec: parseRetryAfterSeconds(readHeaderValue(response.headers, 'retry-after')), + }; } const body = await response.json(); @@ -377,7 +629,12 @@ async function tryTavilySearch(query, timeoutSec = DEFAULT_TIMEOUT_SEC) { if (!response.ok) { const body = await response.text(); - return { success: false, error: `Tavily returned ${response.status}: ${body.slice(0, 160)}` }; + return { + success: false, + error: `Tavily returned ${response.status}: ${body.slice(0, 160)}`, + statusCode: response.status, + retryAfterSec: parseRetryAfterSeconds(readHeaderValue(response.headers, 'retry-after')), + }; } const body = await response.json(); @@ -414,7 +671,12 @@ async function tryDuckDuckGoSearch(query, timeoutSec = DEFAULT_TIMEOUT_SEC) { ); if (!response.ok) { - return { success: false, error: `DuckDuckGo returned ${response.status}` }; + return { + success: false, + error: `DuckDuckGo returned ${response.status}`, + statusCode: response.status, + retryAfterSec: parseRetryAfterSeconds(readHeaderValue(response.headers, 'retry-after')), + }; } const html = await response.text(); @@ -576,9 +838,318 @@ function outputAllFailedMessage(query, errors) { emitHookOutput(buildFailureHookOutput(query, errors)); } +function getConfiguredProviders() { + return [ + { + name: 'Exa', + id: 'exa', + available: () => isProviderEnabled('exa') && Boolean(getProviderApiKey('exa')), + fn: tryExaSearch, + }, + { + name: 'Tavily', + id: 'tavily', + available: () => isProviderEnabled('tavily') && Boolean(getProviderApiKey('tavily')), + fn: tryTavilySearch, + }, + { + name: 'Brave Search', + id: 'brave', + available: () => isProviderEnabled('brave') && Boolean(getProviderApiKey('brave')), + fn: tryBraveSearch, + }, + { + name: 'DuckDuckGo', + id: 'duckduckgo', + available: () => isProviderEnabled('duckduckgo'), + fn: tryDuckDuckGoSearch, + }, + { + name: 'Gemini CLI', + id: 'gemini', + available: () => isProviderEnabled('gemini') && isCliAvailable('gemini'), + fn: tryGeminiSearch, + }, + { + name: 'OpenCode', + id: 'opencode', + available: () => isProviderEnabled('opencode') && isCliAvailable('opencode'), + fn: tryOpenCodeSearch, + }, + { + name: 'Grok CLI', + id: 'grok', + available: () => isProviderEnabled('grok') && isCliAvailable('grok'), + fn: tryGrokSearch, + }, + ]; +} + +function looksLikeQuotaExhaustion(errorMessage) { + const lower = String(errorMessage || '').toLowerCase(); + return ( + (lower.includes('quota') && + (lower.includes('exceed') || + lower.includes('exhaust') || + lower.includes('deplet') || + lower.includes('limit') || + lower.includes('used up'))) || + lower.includes('insufficient credits') || + lower.includes('credit balance') || + lower.includes('out of credits') || + lower.includes('billing hard limit') || + lower.includes('monthly usage cap') + ); +} + +function looksLikeTransientFailure(errorMessage) { + const lower = String(errorMessage || '').toLowerCase(); + return ( + lower.includes('timed out') || + lower.includes('timeout') || + lower.includes('temporarily unavailable') || + lower.includes('service unavailable') || + lower.includes('bad gateway') || + lower.includes('gateway timeout') || + lower.includes('socket hang up') || + lower.includes('econnreset') || + lower.includes('fetch failed') || + lower.includes('network') + ); +} + +function classifyProviderFailure(result) { + const errorMessage = String(result.error || ''); + const statusCode = + Number.isFinite(result.statusCode) && result.statusCode > 0 ? result.statusCode : null; + const retryAfterSec = Number.isFinite(result.retryAfterSec) ? result.retryAfterSec : null; + + if (looksLikeQuotaExhaustion(errorMessage)) { + return { + kind: 'cooldown', + reason: 'quota_exhausted', + cooldownSec: retryAfterSec || DEFAULT_QUOTA_COOLDOWN_SEC, + retryAfterSec, + }; + } + + if (statusCode === 429 || /too many requests|rate limit/i.test(errorMessage)) { + if (retryAfterSec && retryAfterSec <= SHORT_RETRY_AFTER_MAX_SEC) { + return { + kind: 'retry', + delayMs: retryAfterSec * 1000, + reason: 'rate_limited_short_backoff', + retryAfterSec, + }; + } + + return { + kind: 'cooldown', + reason: 'rate_limited', + cooldownSec: retryAfterSec || DEFAULT_RATE_LIMIT_COOLDOWN_SEC, + retryAfterSec, + }; + } + + if ( + (statusCode && [502, 503, 504].includes(statusCode)) || + looksLikeTransientFailure(errorMessage) + ) { + return { + kind: 'retry', + delayMs: TRANSIENT_RETRY_DELAY_MS, + reason: 'transient_failure', + retryAfterSec, + }; + } + + return { + kind: 'fail', + reason: 'non_retryable', + retryAfterSec, + }; +} + +async function runProviderWithPolicy(provider, query, timeoutSec, fingerprint) { + for (let attempt = 0; attempt <= TRANSIENT_RETRY_ATTEMPTS; attempt += 1) { + traceWebSearchEvent('websearch_provider_attempt', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + attempt: attempt + 1, + ...fingerprint, + }); + + const result = await provider.fn(query, timeoutSec); + if (result.success) { + clearProviderCooldown(provider.id); + return result; + } + + const policy = classifyProviderFailure(result); + if (policy.kind === 'retry' && attempt < TRANSIENT_RETRY_ATTEMPTS) { + traceWebSearchEvent('websearch_provider_retry_scheduled', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + attempt: attempt + 1, + delayMs: policy.delayMs, + reason: policy.reason, + retryAfterSec: policy.retryAfterSec, + ...fingerprint, + }); + await sleep(policy.delayMs); + continue; + } + + if (policy.kind === 'retry' && policy.reason === 'rate_limited_short_backoff') { + const cooldownSec = policy.retryAfterSec || DEFAULT_RATE_LIMIT_COOLDOWN_SEC; + const until = applyProviderCooldown(provider.id, cooldownSec, 'rate_limited', result.error); + traceWebSearchEvent('websearch_provider_cooldown_applied', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + cooldownUntil: until, + cooldownSec, + reason: 'rate_limited', + retryAfterSec: policy.retryAfterSec, + afterRetryExhausted: true, + ...fingerprint, + }); + return { + ...result, + error: `${result.error} (cooldown ${cooldownSec}s)`, + }; + } + + if (policy.kind === 'cooldown') { + const until = applyProviderCooldown( + provider.id, + policy.cooldownSec, + policy.reason, + result.error + ); + traceWebSearchEvent('websearch_provider_cooldown_applied', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + cooldownUntil: until, + cooldownSec: policy.cooldownSec, + reason: policy.reason, + retryAfterSec: policy.retryAfterSec, + ...fingerprint, + }); + return { + ...result, + error: `${result.error} (cooldown ${policy.cooldownSec}s)`, + }; + } + + return result; + } + + return { success: false, error: 'Provider retry policy exhausted' }; +} + +function getActiveProviders() { + return getConfiguredProviders().filter((provider) => !getProviderCooldown(provider.id) && provider.available()); +} + +function getActiveProviderIds() { + return getActiveProviders().map((provider) => provider.id); +} + +function hasAnyActiveProviders() { + return getActiveProviders().length > 0; +} + +async function runLocalWebSearch(query, timeoutSec = DEFAULT_TIMEOUT_SEC) { + const fingerprint = getQueryFingerprint(query); + const configuredProviders = getConfiguredProviders(); + const activeProviders = []; + + for (const provider of configuredProviders) { + const cooldown = getProviderCooldown(provider.id); + if (cooldown) { + traceWebSearchEvent('websearch_provider_cooldown_skip', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + cooldownUntil: cooldown.until, + cooldownReason: cooldown.reason, + remainingMs: Math.max(0, cooldown.until - Date.now()), + ...fingerprint, + }); + continue; + } + + if (provider.available()) { + activeProviders.push(provider); + } + } + + debug( + `Enabled providers: ${activeProviders.map((provider) => provider.name).join(', ') || 'none'}` + ); + traceWebSearchEvent('websearch_provider_run_started', { + source: 'provider', + activeProviderIds: activeProviders.map((provider) => provider.id), + ...fingerprint, + }); + + if (activeProviders.length === 0) { + traceWebSearchEvent('websearch_provider_run_unavailable', { + source: 'provider', + activeProviderIds: [], + ...fingerprint, + }); + return { success: false, noActiveProviders: true, errors: [] }; + } + + const errors = []; + for (const provider of activeProviders) { + debug(`Trying ${provider.name}`); + const result = await runProviderWithPolicy(provider, query, timeoutSec, fingerprint); + if (result.success) { + traceWebSearchEvent('websearch_provider_success', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + ...fingerprint, + }); + return { + success: true, + providerId: provider.id, + providerName: provider.name, + content: result.content, + }; + } + traceWebSearchEvent('websearch_provider_failure', { + source: 'provider', + providerId: provider.id, + providerName: provider.name, + error: result.error, + ...fingerprint, + }); + errors.push({ provider: provider.name, error: result.error }); + } + + traceWebSearchEvent('websearch_provider_run_failed', { + source: 'provider', + errorCount: errors.length, + activeProviderIds: activeProviders.map((provider) => provider.id), + ...fingerprint, + }); + return { success: false, noActiveProviders: false, errors }; +} + async function processHook(input) { try { if (shouldSkipHook()) { + traceWebSearchEvent('websearch_hook_skipped', { + source: 'hook', + reason: getSkipReason(), + }); process.exit(0); } @@ -592,78 +1163,47 @@ async function processHook(input) { process.exit(0); } + traceWebSearchEvent('websearch_hook_invoked', { + source: 'hook', + ...getQueryFingerprint(query), + }); + const timeout = Number.parseInt( process.env.CCS_WEBSEARCH_TIMEOUT || `${DEFAULT_TIMEOUT_SEC}`, 10 ); - const providers = [ - { - name: 'Exa', - id: 'exa', - available: () => isProviderEnabled('exa') && Boolean(getProviderApiKey('exa')), - fn: tryExaSearch, - }, - { - name: 'Tavily', - id: 'tavily', - available: () => isProviderEnabled('tavily') && Boolean(getProviderApiKey('tavily')), - fn: tryTavilySearch, - }, - { - name: 'Brave Search', - id: 'brave', - available: () => isProviderEnabled('brave') && Boolean(getProviderApiKey('brave')), - fn: tryBraveSearch, - }, - { - name: 'DuckDuckGo', - id: 'duckduckgo', - available: () => isProviderEnabled('duckduckgo'), - fn: tryDuckDuckGoSearch, - }, - { - name: 'Gemini CLI', - id: 'gemini', - available: () => isProviderEnabled('gemini') && isCliAvailable('gemini'), - fn: tryGeminiSearch, - }, - { - name: 'OpenCode', - id: 'opencode', - available: () => isProviderEnabled('opencode') && isCliAvailable('opencode'), - fn: tryOpenCodeSearch, - }, - { - name: 'Grok CLI', - id: 'grok', - available: () => isProviderEnabled('grok') && isCliAvailable('grok'), - fn: tryGrokSearch, - }, - ]; - - const activeProviders = providers.filter((provider) => provider.available()); - debug( - `Enabled providers: ${activeProviders.map((provider) => provider.name).join(', ') || 'none'}` - ); - - if (activeProviders.length === 0) { + const result = await runLocalWebSearch(query, timeout); + if (result.noActiveProviders) { + traceWebSearchEvent('websearch_hook_no_active_providers', { + source: 'hook', + ...getQueryFingerprint(query), + }); process.exit(0); } - const errors = []; - for (const provider of activeProviders) { - debug(`Trying ${provider.name}`); - const result = await provider.fn(query, timeout); - if (result.success) { - outputSuccess(query, result.content, provider.name); - return; - } - errors.push({ provider: provider.name, error: result.error }); + if (result.success) { + traceWebSearchEvent('websearch_hook_success', { + source: 'hook', + providerId: result.providerId, + providerName: result.providerName, + ...getQueryFingerprint(query), + }); + outputSuccess(query, result.content, result.providerName); + return; } - outputAllFailedMessage(query, errors); + traceWebSearchEvent('websearch_hook_failure', { + source: 'hook', + errorCount: result.errors.length, + ...getQueryFingerprint(query), + }); + outputAllFailedMessage(query, result.errors); } catch (error) { debug(`Hook error: ${error.message}`); + traceWebSearchEvent('websearch_hook_error', { + source: 'hook', + error: error.message, + }); process.exit(0); } } @@ -691,6 +1231,16 @@ module.exports = { buildSuccessHookOutput, extractDuckDuckGoResults, formatStructuredSearchResults, + getActiveProviders, + hasAnyActiveProviders, + runLocalWebSearch, + shouldSkipHook, + getActiveProviderIds, + classifyProviderFailure, + getQueryFingerprint, + getSkipReason, + parseRetryAfterSeconds, + traceWebSearchEvent, tryExaSearch, tryTavilySearch, tryDuckDuckGoSearch, diff --git a/lib/mcp/ccs-websearch-server.cjs b/lib/mcp/ccs-websearch-server.cjs new file mode 100644 index 00000000..13880600 --- /dev/null +++ b/lib/mcp/ccs-websearch-server.cjs @@ -0,0 +1,339 @@ +#!/usr/bin/env node + +const { + getActiveProviderIds, + getQueryFingerprint, + getSkipReason, + hasAnyActiveProviders, + runLocalWebSearch, + shouldSkipHook, + traceWebSearchEvent, +} = require('../hooks/websearch-transformer.cjs'); + +const PROTOCOL_VERSION = '2024-11-05'; +const SERVER_NAME = 'ccs-websearch'; +const SERVER_VERSION = '1.0.0'; +const TOOL_NAME = 'WebSearch'; +const TOOL_ALIASES = ['search']; +const TOOL_DESCRIPTION = + 'Third-party WebSearch replacement for CCS-managed Claude launches. Use this instead of Bash/curl/http fetches for web lookups. Provider order: Exa, Tavily, Brave Search, DuckDuckGo, then optional legacy CLI fallback.'; + +function isSupportedToolName(name) { + return name === TOOL_NAME || TOOL_ALIASES.includes(name); +} + +let inputBuffer = Buffer.alloc(0); +const sessionState = { + initializeCount: 0, + toolsListCount: 0, + exposed: false, + toolCalls: 0, +}; +let sessionSummaryWritten = false; + +function shouldExposeTools() { + return !shouldSkipHook() && hasAnyActiveProviders(); +} + +function getTools() { + if (!shouldExposeTools()) { + return []; + } + + return [ + { + name: TOOL_NAME, + description: TOOL_DESCRIPTION, + inputSchema: { + type: 'object', + properties: { + query: { + type: 'string', + description: + 'Web query to resolve through CCS providers. Prefer this tool over ad hoc Bash/curl lookups when you need current web information.', + }, + }, + required: ['query'], + additionalProperties: false, + }, + }, + ]; +} + +function writeMessage(message) { + process.stdout.write(`${JSON.stringify(message)}\n`); +} + +function writeResponse(id, result) { + writeMessage({ + jsonrpc: '2.0', + id, + result, + }); +} + +function writeError(id, code, message) { + writeMessage({ + jsonrpc: '2.0', + id, + error: { + code, + message, + }, + }); +} + +async function handleToolCall(message) { + const id = message.id; + const params = message.params || {}; + const toolArgs = params.arguments || {}; + const toolName = params.name || ''; + const query = typeof toolArgs.query === 'string' ? toolArgs.query.trim() : ''; + const fingerprint = getQueryFingerprint(query); + + if (!isSupportedToolName(toolName)) { + traceWebSearchEvent('mcp_tool_call_rejected', { + source: 'mcp', + reason: 'unknown_tool', + toolName, + }); + writeError(id, -32602, `Unknown tool: ${toolName}`); + return; + } + + sessionState.toolCalls += 1; + traceWebSearchEvent('mcp_tool_call_received', { + source: 'mcp', + toolName, + ...fingerprint, + }); + + if (!shouldExposeTools()) { + traceWebSearchEvent('mcp_tool_call_unavailable', { + source: 'mcp', + toolName, + exposed: false, + skipReason: getSkipReason(), + activeProviderIds: getActiveProviderIds(), + ...fingerprint, + }); + writeResponse(id, { + content: [ + { + type: 'text', + text: 'CCS WebSearch is unavailable for this profile or no providers are ready.', + }, + ], + isError: true, + }); + return; + } + + if (!query) { + traceWebSearchEvent('mcp_tool_call_rejected', { + source: 'mcp', + reason: 'empty_query', + toolName, + }); + writeError(id, -32602, `Tool "${TOOL_NAME}" requires a non-empty string query.`); + return; + } + + const result = await runLocalWebSearch(query); + if (result.success) { + traceWebSearchEvent('mcp_tool_call_result', { + source: 'mcp', + toolName, + success: true, + providerId: result.providerId, + providerName: result.providerName, + ...fingerprint, + }); + writeResponse(id, { + content: [{ type: 'text', text: result.content }], + }); + return; + } + + traceWebSearchEvent('mcp_tool_call_result', { + source: 'mcp', + toolName, + success: false, + noActiveProviders: Boolean(result.noActiveProviders), + errorCount: result.errors.length, + ...fingerprint, + }); + + const errorDetail = + result.noActiveProviders || result.errors.length === 0 + ? 'No active WebSearch providers are ready.' + : result.errors.map((entry) => `${entry.provider}: ${entry.error}`).join(' | '); + + writeResponse(id, { + content: [ + { + type: 'text', + text: `CCS local WebSearch failed for "${query}". ${errorDetail}`, + }, + ], + isError: true, + }); +} + +async function handleMessage(message) { + if (!message || message.jsonrpc !== '2.0' || typeof message.method !== 'string') { + return; + } + + switch (message.method) { + case 'initialize': + sessionState.initializeCount += 1; + sessionState.exposed = sessionState.exposed || shouldExposeTools(); + traceWebSearchEvent('mcp_initialize', { + source: 'mcp', + exposed: shouldExposeTools(), + skipReason: getSkipReason(), + activeProviderIds: getActiveProviderIds(), + }); + writeResponse(message.id, { + protocolVersion: PROTOCOL_VERSION, + capabilities: { + tools: {}, + }, + serverInfo: { + name: SERVER_NAME, + version: SERVER_VERSION, + }, + }); + return; + case 'notifications/initialized': + return; + case 'ping': + writeResponse(message.id, {}); + return; + case 'tools/list': + sessionState.toolsListCount += 1; + { + const tools = getTools(); + const exposed = tools.length > 0; + sessionState.exposed = sessionState.exposed || exposed; + traceWebSearchEvent('mcp_tools_list', { + source: 'mcp', + exposed, + toolNames: tools.map((tool) => tool.name), + activeProviderIds: getActiveProviderIds(), + skipReason: getSkipReason(), + }); + writeResponse(message.id, { tools }); + } + return; + case 'tools/call': + await handleToolCall(message); + return; + default: + if (message.id !== undefined) { + writeError(message.id, -32601, `Method not found: ${message.method}`); + } + } +} + +function writeSessionSummary(exitCodeOrSignal) { + if (sessionSummaryWritten) { + return; + } + + sessionSummaryWritten = true; + traceWebSearchEvent('mcp_session_summary', { + source: 'mcp', + initializeCount: sessionState.initializeCount, + toolsListCount: sessionState.toolsListCount, + exposed: sessionState.exposed, + toolCalls: sessionState.toolCalls, + calledWebSearch: sessionState.toolCalls > 0, + likelyBypassed: sessionState.exposed && sessionState.toolCalls === 0 ? 'unknown' : false, + activeProviderIds: getActiveProviderIds(), + skipReason: getSkipReason(), + exitCode: typeof exitCodeOrSignal === 'number' ? exitCodeOrSignal : null, + exitSignal: typeof exitCodeOrSignal === 'string' ? exitCodeOrSignal : null, + }); +} + +function parseMessages() { + while (true) { + let body; + const startsWithLegacyHeaders = inputBuffer + .slice(0, Math.min(inputBuffer.length, 32)) + .toString('utf8') + .toLowerCase() + .startsWith('content-length:'); + + if (startsWithLegacyHeaders) { + const headerEnd = inputBuffer.indexOf('\r\n\r\n'); + if (headerEnd === -1) { + return; + } + + const headerText = inputBuffer.slice(0, headerEnd).toString('utf8'); + const contentLengthMatch = headerText.match(/content-length:\s*(\d+)/i); + if (!contentLengthMatch) { + inputBuffer = Buffer.alloc(0); + return; + } + + const contentLength = Number.parseInt(contentLengthMatch[1], 10); + const messageEnd = headerEnd + 4 + contentLength; + if (inputBuffer.length < messageEnd) { + return; + } + + body = inputBuffer.slice(headerEnd + 4, messageEnd).toString('utf8'); + inputBuffer = inputBuffer.slice(messageEnd); + } else { + const newlineIndex = inputBuffer.indexOf('\n'); + if (newlineIndex === -1) { + return; + } + + body = inputBuffer.slice(0, newlineIndex).toString('utf8').replace(/\r$/, '').trim(); + inputBuffer = inputBuffer.slice(newlineIndex + 1); + if (!body) { + continue; + } + } + + let message; + try { + message = JSON.parse(body); + } catch { + continue; + } + + Promise.resolve(handleMessage(message)).catch((error) => { + if (message && message.id !== undefined) { + writeError(message.id, -32603, (error && error.message) || 'Internal error'); + } + }); + } +} + +process.stdin.on('data', (chunk) => { + inputBuffer = Buffer.concat([inputBuffer, chunk]); + parseMessages(); +}); + +process.stdin.on('error', () => { + process.exit(0); +}); + +process.on('exit', (code) => { + writeSessionSummary(code); +}); + +['SIGINT', 'SIGTERM', 'SIGHUP'].forEach((signal) => { + process.on(signal, () => { + writeSessionSummary(signal); + process.exit(0); + }); +}); + +process.stdin.resume(); diff --git a/src/api/services/profile-lifecycle-service.ts b/src/api/services/profile-lifecycle-service.ts index e54a3e92..50f3b86b 100644 --- a/src/api/services/profile-lifecycle-service.ts +++ b/src/api/services/profile-lifecycle-service.ts @@ -10,7 +10,7 @@ import type { Config, Settings } from '../../types'; import type { TargetType } from '../../targets/target-adapter'; import { getPersistedTargetChoices, isPersistedTargetType } from '../../targets/target-metadata'; import { getCcsDir, getConfigPath, loadConfigSafe } from '../../utils/config-manager'; -import { ensureProfileHooksOrThrow } from '../../utils/websearch/profile-hook-injector'; +import { ensureWebSearchMcpOrThrow } from '../../utils/websearch-manager'; import { isSensitiveKey } from '../../utils/sensitive-keys'; import { isReservedName } from '../../config/reserved-names'; import { isUnifiedMode, mutateUnifiedConfig } from '../../config/unified-config-loader'; @@ -218,7 +218,7 @@ export function registerApiProfileOrphans(options?: { try { if (orphan.validation.valid) { - ensureProfileHooksOrThrow(orphan.name); + ensureWebSearchMcpOrThrow(); } registerApiProfileInConfig(orphan.name, options?.target || 'claude', options?.force || false); result.registered.push(orphan.name); @@ -268,7 +268,7 @@ export function copyApiProfile( writeJsonObjectAtomically(destinationSettingsPath, sourceSettings); try { - ensureProfileHooksOrThrow(destination); + ensureWebSearchMcpOrThrow(); } catch (hookError) { rollbackSettingsFile(destinationSettingsPath, previousDestinationContent, destinationExisted); throw hookError; @@ -393,7 +393,7 @@ export function importApiProfileBundle( writeJsonObjectAtomically(settingsPath, settings); try { - ensureProfileHooksOrThrow(name); + ensureWebSearchMcpOrThrow(); } catch (hookError) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw hookError; diff --git a/src/api/services/profile-writer.ts b/src/api/services/profile-writer.ts index 912dc5b3..efb70c32 100644 --- a/src/api/services/profile-writer.ts +++ b/src/api/services/profile-writer.ts @@ -8,7 +8,7 @@ import { getCcsDir, getConfigPath, loadConfigSafe } from '../../utils/config-man import { expandPath } from '../../utils/helpers'; import { validateApiName } from './validation-service'; import { mutateUnifiedConfig, isUnifiedMode } from '../../config/unified-config-loader'; -import { ensureProfileHooksOrThrow } from '../../utils/websearch/profile-hook-injector'; +import { ensureWebSearchMcpOrThrow } from '../../utils/websearch-manager'; import type { TargetType } from '../../targets/target-adapter'; import { resolveDroidProvider } from '../../targets/droid-provider'; import { isReservedName } from '../../config/reserved-names'; @@ -126,8 +126,7 @@ function createSettingsFile( fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2) + '\n', 'utf8'); try { - // Inject WebSearch hooks into profile settings - ensureProfileHooksOrThrow(name); + ensureWebSearchMcpOrThrow(); } catch (error) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw error; @@ -216,8 +215,7 @@ function createApiProfileUnified( fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2) + '\n', 'utf8'); try { - // Inject WebSearch hooks into profile settings - ensureProfileHooksOrThrow(name); + ensureWebSearchMcpOrThrow(); } catch (error) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw error; diff --git a/src/ccs.ts b/src/ccs.ts index ca2625fb..d427d234 100644 --- a/src/ccs.ts +++ b/src/ccs.ts @@ -24,10 +24,12 @@ import { import { getEffectiveEnvVars, getCompositeEnvVars } from './cliproxy/config/env-builder'; import { CLIPROXY_DEFAULT_PORT } from './cliproxy/config/port-manager'; import { - ensureMcpWebSearch, + ensureWebSearchMcpOrThrow, displayWebSearchStatus, getWebSearchHookEnv, - ensureProfileHooksOrThrow, + syncWebSearchMcpToConfigDir, + appendThirdPartyWebSearchToolArgs, + createWebSearchTraceContext, } from './utils/websearch-manager'; import { getGlobalEnvConfig, getOfficialChannelsConfig } from './config/unified-config-loader'; import { ensureProfileHooks as ensureImageAnalyzerHooks } from './utils/hooks/image-analyzer-profile-hook-injector'; @@ -677,8 +679,9 @@ async function main(): Promise { if (profileInfo.type === 'cliproxy') { // CLIPROXY FLOW: OAuth-based profiles (gemini, codex, agy, qwen) or user-defined variants - // Inject WebSearch hook into profile settings before launch - ensureProfileHooksOrThrow(profileInfo.name); + if (resolvedTarget === 'claude') { + ensureWebSearchMcpOrThrow(); + } // Inject Image Analyzer hook into profile settings before launch ensureImageAnalyzerHooks(profileInfo.name); @@ -835,8 +838,7 @@ async function main(): Promise { }); } else if (profileInfo.type === 'copilot') { // COPILOT FLOW: GitHub Copilot subscription via copilot-api proxy - // Inject WebSearch hook into profile settings before launch - ensureProfileHooksOrThrow(profileInfo.name); + ensureWebSearchMcpOrThrow(); // Inject Image Analyzer hook into profile settings before launch ensureImageAnalyzerHooks(profileInfo.name); @@ -867,14 +869,12 @@ async function main(): Promise { process.exit(exitCode); } else if (profileInfo.type === 'settings') { // Settings-based profiles (glm, glmt) are third-party providers - // WebSearch is server-side tool - third-party providers have no access - // Inject WebSearch hook into profile settings before launch - ensureProfileHooksOrThrow(profileInfo.name); + if (resolvedTarget === 'claude') { + ensureWebSearchMcpOrThrow(); + } // Inject Image Analyzer hook into profile settings before launch ensureImageAnalyzerHooks(profileInfo.name); - ensureMcpWebSearch(); - // Display WebSearch status (single line, equilibrium UX) displayWebSearchStatus(); @@ -894,6 +894,7 @@ async function main(): Promise { ); } const inheritedClaudeConfigDir = continuityInheritance.claudeConfigDir; + syncWebSearchMcpToConfigDir(inheritedClaudeConfigDir); const expandedSettingsPath = resolvedSettingsPath ?? (profileInfo.settingsPath @@ -1053,7 +1054,19 @@ async function main(): Promise { return; } - execClaude(claudeCli, ['--settings', expandedSettingsPath, ...remainingArgs], envVars); + const launchArgs = [ + '--settings', + expandedSettingsPath, + ...appendThirdPartyWebSearchToolArgs(remainingArgs), + ]; + const traceEnv = createWebSearchTraceContext({ + launcher: 'ccs.settings-profile', + args: launchArgs, + profile: profileInfo.name, + profileType: profileInfo.type, + settingsPath: expandedSettingsPath, + }); + execClaude(claudeCli, launchArgs, { ...envVars, ...traceEnv }); } else if (profileInfo.type === 'account') { // NEW FLOW: Account-based profile (work, personal) // All platforms: Use instance isolation with CLAUDE_CONFIG_DIR diff --git a/src/cliproxy/executor/index.ts b/src/cliproxy/executor/index.ts index 1715d1bc..6d08788f 100644 --- a/src/cliproxy/executor/index.ts +++ b/src/cliproxy/executor/index.ts @@ -53,9 +53,10 @@ import { } from '../account-manager'; import { formatAccountDisplayName } from '../accounts/email-account-identity'; import { - ensureMcpWebSearch, - installWebSearchHook, + ensureWebSearchMcpOrThrow, displayWebSearchStatus, + appendThirdPartyWebSearchToolArgs, + createWebSearchTraceContext, } from '../../utils/websearch-manager'; import { loadOrCreateUnifiedConfig, getThinkingConfig } from '../../config/unified-config-loader'; import { installImageAnalyzerHook } from '../../utils/hooks'; @@ -197,9 +198,8 @@ export async function execClaudeWithCLIProxy( log(`Remote host: ${proxyConfig.host}:${proxyConfig.port} (${proxyConfig.protocol})`); } - // Setup WebSearch hooks - ensureMcpWebSearch(); - installWebSearchHook(); + // Setup first-class CCS WebSearch runtime + ensureWebSearchMcpOrThrow(); displayWebSearchStatus(); // Sync image analyzer hook from npm package to ~/.ccs/hooks/ @@ -1035,21 +1035,29 @@ export async function execClaudeWithCLIProxy( : getProviderSettingsPath(provider); let claude: ChildProcess; + const launchArgs = ['--settings', settingsPath, ...appendThirdPartyWebSearchToolArgs(claudeArgs)]; + const traceEnv = createWebSearchTraceContext({ + launcher: 'cliproxy.executor', + args: launchArgs, + profile: cfg.profileName || provider, + profileType: 'cliproxy', + settingsPath, + claudeConfigDir: inheritedClaudeConfigDir, + }); + const tracedEnv = { ...env, ...traceEnv }; if (needsShell) { - const cmdString = [claudeCli, '--settings', settingsPath, ...claudeArgs] - .map(escapeShellArg) - .join(' '); + const cmdString = [claudeCli, ...launchArgs].map(escapeShellArg).join(' '); claude = spawn(cmdString, { stdio: 'inherit', windowsHide: true, shell: true, - env, + env: tracedEnv, }); } else { - claude = spawn(claudeCli, ['--settings', settingsPath, ...claudeArgs], { + claude = spawn(claudeCli, launchArgs, { stdio: 'inherit', windowsHide: true, - env, + env: tracedEnv, }); } diff --git a/src/cliproxy/services/variant-settings.ts b/src/cliproxy/services/variant-settings.ts index e55bb866..6e982528 100644 --- a/src/cliproxy/services/variant-settings.ts +++ b/src/cliproxy/services/variant-settings.ts @@ -14,7 +14,7 @@ import { expandPath } from '../../utils/helpers'; import { getClaudeEnvVars, CLIPROXY_DEFAULT_PORT } from '../config-generator'; import { CLIProxyProvider } from '../types'; import { CompositeTierConfig } from '../../config/unified-config-types'; -import { ensureProfileHooksOrThrow } from '../../utils/websearch/profile-hook-injector'; +import { ensureWebSearchMcpOrThrow } from '../../utils/websearch-manager'; import { ensureProfileHooks as ensureImageAnalyzerHooks } from '../../utils/hooks/image-analyzer-profile-hook-injector'; import { getEffectiveApiKey } from '../auth-token-manager'; import { warn } from '../../utils/ui'; @@ -154,8 +154,7 @@ export function createSettingsFile( writeSettings(settingsPath, settings); try { - // Inject WebSearch hooks into variant settings - ensureProfileHooksOrThrow(`${provider}-${name}`); + ensureWebSearchMcpOrThrow(); } catch (error) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw error; @@ -189,8 +188,7 @@ export function createSettingsFileUnified( writeSettings(settingsPath, settings); try { - // Inject WebSearch hooks into variant settings - ensureProfileHooksOrThrow(`${provider}-${name}`); + ensureWebSearchMcpOrThrow(); } catch (error) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw error; @@ -289,7 +287,7 @@ export function createCompositeSettingsFile( // Hook injectors target ~/.ccs/.settings.json; only run for default path. if (path.resolve(settingsPath) === path.resolve(defaultSettingsPath)) { try { - ensureProfileHooksOrThrow(`composite-${name}`); + ensureWebSearchMcpOrThrow(); } catch (error) { rollbackSettingsFile(settingsPath, previousSettingsContent, settingsExisted); throw error; diff --git a/src/commands/install-command.ts b/src/commands/install-command.ts index 076e97fb..fb45f770 100644 --- a/src/commands/install-command.ts +++ b/src/commands/install-command.ts @@ -5,7 +5,7 @@ */ import { info, ok, color, box, initUI } from '../utils/ui'; -import { uninstallWebSearchHook } from '../utils/websearch'; +import { uninstallWebSearchHook, uninstallWebSearchMcp } from '../utils/websearch'; import { ClaudeSymlinkManager } from '../utils/claude-symlink-manager'; /** @@ -42,12 +42,19 @@ export async function handleUninstallCommand(): Promise { removed += 1; // Count as 1 item (the hook file) } - // 2. Remove symlinks from ~/.claude/ + // 2. Remove managed WebSearch MCP runtime/config + const mcpRemoved = uninstallWebSearchMcp(); + if (mcpRemoved) { + console.log(ok('Removed WebSearch MCP runtime')); + removed += 1; + } + + // 3. Remove symlinks from ~/.claude/ const symlinkManager = new ClaudeSymlinkManager(); const symlinksRemoved = symlinkManager.uninstall(); removed += symlinksRemoved; // Add actual count of symlinks removed - // 3. Summary + // 4. Summary console.log(''); if (removed > 0) { console.log(ok('Uninstall complete!')); diff --git a/src/copilot/copilot-executor.ts b/src/copilot/copilot-executor.ts index 8f885cbd..d1487cbf 100644 --- a/src/copilot/copilot-executor.ts +++ b/src/copilot/copilot-executor.ts @@ -14,7 +14,12 @@ import { ensureCopilotApi } from './copilot-package-manager'; import { normalizeCopilotConfigWithWarnings } from './copilot-model-normalizer'; import { CopilotStatus } from './types'; import { fail, info, ok, warn } from '../utils/ui'; -import { getWebSearchHookEnv } from '../utils/websearch-manager'; +import { + getWebSearchHookEnv, + appendThirdPartyWebSearchToolArgs, + createWebSearchTraceContext, + syncWebSearchMcpToConfigDir, +} from '../utils/websearch-manager'; import { getImageAnalysisHookEnv } from '../utils/hooks'; import { stripClaudeCodeEnv } from '../utils/shell-executor'; @@ -173,11 +178,22 @@ export async function executeCopilotProfile( console.log(info(`Using GitHub Copilot proxy (model: ${normalizedConfig.model})`)); console.log(''); + syncWebSearchMcpToConfigDir(claudeConfigDir); + // Spawn Claude CLI return new Promise((resolve) => { - const proc = spawn(claudeCliPath, claudeArgs, { + const launchArgs = appendThirdPartyWebSearchToolArgs(claudeArgs); + const traceEnv = createWebSearchTraceContext({ + launcher: 'copilot.executor', + args: launchArgs, + profile: 'copilot', + profileType: 'copilot', + claudeConfigDir, + }); + + const proc = spawn(claudeCliPath, launchArgs, { stdio: 'inherit', - env, + env: { ...env, ...traceEnv }, shell: process.platform === 'win32', }); diff --git a/src/delegation/executor/result-aggregator.ts b/src/delegation/executor/result-aggregator.ts index c7415ba6..821313cd 100644 --- a/src/delegation/executor/result-aggregator.ts +++ b/src/delegation/executor/result-aggregator.ts @@ -2,9 +2,33 @@ * Result aggregation utilities for headless executor */ -import type { ExecutionResult, StreamMessage } from './types'; +import type { ExecutionResult, StreamMessage, ToolUsageSummary } from './types'; import { warn } from '../../utils/ui'; +const WEBSEARCH_FALLBACK_TOOLS = new Set(['Bash', 'WebFetch']); + +export function summarizeToolUsage(messages: StreamMessage[]): ToolUsageSummary { + const toolNames = new Set(); + + for (const message of messages) { + const content = message.message?.content || []; + for (const entry of content) { + if (entry.type === 'tool_use' && entry.name) { + toolNames.add(entry.name); + } + } + } + + const orderedToolNames = [...toolNames]; + return { + toolNames: orderedToolNames, + calledWebSearch: toolNames.has('WebSearch') || toolNames.has('search'), + fallbackToolsUsed: orderedToolNames.filter((toolName) => + WEBSEARCH_FALLBACK_TOOLS.has(toolName) + ), + }; +} + /** * Build execution result from stream messages * @param params - Parameters for building result @@ -32,6 +56,7 @@ export function buildExecutionResult(params: { timedOut, success: exitCode === 0 && !timedOut, messages, + toolUsageSummary: summarizeToolUsage(messages), }; // Extract metadata from final 'result' message in stream-json diff --git a/src/delegation/executor/types.ts b/src/delegation/executor/types.ts index 0bb0b922..00162015 100644 --- a/src/delegation/executor/types.ts +++ b/src/delegation/executor/types.ts @@ -44,6 +44,12 @@ export interface ExecutionError { [key: string]: unknown; } +export interface ToolUsageSummary { + toolNames: string[]; + calledWebSearch: boolean; + fallbackToolsUsed: string[]; +} + /** * Options for headless execution */ @@ -86,6 +92,7 @@ export interface ExecutionResult { permissionDenials?: PermissionDenial[]; errors?: ExecutionError[]; content?: string; + toolUsageSummary?: ToolUsageSummary; } /** diff --git a/src/delegation/headless-executor.ts b/src/delegation/headless-executor.ts index 85265534..074016b9 100644 --- a/src/delegation/headless-executor.ts +++ b/src/delegation/headless-executor.ts @@ -18,6 +18,16 @@ import { buildExecutionResult } from './executor/result-aggregator'; import { getCcsDir, getModelDisplayName } from '../utils/config-manager'; import { getProfileLookupCandidates } from '../utils/profile-compat'; import { getClaudeLaunchEnvOverrides, stripClaudeCodeEnv } from '../utils/shell-executor'; +import { resolveProfileContinuityInheritance } from '../auth/profile-continuity-inheritance'; +import { + appendThirdPartyWebSearchToolArgs, + appendWebSearchTrace, + createWebSearchTraceContext, + ensureWebSearchMcpOrThrow, + getWebSearchHookEnv, + readWebSearchTraceRecords, + syncWebSearchMcpToConfigDir, +} from '../utils/websearch-manager'; // Re-export types for consumers export type { ExecutionOptions, ExecutionResult, StreamMessage } from './executor/types'; @@ -80,6 +90,23 @@ export class HeadlessExecutor { ); } + const continuityInheritance = await resolveProfileContinuityInheritance({ + profileName: profile, + profileType: 'settings', + target: 'claude', + }); + const inheritedClaudeConfigDir = continuityInheritance.claudeConfigDir; + if (continuityInheritance.sourceAccount && process.env.CCS_DEBUG) { + console.error( + info( + `Continuity inheritance active: profile "${profile}" -> account "${continuityInheritance.sourceAccount}"` + ) + ); + } + + ensureWebSearchMcpOrThrow(); + syncWebSearchMcpToConfigDir(inheritedClaudeConfigDir); + // Smart slash command detection and preservation const processedPrompt = this._processSlashCommand(enhancedPrompt); @@ -125,7 +152,7 @@ export class HeadlessExecutor { args.push('--allowedTools', ...toolRestrictions.allowedTools); } if (toolRestrictions.disallowedTools.length > 0) { - args.push('--disallowedTools', ...toolRestrictions.disallowedTools); + args.push('--disallowedTools', toolRestrictions.disallowedTools.join(',')); } // Claude Code CLI passthrough flags (explicit, validated) @@ -163,21 +190,34 @@ export class HeadlessExecutor { } } + const launchArgs = appendThirdPartyWebSearchToolArgs(args); + const traceEnv = createWebSearchTraceContext({ + launcher: 'delegation.headless-executor', + args: launchArgs, + cwd, + profile, + profileType: 'settings', + settingsPath, + claudeConfigDir: inheritedClaudeConfigDir, + }); + if (process.env.CCS_DEBUG) { - console.error(info(`Claude CLI args: ${args.join(' ')}`)); + console.error(info(`Claude CLI args: ${launchArgs.join(' ')}`)); } // Initialize UI before spawning await ui.init(); // Execute with spawn - return this._spawnAndExecute(claudeCli, args, { + return this._spawnAndExecute(claudeCli, launchArgs, { cwd, profile, timeout, resumeSession, sessionId, sessionMgr, + claudeConfigDir: inheritedClaudeConfigDir, + traceEnv, }); } @@ -194,9 +234,20 @@ export class HeadlessExecutor { resumeSession: boolean; sessionId: string | null; sessionMgr: SessionManager; + claudeConfigDir?: string; + traceEnv?: Record; } ): Promise { - const { cwd, profile, timeout, resumeSession, sessionId, sessionMgr } = ctx; + const { + cwd, + profile, + timeout, + resumeSession, + sessionId, + sessionMgr, + claudeConfigDir, + traceEnv = {}, + } = ctx; return new Promise((resolve, reject) => { const startTime = Date.now(); @@ -213,6 +264,10 @@ export class HeadlessExecutor { const cleanEnv = stripClaudeCodeEnv({ ...process.env, ...getClaudeLaunchEnvOverrides(), + ...getWebSearchHookEnv(), + ...traceEnv, + ...(claudeConfigDir ? { CLAUDE_CONFIG_DIR: claudeConfigDir } : {}), + CCS_PROFILE_TYPE: 'settings', }); const proc = spawn(claudeCli, args, { @@ -313,6 +368,51 @@ export class HeadlessExecutor { messages, }); + const launchId = traceEnv.CCS_WEBSEARCH_TRACE_LAUNCH_ID; + if (launchId) { + const launchTraceRecords = readWebSearchTraceRecords(launchId, { + ...process.env, + ...traceEnv, + CCS_PROFILE_TYPE: 'settings', + }); + const mcpSessionSummary = [...launchTraceRecords] + .reverse() + .find((record) => record.event === 'mcp_session_summary'); + const providerSuccess = [...launchTraceRecords] + .reverse() + .find((record) => record.event === 'websearch_provider_success'); + + const exposed = mcpSessionSummary?.exposed === true; + const calledWebSearch = result.toolUsageSummary?.calledWebSearch === true; + const fallbackToolsUsed = result.toolUsageSummary?.fallbackToolsUsed || []; + + appendWebSearchTrace( + 'headless_websearch_summary', + { + profile, + sessionId: result.sessionId || null, + calledWebSearch, + fallbackToolsUsed, + providerUsed: + typeof providerSuccess?.providerName === 'string' + ? providerSuccess.providerName + : null, + exposed, + likelyBypassed: + exposed && !calledWebSearch + ? fallbackToolsUsed.length > 0 + ? true + : 'unknown' + : false, + }, + { + ...process.env, + ...traceEnv, + CCS_PROFILE_TYPE: 'settings', + } + ); + } + // Store session if (result.sessionId) { if (resumeSession || sessionId) { diff --git a/src/management/instance-manager.ts b/src/management/instance-manager.ts index 576bc828..776ebf98 100644 --- a/src/management/instance-manager.ts +++ b/src/management/instance-manager.ts @@ -10,9 +10,12 @@ import * as fs from 'fs'; import * as path from 'path'; import SharedManager from './shared-manager'; import ProfileContextSyncLock from './profile-context-sync-lock'; -import { AccountContextPolicy, DEFAULT_ACCOUNT_CONTEXT_MODE } from '../auth/account-context'; +import { DEFAULT_ACCOUNT_CONTEXT_MODE } from '../auth/account-context'; +import type { AccountContextPolicy } from '../auth/account-context'; import { getCcsDir, getCcsHome } from '../utils/config-manager'; +const MANAGED_MCP_SERVER_NAMES = new Set(['ccs-websearch']); + /** Options for instance creation */ export interface InstanceOptions { /** Skip shared symlinks (commands, skills, agents, settings.json) */ @@ -234,13 +237,20 @@ class InstanceManager { } } - // Merge: global MCP servers as base, instance-specific overrides on top + // Merge: global MCP servers as base, instance-specific overrides on top, + // except for CCS-managed entries which must stay aligned with the global runtime. const rawExistingMcp = instanceContent.mcpServers; const existingMcp = rawExistingMcp && typeof rawExistingMcp === 'object' && !Array.isArray(rawExistingMcp) ? (rawExistingMcp as Record) : {}; - instanceContent.mcpServers = { ...mcpServers, ...existingMcp }; + const mergedMcpServers = { ...mcpServers, ...existingMcp }; + for (const managedName of MANAGED_MCP_SERVER_NAMES) { + if (managedName in mcpServers) { + mergedMcpServers[managedName] = mcpServers[managedName]; + } + } + instanceContent.mcpServers = mergedMcpServers; fs.writeFileSync(instanceClaudeJson, JSON.stringify(instanceContent, null, 2), { encoding: 'utf8', diff --git a/src/utils/claude-config-path.ts b/src/utils/claude-config-path.ts index 351c5be4..1c840041 100644 --- a/src/utils/claude-config-path.ts +++ b/src/utils/claude-config-path.ts @@ -25,6 +25,11 @@ export function getClaudeConfigDir(): string { return getDefaultClaudeConfigDir(); } +/** Resolve user-scope Claude JSON config path (~/.claude.json). */ +export function getClaudeUserConfigPath(): string { + return path.join(getCcsHome(), '.claude.json'); +} + /** Resolve Claude settings.json path. */ export function getClaudeSettingsPath(): string { return path.join(getClaudeConfigDir(), 'settings.json'); diff --git a/src/utils/websearch-manager.ts b/src/utils/websearch-manager.ts index 9c4a6dc7..e78bf4f8 100644 --- a/src/utils/websearch-manager.ts +++ b/src/utils/websearch-manager.ts @@ -1,15 +1,15 @@ /** - * WebSearch Manager - Manages WebSearch hook for CCS + * WebSearch Manager - Manages CCS WebSearch runtime * * WebSearch is a server-side tool executed by Anthropic's API. * Third-party providers (gemini, agy, codex, qwen) don't have access. - * This manager installs a hook that uses deterministic local search backends, - * with legacy AI CLI tools kept only as optional fallback. + * CCS exposes a first-class local WebSearch tool for those profiles and keeps + * the legacy hook runtime only as a compatibility fallback. * - * Simplified Architecture: - * - No MCP dependency for the default path - * - Uses real search providers first (DuckDuckGo, Brave) - * - Keeps Gemini/OpenCode/Grok as compatibility fallback only + * Runtime Architecture: + * - User-scope MCP server in ~/.claude.json for third-party profiles + * - Real search providers first (Exa, Tavily, Brave Search, DuckDuckGo) + * - Gemini/OpenCode/Grok retained as optional legacy fallback * * @module utils/websearch-manager */ @@ -53,6 +53,31 @@ export { // Re-export hook environment export { getWebSearchHookEnv } from './websearch/hook-env'; +// Re-export MCP runtime helpers +export { + getWebSearchMcpServerName, + getWebSearchMcpServerPath, + installWebSearchMcpServer, + ensureWebSearchMcpConfig, + ensureWebSearchMcp, + uninstallWebSearchMcpServer, + removeWebSearchMcpConfig, + uninstallWebSearchMcp, + syncWebSearchMcpToConfigDir, + ensureWebSearchMcpOrThrow, +} from './websearch/mcp-installer'; + +// Re-export Claude launch arg helpers +export { appendThirdPartyWebSearchToolArgs } from './websearch/claude-tool-args'; + +// Re-export trace helpers +export { + appendWebSearchTrace, + createWebSearchTraceContext, + isWebSearchTraceEnabled, + readWebSearchTraceRecords, +} from './websearch/trace'; + // Re-export status and readiness functions export { getWebSearchCliProviders, @@ -62,7 +87,7 @@ export { displayWebSearchStatus, } from './websearch/status'; -// Re-export profile hook injection +// Re-export profile compatibility hook injection export { ensureProfileHooks, ensureProfileHooksOrThrow } from './websearch/profile-hook-injector'; // Import for local use @@ -77,11 +102,4 @@ export function clearAllCliCaches(): void { clearOpenCodeCliCache(); } -// ========== Backward Compatibility Exports ========== - -/** - * @deprecated Use installWebSearchHook instead - MCP is no longer used - */ -export function ensureMcpWebSearch(): boolean { - return false; -} +export { ensureWebSearchMcp as ensureMcpWebSearch } from './websearch/mcp-installer'; diff --git a/src/utils/websearch/claude-tool-args.ts b/src/utils/websearch/claude-tool-args.ts new file mode 100644 index 00000000..15028178 --- /dev/null +++ b/src/utils/websearch/claude-tool-args.ts @@ -0,0 +1,153 @@ +/** + * Claude launch argument helpers for third-party WebSearch. + */ + +const NATIVE_WEBSEARCH_TOOL = 'WebSearch'; +const DISALLOWED_TOOLS_FLAG = '--disallowedTools'; +const APPEND_SYSTEM_PROMPT_FLAG = '--append-system-prompt'; +const THIRD_PARTY_WEBSEARCH_STEERING_PROMPT = + 'For web lookup or current-information requests, prefer the CCS MCP tool WebSearch instead of Bash/curl/http fetches. If the user explicitly wants shell commands, or WebSearch is unavailable or fails, you may fall back to Bash/network tools.'; + +function parseToolValue(rawValue: string): string[] { + return rawValue + .split(',') + .map((value) => value.trim()) + .filter((value) => value.length > 0); +} + +function mergeToolValues(rawValues: string[], toolName: string): string { + const merged = rawValues.flatMap(parseToolValue); + if (!merged.includes(toolName)) { + merged.push(toolName); + } + return merged.join(','); +} + +function splitArgsAtTerminator(args: string[]): { optionArgs: string[]; trailingArgs: string[] } { + const terminatorIndex = args.indexOf('--'); + if (terminatorIndex === -1) { + return { optionArgs: args, trailingArgs: [] }; + } + + return { + optionArgs: args.slice(0, terminatorIndex), + trailingArgs: args.slice(terminatorIndex), + }; +} + +function getImmediateFlagValue(args: string[], index: number): string | null { + const value = args[index + 1]; + if (value === undefined || value === '--' || value.startsWith('--')) { + return null; + } + return value; +} + +function hasToolInFlag(args: string[], flag: string, toolName: string): boolean { + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + + if (arg === flag) { + const value = getImmediateFlagValue(args, index); + if (value && parseToolValue(value).includes(toolName)) { + return true; + } + continue; + } + + if (!arg.startsWith(`${flag}=`)) { + continue; + } + + const rawValue = arg.slice(flag.length + 1); + if (parseToolValue(rawValue).includes(toolName)) { + return true; + } + } + + return false; +} + +function hasExactFlagValue(args: string[], flag: string, expectedValue: string): boolean { + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + + if (arg === flag) { + const value = getImmediateFlagValue(args, index); + if (value === expectedValue) { + return true; + } + continue; + } + + if (arg === `${flag}=${expectedValue}`) { + return true; + } + + if (arg.startsWith(`${flag}=`) && arg.slice(flag.length + 1) === expectedValue) { + return true; + } + } + + return false; +} + +function ensureDisallowedNativeWebSearchTool(args: string[]): string[] { + const { optionArgs, trailingArgs } = splitArgsAtTerminator(args); + + if (hasToolInFlag(optionArgs, DISALLOWED_TOOLS_FLAG, NATIVE_WEBSEARCH_TOOL)) { + return args; + } + + for (let index = 0; index < optionArgs.length; index += 1) { + const arg = optionArgs[index]; + + if (arg === DISALLOWED_TOOLS_FLAG) { + const currentValue = getImmediateFlagValue(optionArgs, index); + const mergedValue = mergeToolValues( + currentValue ? [currentValue] : [], + NATIVE_WEBSEARCH_TOOL + ); + + return [ + ...optionArgs.slice(0, index + 1), + mergedValue, + ...optionArgs.slice(currentValue === null ? index + 1 : index + 2), + ...trailingArgs, + ]; + } + + if (arg.startsWith(`${DISALLOWED_TOOLS_FLAG}=`)) { + const rawValue = arg.slice(DISALLOWED_TOOLS_FLAG.length + 1); + return [ + ...optionArgs.slice(0, index), + `${DISALLOWED_TOOLS_FLAG}=${mergeToolValues([rawValue], NATIVE_WEBSEARCH_TOOL)}`, + ...optionArgs.slice(index + 1), + ...trailingArgs, + ]; + } + } + + return [...optionArgs, DISALLOWED_TOOLS_FLAG, NATIVE_WEBSEARCH_TOOL, ...trailingArgs]; +} + +function ensureWebSearchSteeringPrompt(args: string[]): string[] { + const { optionArgs, trailingArgs } = splitArgsAtTerminator(args); + + if ( + hasExactFlagValue(optionArgs, APPEND_SYSTEM_PROMPT_FLAG, THIRD_PARTY_WEBSEARCH_STEERING_PROMPT) + ) { + return args; + } + + return [ + ...optionArgs, + APPEND_SYSTEM_PROMPT_FLAG, + THIRD_PARTY_WEBSEARCH_STEERING_PROMPT, + ...trailingArgs, + ]; +} + +export function appendThirdPartyWebSearchToolArgs(args: string[]): string[] { + return ensureWebSearchSteeringPrompt(ensureDisallowedNativeWebSearchTool(args)); +} diff --git a/src/utils/websearch/hook-env.ts b/src/utils/websearch/hook-env.ts index ac0e9e1b..d5e69018 100644 --- a/src/utils/websearch/hook-env.ts +++ b/src/utils/websearch/hook-env.ts @@ -7,6 +7,7 @@ */ import { getWebSearchConfig } from '../../config/unified-config-loader'; +import { resolveAllowedWebSearchTraceFile } from './trace'; /** * Get environment variables for WebSearch hook configuration. @@ -19,6 +20,14 @@ export function getWebSearchHookEnv(): Record { const wsConfig = getWebSearchConfig(); const env: Record = {}; + if (process.env.CCS_WEBSEARCH_TRACE === '1' || process.env.CCS_DEBUG === '1') { + env.CCS_WEBSEARCH_TRACE = '1'; + } + const traceFileOverride = resolveAllowedWebSearchTraceFile(process.env); + if (traceFileOverride) { + env.CCS_WEBSEARCH_TRACE_FILE = traceFileOverride; + } + // Skip hook entirely if disabled if (!wsConfig.enabled) { env.CCS_WEBSEARCH_SKIP = '1'; diff --git a/src/utils/websearch/hook-installer.ts b/src/utils/websearch/hook-installer.ts index a2c7149a..dc5c1f70 100644 --- a/src/utils/websearch/hook-installer.ts +++ b/src/utils/websearch/hook-installer.ts @@ -1,7 +1,7 @@ /** * WebSearch Hook Installer * - * Manages installation and uninstallation of the WebSearch hook. + * Manages installation and uninstallation of the legacy WebSearch hook runtime. * * @module utils/websearch/hook-installer */ @@ -70,7 +70,8 @@ export function hasWebSearchHook(): boolean { /** * Install WebSearch hook to ~/.ccs/hooks/ * - * This hook intercepts WebSearch and executes via Gemini CLI. + * This hook now serves as a compatibility fallback and a shared provider runtime + * for the first-class CCS WebSearch MCP server. * * @returns true if hook installed successfully */ diff --git a/src/utils/websearch/index.ts b/src/utils/websearch/index.ts index 83c240de..0d375dfc 100644 --- a/src/utils/websearch/index.ts +++ b/src/utils/websearch/index.ts @@ -50,6 +50,31 @@ export { removeHookConfig } from './hook-config'; // Hook Environment export { getWebSearchHookEnv } from './hook-env'; +// MCP Runtime +export { + getWebSearchMcpServerName, + getWebSearchMcpServerPath, + installWebSearchMcpServer, + ensureWebSearchMcpConfig, + ensureWebSearchMcp, + uninstallWebSearchMcpServer, + removeWebSearchMcpConfig, + uninstallWebSearchMcp, + syncWebSearchMcpToConfigDir, + ensureWebSearchMcpOrThrow, +} from './mcp-installer'; + +// Claude launch args +export { appendThirdPartyWebSearchToolArgs } from './claude-tool-args'; + +// Trace helpers +export { + appendWebSearchTrace, + createWebSearchTraceContext, + isWebSearchTraceEnabled, + readWebSearchTraceRecords, +} from './trace'; + // Status and Readiness export { getWebSearchCliProviders, @@ -61,5 +86,5 @@ export { export { WEBSEARCH_API_KEY_PROVIDERS, getWebSearchApiKeyStates } from './provider-secrets'; -// Profile Hook Injection +// Profile compatibility hook injection export { ensureProfileHooks, ensureProfileHooksOrThrow } from './profile-hook-injector'; diff --git a/src/utils/websearch/mcp-installer.ts b/src/utils/websearch/mcp-installer.ts new file mode 100644 index 00000000..5f7260e9 --- /dev/null +++ b/src/utils/websearch/mcp-installer.ts @@ -0,0 +1,378 @@ +/** + * WebSearch MCP installer and ~/.claude.json provisioning. + */ + +import * as fs from 'fs'; +import * as path from 'path'; +import { getWebSearchConfig } from '../../config/unified-config-loader'; +import { getCcsDir } from '../config-manager'; +import { getClaudeUserConfigPath } from '../claude-config-path'; +import { info, warn } from '../ui'; +import { InstanceManager } from '../../management/instance-manager'; +import { installWebSearchHook } from './hook-installer'; +import { appendWebSearchTrace } from './trace'; + +const WEBSEARCH_MCP_SERVER = 'ccs-websearch-server.cjs'; +const WEBSEARCH_MCP_SERVER_NAME = 'ccs-websearch'; + +interface ClaudeUserConfig { + mcpServers?: Record; + [key: string]: unknown; +} + +interface ManagedWebSearchMcpConfig { + type: 'stdio'; + command: 'node'; + args: [string]; + env: Record; +} + +function getCcsMcpDir(): string { + return path.join(getCcsDir(), 'mcp'); +} + +export function getWebSearchMcpServerName(): string { + return WEBSEARCH_MCP_SERVER_NAME; +} + +export function getWebSearchMcpServerPath(): string { + return path.join(getCcsMcpDir(), WEBSEARCH_MCP_SERVER); +} + +function hasMatchingContents(sourcePath: string, destinationPath: string): boolean { + if (!fs.existsSync(destinationPath)) { + return false; + } + + const source = fs.readFileSync(sourcePath); + try { + const destination = fs.readFileSync(destinationPath); + return source.equals(destination); + } catch (error) { + if (process.env.CCS_DEBUG) { + console.error( + warn(`Existing WebSearch MCP server is unreadable: ${(error as Error).message}`) + ); + } + return false; + } +} + +function getTempPath(targetPath: string): string { + const suffix = `${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`; + return `${targetPath}.${suffix}.tmp`; +} + +function resolveBundledServerSourcePath(): string | null { + const possiblePaths = [ + path.join(__dirname, '..', '..', '..', 'lib', 'mcp', WEBSEARCH_MCP_SERVER), + path.join(__dirname, '..', '..', 'lib', 'mcp', WEBSEARCH_MCP_SERVER), + path.join(__dirname, '..', 'lib', 'mcp', WEBSEARCH_MCP_SERVER), + ]; + + for (const candidate of possiblePaths) { + if (fs.existsSync(candidate)) { + return candidate; + } + } + + return null; +} + +function readClaudeUserConfig(configPath: string): ClaudeUserConfig | null { + if (!fs.existsSync(configPath)) { + return {}; + } + + try { + const raw = fs.readFileSync(configPath, 'utf8'); + const parsed = JSON.parse(raw); + if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) { + return null; + } + return parsed as ClaudeUserConfig; + } catch { + return null; + } +} + +function writeClaudeUserConfig(configPath: string, config: ClaudeUserConfig): boolean { + const tempPath = getTempPath(configPath); + const fileMode = fs.existsSync(configPath) ? fs.statSync(configPath).mode & 0o777 : 0o600; + + try { + fs.writeFileSync(tempPath, JSON.stringify(config, null, 2) + '\n', 'utf8'); + fs.chmodSync(tempPath, fileMode); + fs.renameSync(tempPath, configPath); + return true; + } finally { + if (fs.existsSync(tempPath)) { + fs.unlinkSync(tempPath); + } + } +} + +function removeManagedServerConfig(configPath: string): boolean { + if (!fs.existsSync(configPath)) { + return false; + } + + const config = readClaudeUserConfig(configPath); + if (config === null) { + if (process.env.CCS_DEBUG) { + console.error(warn(`Malformed Claude config prevents MCP cleanup: ${configPath}`)); + } + return false; + } + + const existingServers = + config.mcpServers && typeof config.mcpServers === 'object' && !Array.isArray(config.mcpServers) + ? { ...(config.mcpServers as Record) } + : {}; + + if (!(WEBSEARCH_MCP_SERVER_NAME in existingServers)) { + return false; + } + + delete existingServers[WEBSEARCH_MCP_SERVER_NAME]; + + const nextConfig: ClaudeUserConfig = { ...config }; + if (Object.keys(existingServers).length === 0) { + delete nextConfig.mcpServers; + } else { + nextConfig.mcpServers = existingServers; + } + + try { + writeClaudeUserConfig(configPath, nextConfig); + if (process.env.CCS_DEBUG) { + console.error(info(`Removed WebSearch MCP config from ${configPath}`)); + } + return true; + } catch (error) { + if (process.env.CCS_DEBUG) { + console.error( + warn( + `Failed to remove WebSearch MCP config from ${configPath}: ${(error as Error).message}` + ) + ); + } + return false; + } +} + +export function installWebSearchMcpServer(): boolean { + const wsConfig = getWebSearchConfig(); + if (!wsConfig.enabled) { + appendWebSearchTrace('websearch_mcp_install_skipped', { reason: 'disabled' }); + return false; + } + + if (!installWebSearchHook()) { + appendWebSearchTrace('websearch_mcp_install_failed', { reason: 'hook_unavailable' }); + if (process.env.CCS_DEBUG) { + console.error( + warn('WebSearch MCP server install skipped because hook runtime is unavailable') + ); + } + return false; + } + + const sourcePath = resolveBundledServerSourcePath(); + if (!sourcePath) { + appendWebSearchTrace('websearch_mcp_install_failed', { reason: 'source_missing' }); + if (process.env.CCS_DEBUG) { + console.error(warn(`WebSearch MCP server source not found: ${WEBSEARCH_MCP_SERVER}`)); + } + return false; + } + + const mcpDir = getCcsMcpDir(); + if (!fs.existsSync(mcpDir)) { + fs.mkdirSync(mcpDir, { recursive: true, mode: 0o700 }); + } + + const serverPath = getWebSearchMcpServerPath(); + if (hasMatchingContents(sourcePath, serverPath)) { + appendWebSearchTrace('websearch_mcp_install_ready', { serverPath }); + return true; + } + + const tempPath = getTempPath(serverPath); + + try { + fs.copyFileSync(sourcePath, tempPath); + fs.chmodSync(tempPath, 0o755); + try { + fs.renameSync(tempPath, serverPath); + } catch (renameError) { + const errorCode = (renameError as NodeJS.ErrnoException).code; + if (errorCode !== 'EEXIST' && errorCode !== 'EPERM') { + throw renameError; + } + + if (!hasMatchingContents(sourcePath, serverPath)) { + fs.copyFileSync(tempPath, serverPath); + fs.chmodSync(serverPath, 0o755); + } + } + appendWebSearchTrace('websearch_mcp_install_ready', { serverPath }); + return true; + } catch (error) { + appendWebSearchTrace('websearch_mcp_install_failed', { + reason: 'copy_failed', + error: (error as Error).message, + }); + if (process.env.CCS_DEBUG) { + console.error(warn(`Failed to install WebSearch MCP server: ${(error as Error).message}`)); + } + return false; + } finally { + if (fs.existsSync(tempPath)) { + fs.unlinkSync(tempPath); + } + } +} + +export function ensureWebSearchMcpConfig(): boolean { + const wsConfig = getWebSearchConfig(); + if (!wsConfig.enabled) { + appendWebSearchTrace('websearch_mcp_config_skipped', { reason: 'disabled' }); + return false; + } + + const claudeUserConfigPath = getClaudeUserConfigPath(); + const claudeUserConfigDir = path.dirname(claudeUserConfigPath); + const config = readClaudeUserConfig(claudeUserConfigPath); + + if (config === null) { + appendWebSearchTrace('websearch_mcp_config_failed', { reason: 'malformed_user_config' }); + if (process.env.CCS_DEBUG) { + console.error(warn('Malformed ~/.claude.json prevents WebSearch MCP provisioning')); + } + return false; + } + + if (!fs.existsSync(claudeUserConfigDir)) { + fs.mkdirSync(claudeUserConfigDir, { recursive: true, mode: 0o700 }); + } + + const existingServers = + config.mcpServers && typeof config.mcpServers === 'object' && !Array.isArray(config.mcpServers) + ? (config.mcpServers as Record) + : {}; + const desiredServerConfig: ManagedWebSearchMcpConfig = { + type: 'stdio', + command: 'node', + args: [getWebSearchMcpServerPath()], + env: {}, + }; + + const currentConfig = existingServers[WEBSEARCH_MCP_SERVER_NAME]; + if ( + typeof currentConfig === 'object' && + currentConfig !== null && + JSON.stringify(currentConfig) === JSON.stringify(desiredServerConfig) + ) { + appendWebSearchTrace('websearch_mcp_config_ready', { configPath: claudeUserConfigPath }); + return true; + } + + const nextConfig: ClaudeUserConfig = { + ...config, + mcpServers: { + ...existingServers, + [WEBSEARCH_MCP_SERVER_NAME]: desiredServerConfig, + }, + }; + + try { + writeClaudeUserConfig(claudeUserConfigPath, nextConfig); + appendWebSearchTrace('websearch_mcp_config_ready', { configPath: claudeUserConfigPath }); + if (process.env.CCS_DEBUG) { + console.error(info(`Ensured WebSearch MCP config in ${claudeUserConfigPath}`)); + } + return true; + } catch (error) { + appendWebSearchTrace('websearch_mcp_config_failed', { + reason: 'write_failed', + configPath: claudeUserConfigPath, + error: (error as Error).message, + }); + if (process.env.CCS_DEBUG) { + console.error(warn(`Failed to update ~/.claude.json: ${(error as Error).message}`)); + } + return false; + } +} + +export function ensureWebSearchMcp(): boolean { + const wsConfig = getWebSearchConfig(); + if (!wsConfig.enabled) { + appendWebSearchTrace('websearch_mcp_ensure_skipped', { reason: 'disabled' }); + return false; + } + + const installed = installWebSearchMcpServer(); + const configured = installed && ensureWebSearchMcpConfig(); + appendWebSearchTrace('websearch_mcp_ensure_result', { installed, configured }); + return installed && configured; +} + +export function syncWebSearchMcpToConfigDir(claudeConfigDir: string | undefined): boolean { + if (!claudeConfigDir) { + appendWebSearchTrace('websearch_mcp_sync_skipped', { reason: 'missing_config_dir' }); + return false; + } + + const synced = new InstanceManager().syncMcpServers(claudeConfigDir); + appendWebSearchTrace('websearch_mcp_sync_result', { claudeConfigDir, synced }); + return synced; +} + +export function uninstallWebSearchMcpServer(): boolean { + const serverPath = getWebSearchMcpServerPath(); + if (!fs.existsSync(serverPath)) { + return false; + } + + try { + fs.unlinkSync(serverPath); + return true; + } catch (error) { + if (process.env.CCS_DEBUG) { + console.error(warn(`Failed to remove WebSearch MCP server: ${(error as Error).message}`)); + } + return false; + } +} + +export function removeWebSearchMcpConfig(): boolean { + let removed = removeManagedServerConfig(getClaudeUserConfigPath()); + + const instanceManager = new InstanceManager(); + for (const instanceName of instanceManager.listInstances()) { + const instancePath = instanceManager.getInstancePath(instanceName); + const instanceClaudeConfigPath = path.join(instancePath, '.claude.json'); + removed = removeManagedServerConfig(instanceClaudeConfigPath) || removed; + } + + return removed; +} + +export function uninstallWebSearchMcp(): boolean { + const removedConfig = removeWebSearchMcpConfig(); + const removedServer = uninstallWebSearchMcpServer(); + return removedConfig || removedServer; +} + +export function ensureWebSearchMcpOrThrow(): void { + const wsConfig = getWebSearchConfig(); + if (!wsConfig.enabled) { + return; + } + + if (!ensureWebSearchMcp()) { + throw new Error('WebSearch is enabled, but CCS could not prepare the local WebSearch tool.'); + } +} diff --git a/src/utils/websearch/profile-hook-injector.ts b/src/utils/websearch/profile-hook-injector.ts index cb98412d..62bbed64 100644 --- a/src/utils/websearch/profile-hook-injector.ts +++ b/src/utils/websearch/profile-hook-injector.ts @@ -1,8 +1,9 @@ /** * Profile Hook Injector * - * Injects WebSearch hooks into per-profile settings files. - * This replaces the global ~/.claude/settings.json approach. + * Injects the legacy WebSearch compatibility hook into per-profile settings files. + * The first-class runtime now uses the CCS-managed MCP server; these hooks remain + * for compatibility and migration safety only. * * @module utils/websearch/profile-hook-injector */ @@ -76,7 +77,8 @@ function migrateGlobalHook(): void { } /** - * Ensure WebSearch hook is configured in profile's settings file + * Ensure the legacy WebSearch compatibility hook is configured in a profile's + * settings file when that path is still needed. * * @param profileName - Name of the profile (e.g., 'agy', 'gemini', 'glm') * @returns true if hook is configured (existing or newly added) diff --git a/src/utils/websearch/status.ts b/src/utils/websearch/status.ts index b812f7d9..ecb5ce94 100644 --- a/src/utils/websearch/status.ts +++ b/src/utils/websearch/status.ts @@ -6,18 +6,105 @@ * @module utils/websearch/status */ +import { existsSync, readFileSync } from 'fs'; +import { join } from 'path'; import { ok, warn, fail, info } from '../ui'; import { getWebSearchConfig } from '../../config/unified-config-loader'; +import { getCcsDir } from '../config-manager'; import { getGeminiCliStatus, isGeminiAuthenticated } from './gemini-cli'; import { getGrokCliStatus } from './grok-cli'; import { getOpenCodeCliStatus } from './opencode-cli'; import { getWebSearchApiKeyStates } from './provider-secrets'; import type { WebSearchCliInfo, WebSearchStatus } from './types'; +const PROVIDER_STATE_FILE = 'websearch-provider-state.json'; + +type ProviderCooldown = { + reason: string; + until: number; +}; + function hasEnvValue(name: string): boolean { return (process.env[name] || '').trim().length > 0; } +function getProviderStatePath(): string { + return join(getCcsDir(), 'cache', PROVIDER_STATE_FILE); +} + +function readProviderCooldowns(now = Date.now()): Record { + try { + const statePath = getProviderStatePath(); + if (!existsSync(statePath)) { + return {}; + } + + const parsed = JSON.parse(readFileSync(statePath, 'utf8')) as { + cooldowns?: Record; + }; + const nextCooldowns: Record = {}; + + for (const [providerId, entry] of Object.entries(parsed.cooldowns || {})) { + const until = Number.parseInt(String(entry?.until || ''), 10); + if (!Number.isFinite(until) || until <= now) { + continue; + } + + nextCooldowns[providerId] = { + reason: typeof entry?.reason === 'string' ? entry.reason : 'rate_limited', + until, + }; + } + + return nextCooldowns; + } catch { + return {}; + } +} + +function formatCooldownDuration(until: number, now = Date.now()): string { + const remainingSec = Math.max(1, Math.ceil((until - now) / 1000)); + if (remainingSec >= 3600) { + return `~${Math.ceil(remainingSec / 3600)}h`; + } + if (remainingSec >= 60) { + return `~${Math.ceil(remainingSec / 60)}m`; + } + return `~${remainingSec}s`; +} + +function formatCooldownReason(reason: string): string { + switch (reason) { + case 'quota_exhausted': + return 'quota exhaustion'; + case 'rate_limited': + return 'rate limiting'; + default: + return 'a temporary provider error'; + } +} + +function applyCooldownStatus( + provider: WebSearchCliInfo, + cooldowns: Record, + now = Date.now() +): WebSearchCliInfo { + if (!(provider.enabled && provider.available)) { + return provider; + } + + const cooldown = cooldowns[provider.id]; + if (!cooldown) { + return provider; + } + + return { + ...provider, + available: false, + detail: `Cooling down ${formatCooldownDuration(cooldown.until, now)} after ${formatCooldownReason(cooldown.reason)}`, + }; +} + function getLegacyProviderStatuses(): WebSearchCliInfo[] { const wsConfig = getWebSearchConfig(); const geminiStatus = getGeminiCliStatus(); @@ -86,6 +173,7 @@ function getLegacyProviderStatuses(): WebSearchCliInfo[] { export function getWebSearchCliProviders(): WebSearchCliInfo[] { const wsConfig = getWebSearchConfig(); const apiKeyStates = getWebSearchApiKeyStates(); + const cooldowns = readProviderCooldowns(); const providers: WebSearchCliInfo[] = [ { id: 'exa', @@ -152,7 +240,9 @@ export function getWebSearchCliProviders(): WebSearchCliInfo[] { }, ]; - return [...providers, ...getLegacyProviderStatuses()]; + return [...providers, ...getLegacyProviderStatuses()].map((provider) => + applyCooldownStatus(provider, cooldowns) + ); } /** diff --git a/src/utils/websearch/trace.ts b/src/utils/websearch/trace.ts new file mode 100644 index 00000000..6f4f9880 --- /dev/null +++ b/src/utils/websearch/trace.ts @@ -0,0 +1,234 @@ +/** + * Best-effort WebSearch trace helpers. + * + * Writes opt-in JSONL trace records to ~/.ccs/logs/websearch-trace.jsonl so + * CCS can explain launch intent, MCP exposure, provider selection, and likely + * bypass scenarios without polluting Claude/MCP stdout. + */ + +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { getCcsDir } from '../config-manager'; + +const TRACE_FILE_NAME = 'websearch-trace.jsonl'; +const NATIVE_WEBSEARCH_TOOL = 'WebSearch'; +const DISALLOWED_TOOLS_FLAG = '--disallowedTools'; +const APPEND_SYSTEM_PROMPT_FLAG = '--append-system-prompt'; +const THIRD_PARTY_WEBSEARCH_STEERING_PROMPT = + 'For web lookup or current-information requests, prefer the CCS MCP tool WebSearch instead of Bash/curl/http fetches. If the user explicitly wants shell commands, or WebSearch is unavailable or fails, you may fall back to Bash/network tools.'; + +function parseToolValue(rawValue: string): string[] { + return rawValue + .split(',') + .map((value) => value.trim()) + .filter((value) => value.length > 0); +} + +function getImmediateFlagValue(args: string[], index: number): string | null { + const value = args[index + 1]; + if (value === undefined || value === '--' || value.startsWith('--')) { + return null; + } + return value; +} + +function hasToolInFlag(args: string[], flag: string, toolName: string): boolean { + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + + if (arg === flag) { + const value = getImmediateFlagValue(args, index); + if (value && parseToolValue(value).includes(toolName)) { + return true; + } + continue; + } + + if (arg.startsWith(`${flag}=`)) { + const rawValue = arg.slice(flag.length + 1); + if (parseToolValue(rawValue).includes(toolName)) { + return true; + } + } + } + + return false; +} + +function hasExactFlagValue(args: string[], flag: string, expectedValue: string): boolean { + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + + if (arg === flag) { + if (getImmediateFlagValue(args, index) === expectedValue) { + return true; + } + continue; + } + + if (arg === `${flag}=${expectedValue}`) { + return true; + } + } + + return false; +} + +function normalizeSafePrefix(inputPath: string): string { + return `${path.resolve(inputPath)}${path.sep}`; +} + +function getSafeTracePrefixes(): string[] { + return [ + normalizeSafePrefix(path.join(getCcsDir(), 'logs')), + normalizeSafePrefix(os.tmpdir()), + normalizeSafePrefix('/var/log'), + ]; +} + +export function resolveAllowedWebSearchTraceFile( + env: NodeJS.ProcessEnv = process.env +): string | null { + const configured = env.CCS_WEBSEARCH_TRACE_FILE?.trim(); + if (!configured) { + return null; + } + + const resolved = path.resolve(configured); + if (getSafeTracePrefixes().some((prefix) => resolved.startsWith(prefix))) { + return resolved; + } + + return null; +} + +function getTraceFilePath(env: NodeJS.ProcessEnv): string { + return resolveAllowedWebSearchTraceFile(env) ?? path.join(getCcsDir(), 'logs', TRACE_FILE_NAME); +} + +export function isWebSearchTraceEnabled(env: NodeJS.ProcessEnv = process.env): boolean { + return env.CCS_WEBSEARCH_TRACE === '1' || env.CCS_DEBUG === '1'; +} + +export function appendWebSearchTrace( + event: string, + payload: Record = {}, + env: NodeJS.ProcessEnv = process.env +): void { + if (!isWebSearchTraceEnabled(env)) { + return; + } + + try { + const traceFilePath = getTraceFilePath(env); + fs.mkdirSync(path.dirname(traceFilePath), { recursive: true }); + fs.appendFileSync( + traceFilePath, + JSON.stringify({ + at: new Date().toISOString(), + event, + launchId: env.CCS_WEBSEARCH_TRACE_LAUNCH_ID || null, + launcher: env.CCS_WEBSEARCH_TRACE_LAUNCHER || null, + profileType: env.CCS_PROFILE_TYPE || null, + pid: process.pid, + ...payload, + }) + '\n', + 'utf8' + ); + } catch { + // Tracing must never affect launch behavior. + } +} + +export function readWebSearchTraceRecords( + launchId: string, + env: NodeJS.ProcessEnv = process.env +): Array> { + if (!launchId) { + return []; + } + + try { + const traceFilePath = getTraceFilePath(env); + if (!fs.existsSync(traceFilePath)) { + return []; + } + + return fs + .readFileSync(traceFilePath, 'utf8') + .split('\n') + .map((line) => line.trim()) + .filter((line) => line.length > 0) + .map((line) => JSON.parse(line) as Record) + .filter((record) => record.launchId === launchId); + } catch { + return []; + } +} + +function buildLaunchId(): string { + const random = Math.random().toString(36).slice(2, 10); + return `websearch-${Date.now()}-${process.pid}-${random}`; +} + +function summarizeLaunchArgs(args: string[]): Record { + return { + argCount: args.length, + hasSettingsFlag: args.includes('--settings'), + nativeWebSearchDisallowed: hasToolInFlag(args, DISALLOWED_TOOLS_FLAG, NATIVE_WEBSEARCH_TOOL), + steeringPromptApplied: hasExactFlagValue( + args, + APPEND_SYSTEM_PROMPT_FLAG, + THIRD_PARTY_WEBSEARCH_STEERING_PROMPT + ), + }; +} + +export function createWebSearchTraceContext(params: { + launcher: string; + args: string[]; + cwd?: string; + profile?: string; + profileType?: string; + settingsPath?: string; + claudeConfigDir?: string; + env?: NodeJS.ProcessEnv; +}): Record { + const env = params.env ?? process.env; + if (!isWebSearchTraceEnabled(env)) { + return {}; + } + + const launchId = buildLaunchId(); + const traceEnv: Record = { + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: launchId, + CCS_WEBSEARCH_TRACE_LAUNCHER: params.launcher, + }; + + const traceFileOverride = resolveAllowedWebSearchTraceFile(env); + if (traceFileOverride) { + traceEnv.CCS_WEBSEARCH_TRACE_FILE = traceFileOverride; + } + + appendWebSearchTrace( + 'ccs_websearch_launch', + { + launcher: params.launcher, + profile: params.profile || null, + profileType: params.profileType || null, + cwd: params.cwd || null, + settingsPath: params.settingsPath || null, + claudeConfigDir: params.claudeConfigDir || null, + ...summarizeLaunchArgs(params.args), + }, + { + ...env, + ...traceEnv, + CCS_PROFILE_TYPE: params.profileType || env.CCS_PROFILE_TYPE || '', + } + ); + + return traceEnv; +} diff --git a/tests/unit/api/profile-lifecycle-service.test.ts b/tests/unit/api/profile-lifecycle-service.test.ts index 758c9aec..bd832b47 100644 --- a/tests/unit/api/profile-lifecycle-service.test.ts +++ b/tests/unit/api/profile-lifecycle-service.test.ts @@ -128,7 +128,7 @@ describe('profile lifecycle service', () => { expect(result.skipped).toEqual([]); }); - it('does not register orphan profiles when WebSearch hook setup fails', async () => { + it('does not register orphan profiles when local WebSearch tool setup fails', async () => { const ccsDir = path.join(tempHome, '.ccs'); fs.mkdirSync(ccsDir, { recursive: true }); @@ -152,7 +152,7 @@ describe('profile lifecycle service', () => { expect(copyFileSpy).toHaveBeenCalled(); expect(result.registered).toEqual([]); expect(result.skipped).toHaveLength(1); - expect(result.skipped[0]?.reason).toContain('could not prepare the profile hook'); + expect(result.skipped[0]?.reason).toContain('could not prepare the local WebSearch tool'); expect(config.profiles.extra).toBeUndefined(); }); @@ -240,7 +240,7 @@ describe('profile lifecycle service', () => { expect(result.error).toContain('Invalid source profile name'); }); - it('rolls back copied settings when WebSearch hook setup fails', async () => { + it('rolls back copied settings when local WebSearch tool setup fails', async () => { const ccsDir = path.join(tempHome, '.ccs'); fs.mkdirSync(ccsDir, { recursive: true }); fs.writeFileSync( @@ -263,7 +263,7 @@ describe('profile lifecycle service', () => { const result = await runInScopedCcsDir(() => copyApiProfile('source', 'copy-dest')); expect(result.success).toBe(false); - expect(result.error).toContain('could not prepare the profile hook'); + expect(result.error).toContain('could not prepare the local WebSearch tool'); expect(copyFileSpy).toHaveBeenCalled(); expect(fs.existsSync(path.join(ccsDir, 'copy-dest.settings.json'))).toBe(false); }); @@ -287,7 +287,7 @@ describe('profile lifecycle service', () => { expect(result.error).toContain('Invalid bundle profile target'); }); - it('rolls back imported settings when WebSearch hook setup fails', async () => { + it('rolls back imported settings when local WebSearch tool setup fails', async () => { const ccsDir = path.join(tempHome, '.ccs'); fs.mkdirSync(ccsDir, { recursive: true }); fs.writeFileSync( @@ -314,7 +314,7 @@ describe('profile lifecycle service', () => { ); expect(result.success).toBe(false); - expect(result.error).toContain('could not prepare the profile hook'); + expect(result.error).toContain('could not prepare the local WebSearch tool'); expect(copyFileSpy).toHaveBeenCalled(); expect(fs.existsSync(path.join(ccsDir, 'import-failure.settings.json'))).toBe(false); }); diff --git a/tests/unit/api/profile-writer-anthropic.test.ts b/tests/unit/api/profile-writer-anthropic.test.ts index fe1d8973..d3c52fa7 100644 --- a/tests/unit/api/profile-writer-anthropic.test.ts +++ b/tests/unit/api/profile-writer-anthropic.test.ts @@ -104,7 +104,7 @@ describe('profile-writer Anthropic direct', () => { expect(settings.env.ANTHROPIC_API_KEY).toBe(''); }); - it('rolls back the created settings file when WebSearch hook installation fails', () => { + it('rolls back the created settings file when local WebSearch tool setup fails', () => { const copyFileSpy = spyOn(fs, 'copyFileSync').mockImplementation(() => { throw new Error('copy failed'); }); @@ -117,7 +117,7 @@ describe('profile-writer Anthropic direct', () => { ); expect(result.success).toBe(false); - expect(result.error).toContain('could not prepare the profile hook'); + expect(result.error).toContain('could not prepare the local WebSearch tool'); expect(copyFileSpy).toHaveBeenCalled(); expect(fs.existsSync(path.join(tempHome, '.ccs', 'hook-failure.settings.json'))).toBe(false); }); diff --git a/tests/unit/hooks/ccs-websearch-mcp-server.test.ts b/tests/unit/hooks/ccs-websearch-mcp-server.test.ts new file mode 100644 index 00000000..4229687f --- /dev/null +++ b/tests/unit/hooks/ccs-websearch-mcp-server.test.ts @@ -0,0 +1,438 @@ +import { describe, expect, it } from 'bun:test'; +import { spawn } from 'child_process'; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const serverPath = join(process.cwd(), 'lib', 'mcp', 'ccs-websearch-server.cjs'); + +function encodeMessage(message: unknown): string { + return `${JSON.stringify(message)}\n`; +} + +function encodeLegacyMessage(message: unknown): string { + const body = JSON.stringify(message); + return `Content-Length: ${Buffer.byteLength(body)}\r\n\r\n${body}`; +} + +function collectResponses( + child: ReturnType, + expectedCount: number +): Promise>> { + return new Promise((resolve, reject) => { + let buffer = Buffer.alloc(0); + const responses: Array> = []; + const timer = setTimeout(() => reject(new Error('Timed out waiting for MCP responses')), 5000); + + function tryParse(): void { + while (true) { + const startsWithLegacyHeaders = buffer + .slice(0, Math.min(buffer.length, 32)) + .toString('utf8') + .toLowerCase() + .startsWith('content-length:'); + + let body: string; + if (startsWithLegacyHeaders) { + const headerEnd = buffer.indexOf('\r\n\r\n'); + if (headerEnd === -1) { + return; + } + + const headerText = buffer.slice(0, headerEnd).toString('utf8'); + const match = headerText.match(/content-length:\s*(\d+)/i); + if (!match) { + reject(new Error('Missing Content-Length header')); + return; + } + + const contentLength = Number.parseInt(match[1], 10); + const messageEnd = headerEnd + 4 + contentLength; + if (buffer.length < messageEnd) { + return; + } + + body = buffer.slice(headerEnd + 4, messageEnd).toString('utf8'); + buffer = buffer.slice(messageEnd); + } else { + const newlineIndex = buffer.indexOf('\n'); + if (newlineIndex === -1) { + return; + } + + body = buffer.slice(0, newlineIndex).toString('utf8').replace(/\r$/, '').trim(); + buffer = buffer.slice(newlineIndex + 1); + if (!body) { + continue; + } + } + + responses.push(JSON.parse(body) as Record); + + if (responses.length >= expectedCount) { + clearTimeout(timer); + resolve(responses); + return; + } + } + } + + child.stdout.on('data', (chunk: Buffer) => { + buffer = Buffer.concat([buffer, chunk]); + try { + tryParse(); + } catch (error) { + clearTimeout(timer); + reject(error); + } + }); + + child.on('error', (error) => { + clearTimeout(timer); + reject(error); + }); + + child.stderr.on('data', () => { + // Ignore debug noise in tests. + }); + }); +} + +function waitForClose(child: ReturnType): Promise { + return new Promise((resolve, reject) => { + child.once('close', (code) => resolve(code)); + child.once('error', reject); + }); +} + +describe('ccs-websearch MCP server', () => { + it('lists the CCS WebSearch tool and returns provider-backed results', async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'ccs-websearch-mcp-server-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const html = ` + Example title + Example snippet + `.trim(); + writeFileSync( + preloadPath, + `global.fetch = async () => ({ ok: true, text: async () => ${JSON.stringify(html)} });\n`, + 'utf8' + ); + + const child = spawn('node', ['-r', preloadPath, serverPath], { + env: { + ...process.env, + CCS_PROFILE_TYPE: 'settings', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '0', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + + try { + const responsesPromise = collectResponses(child, 3); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { + protocolVersion: '2024-11-05', + capabilities: {}, + clientInfo: { name: 'bun-test', version: '1.0.0' }, + }, + }) + ); + child.stdin.write(encodeMessage({ jsonrpc: '2.0', id: 2, method: 'tools/list' })); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 3, + method: 'tools/call', + params: { name: 'WebSearch', arguments: { query: 'btc price' } }, + }) + ); + + const responses = await responsesPromise; + const toolsList = responses.find((message) => message.id === 2); + const toolCall = responses.find((message) => message.id === 3); + + expect(toolsList?.result).toEqual({ + tools: [ + { + name: 'WebSearch', + description: + 'Third-party WebSearch replacement for CCS-managed Claude launches. Use this instead of Bash/curl/http fetches for web lookups. Provider order: Exa, Tavily, Brave Search, DuckDuckGo, then optional legacy CLI fallback.', + inputSchema: { + type: 'object', + properties: { + query: { + type: 'string', + description: + 'Web query to resolve through CCS providers. Prefer this tool over ad hoc Bash/curl lookups when you need current web information.', + }, + }, + required: ['query'], + additionalProperties: false, + }, + }, + ], + }); + expect(toolCall?.result).toBeDefined(); + expect( + ((toolCall?.result as { content: Array<{ text: string }> }).content[0] || {}).text + ).toContain('CCS local WebSearch evidence'); + expect( + ((toolCall?.result as { content: Array<{ text: string }> }).content[0] || {}).text + ).toContain('Provider: DuckDuckGo'); + } finally { + child.kill(); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + it('accepts the legacy search alias for direct calls', async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'ccs-websearch-mcp-server-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const html = ` + Example title + Example snippet + `.trim(); + writeFileSync( + preloadPath, + `global.fetch = async () => ({ ok: true, text: async () => ${JSON.stringify(html)} });\n`, + 'utf8' + ); + + const child = spawn('node', ['-r', preloadPath, serverPath], { + env: { + ...process.env, + CCS_PROFILE_TYPE: 'settings', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '0', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + + try { + const responsesPromise = collectResponses(child, 2); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { + protocolVersion: '2024-11-05', + capabilities: {}, + clientInfo: { name: 'bun-test', version: '1.0.0' }, + }, + }) + ); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 2, + method: 'tools/call', + params: { name: 'search', arguments: { query: 'btc price' } }, + }) + ); + + const responses = await responsesPromise; + const toolCall = responses.find((message) => message.id === 2); + + expect(toolCall?.result).toBeDefined(); + expect( + ((toolCall?.result as { content: Array<{ text: string }> }).content[0] || {}).text + ).toContain('CCS local WebSearch evidence'); + expect( + ((toolCall?.result as { content: Array<{ text: string }> }).content[0] || {}).text + ).toContain('Provider: DuckDuckGo'); + } finally { + child.kill(); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + it('hides the tool for native account profiles', async () => { + const child = spawn('node', [serverPath], { + env: { + ...process.env, + CCS_PROFILE_TYPE: 'account', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '1', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + + try { + const responsesPromise = collectResponses(child, 2); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { + protocolVersion: '2024-11-05', + capabilities: {}, + clientInfo: { name: 'bun-test', version: '1.0.0' }, + }, + }) + ); + child.stdin.write(encodeMessage({ jsonrpc: '2.0', id: 2, method: 'tools/list' })); + + const responses = await responsesPromise; + const toolsList = responses.find((message) => message.id === 2); + expect(toolsList?.result).toEqual({ tools: [] }); + } finally { + child.kill(); + } + }); + + it('writes trace records for exposure, tool calls, provider success, and session summary', async () => { + const tempDir = mkdtempSync(join(tmpdir(), 'ccs-websearch-mcp-trace-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const ccsHome = join(tempDir, 'home'); + const tracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + const html = ` + Example title + Example snippet + `.trim(); + writeFileSync( + preloadPath, + `global.fetch = async () => ({ ok: true, text: async () => ${JSON.stringify(html)} });\n`, + 'utf8' + ); + + const child = spawn('node', ['-r', preloadPath, serverPath], { + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_PROFILE_TYPE: 'settings', + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'mcp-trace-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '0', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + + try { + const responsesPromise = collectResponses(child, 3); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { + protocolVersion: '2024-11-05', + capabilities: {}, + clientInfo: { name: 'bun-test', version: '1.0.0' }, + }, + }) + ); + child.stdin.write(encodeMessage({ jsonrpc: '2.0', id: 2, method: 'tools/list' })); + child.stdin.write( + encodeMessage({ + jsonrpc: '2.0', + id: 3, + method: 'tools/call', + params: { name: 'WebSearch', arguments: { query: 'btc price' } }, + }) + ); + + await responsesPromise; + } finally { + child.kill(); + await waitForClose(child); + } + + const traceEvents = readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + + expect( + traceEvents.some((event) => event.event === 'mcp_tools_list' && event.exposed === true) + ).toBe(true); + expect( + traceEvents.some( + (event) => event.event === 'mcp_tool_call_received' && event.toolName === 'WebSearch' + ) + ).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_success' && event.providerName === 'DuckDuckGo' + ) + ).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'mcp_session_summary' && + event.calledWebSearch === true && + event.toolCalls === 1 + ) + ).toBe(true); + + rmSync(tempDir, { recursive: true, force: true }); + }); + + it('accepts legacy Content-Length framed requests for compatibility', async () => { + const child = spawn('node', [serverPath], { + env: { + ...process.env, + CCS_PROFILE_TYPE: 'account', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '1', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + + try { + const responsesPromise = collectResponses(child, 2); + child.stdin.write( + encodeLegacyMessage({ + jsonrpc: '2.0', + id: 1, + method: 'initialize', + params: { + protocolVersion: '2024-11-05', + capabilities: {}, + clientInfo: { name: 'bun-test', version: '1.0.0' }, + }, + }) + ); + child.stdin.write(encodeLegacyMessage({ jsonrpc: '2.0', id: 2, method: 'tools/list' })); + + const responses = await responsesPromise; + const toolsList = responses.find((message) => message.id === 2); + expect(toolsList?.result).toEqual({ tools: [] }); + } finally { + child.kill(); + } + }); +}); diff --git a/tests/unit/hooks/websearch-transformer.test.ts b/tests/unit/hooks/websearch-transformer.test.ts index e731e197..43ebde85 100644 --- a/tests/unit/hooks/websearch-transformer.test.ts +++ b/tests/unit/hooks/websearch-transformer.test.ts @@ -1,5 +1,12 @@ import { describe, expect, it } from 'bun:test'; -import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; import { spawnSync } from 'node:child_process'; @@ -29,11 +36,18 @@ const hook = require('../../../lib/hooks/websearch-transformer.cjs') as { url: string; description: string; }>; + classifyProviderFailure: (result: { + error?: string; + retryAfterSec?: number | null; + statusCode?: number | null; + success?: boolean; + }) => Record; formatStructuredSearchResults: ( query: string, providerName: string, results: Array<{ title: string; url: string; description: string }> ) => string; + parseRetryAfterSeconds: (rawValue: string) => number | null; }; function runHookWithMockedFetch(mode: 'success' | 'failure') { @@ -76,6 +90,42 @@ function runHookWithMockedFetch(mode: 'success' | 'failure') { } describe('websearch-transformer hook helpers', () => { + it('parses Retry-After seconds and HTTP dates', () => { + expect(hook.parseRetryAfterSeconds('2')).toBe(2); + expect( + hook.parseRetryAfterSeconds(new Date(Date.now() + 2000).toUTCString()) + ).toBeGreaterThanOrEqual(1); + expect(hook.parseRetryAfterSeconds('invalid')).toBeNull(); + }); + + it('classifies quota exhaustion and short rate limits into the correct provider policy', () => { + expect( + hook.classifyProviderFailure({ + success: false, + statusCode: 429, + error: 'Exa returned 429: quota exceeded for current plan', + }) + ).toMatchObject({ + kind: 'cooldown', + reason: 'quota_exhausted', + cooldownSec: 900, + }); + + expect( + hook.classifyProviderFailure({ + success: false, + statusCode: 429, + retryAfterSec: 2, + error: 'Brave Search returned 429: rate limit exceeded', + }) + ).toMatchObject({ + kind: 'retry', + reason: 'rate_limited_short_backoff', + delayMs: 2000, + retryAfterSec: 2, + }); + }); + it('extracts DuckDuckGo results and unwraps uddg redirect URLs', () => { const html = ` Example title @@ -193,4 +243,447 @@ describe('websearch-transformer hook helpers', () => { ); expect(output).not.toHaveProperty('additionalContext'); }); + + it('writes opt-in trace records with redacted query fingerprints', () => { + const tempDir = mkdtempSync(join(tmpdir(), 'websearch-hook-trace-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const ccsHome = join(tempDir, 'home'); + const tracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + const html = ` + Example title + Example snippet + `.trim(); + + writeFileSync( + preloadPath, + `global.fetch = async () => ({ ok: true, text: async () => ${JSON.stringify(html)} });\n`, + 'utf8' + ); + + try { + const result = spawnSync('node', ['-r', preloadPath, hookPath], { + encoding: 'utf8', + input: JSON.stringify({ + tool_name: 'WebSearch', + tool_input: { query: 'btc price' }, + }), + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'hook-trace-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '0', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + }, + }); + + expect(result.status).toBe(0); + + const traceContents = readFileSync(tracePath, 'utf8'); + expect(traceContents).not.toContain('btc price'); + + const traceEvents = traceContents + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + + expect(traceEvents.some((event) => event.event === 'websearch_hook_invoked')).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_attempt' && event.providerName === 'DuckDuckGo' + ) + ).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_success' && event.providerName === 'DuckDuckGo' + ) + ).toBe(true); + const fingerprintEvent = traceEvents.find( + (event) => event.event === 'websearch_hook_invoked' + ) as { queryHash?: string; queryLength?: number } | undefined; + expect(fingerprintEvent?.queryHash).toBeString(); + expect(fingerprintEvent?.queryLength).toBe(9); + } finally { + rmSync(tempDir, { force: true, recursive: true }); + } + }); + + it('falls back to the default trace file when CCS_WEBSEARCH_TRACE_FILE points outside safe paths', () => { + const tempDir = mkdtempSync(join(tmpdir(), 'websearch-hook-trace-safe-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const ccsHome = join(tempDir, 'home'); + const fallbackTracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + const disallowedTracePath = join(process.cwd(), '.tmp-websearch-trace-unsafe.jsonl'); + const html = ` + Example title + Example snippet + `.trim(); + + writeFileSync( + preloadPath, + `global.fetch = async () => ({ ok: true, text: async () => ${JSON.stringify(html)} });\n`, + 'utf8' + ); + + try { + rmSync(disallowedTracePath, { force: true }); + const result = spawnSync('node', ['-r', preloadPath, hookPath], { + encoding: 'utf8', + input: JSON.stringify({ + tool_name: 'WebSearch', + tool_input: { query: 'btc price' }, + }), + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_FILE: disallowedTracePath, + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'hook-trace-safe-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '0', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + }, + }); + + expect(result.status).toBe(0); + expect(existsSync(disallowedTracePath)).toBe(false); + expect(existsSync(fallbackTracePath)).toBe(true); + } finally { + rmSync(disallowedTracePath, { force: true }); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + it('applies provider cooldown on quota exhaustion and falls back to the next backend', () => { + const tempDir = mkdtempSync(join(tmpdir(), 'websearch-hook-quota-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const requestLogPath = join(tempDir, 'requests.json'); + const ccsHome = join(tempDir, 'home'); + const statePath = join(ccsHome, '.ccs', 'cache', 'websearch-provider-state.json'); + const tracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + const html = ` + Fallback title + Fallback snippet + `.trim(); + + writeFileSync( + preloadPath, + ` +const fs = require('fs'); +const requestLogPath = ${JSON.stringify(requestLogPath)}; +const html = ${JSON.stringify(html)}; +function record(url) { + const requests = fs.existsSync(requestLogPath) + ? JSON.parse(fs.readFileSync(requestLogPath, 'utf8')) + : []; + requests.push(String(url)); + fs.writeFileSync(requestLogPath, JSON.stringify(requests), 'utf8'); +} +global.fetch = async (url) => { + const resolvedUrl = String(url); + record(resolvedUrl); + if (resolvedUrl.includes('api.exa.ai')) { + return { + ok: false, + status: 429, + headers: { get: () => null }, + text: async () => 'quota exceeded for current plan', + }; + } + return { + ok: true, + headers: { get: () => null }, + text: async () => html, + }; +}; + `.trimStart(), + 'utf8' + ); + + try { + const result = spawnSync('node', ['-r', preloadPath, hookPath], { + encoding: 'utf8', + input: JSON.stringify({ + tool_name: 'WebSearch', + tool_input: { query: 'btc price' }, + }), + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'quota-fallback-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '1', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + EXA_API_KEY: 'exa-test-key', + }, + }); + + expect(result.status).toBe(0); + const output = JSON.parse(result.stdout.trim()) as HookOutput; + expect(output.hookSpecificOutput.additionalContext).toContain('Provider: DuckDuckGo'); + + const providerState = JSON.parse(readFileSync(statePath, 'utf8')) as { + cooldowns?: Record; + }; + expect(providerState.cooldowns?.exa?.reason).toBe('quota_exhausted'); + expect(providerState.cooldowns?.exa?.until).toBeGreaterThan(Date.now()); + + const traceEvents = readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_cooldown_applied' && + event.providerId === 'exa' && + event.reason === 'quota_exhausted' + ) + ).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_success' && + event.providerId === 'duckduckgo' + ) + ).toBe(true); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + it('skips providers that are already cooling down on later WebSearch calls', () => { + const tempDir = mkdtempSync(join(tmpdir(), 'websearch-hook-cooldown-skip-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const requestLogPath = join(tempDir, 'requests.json'); + const ccsHome = join(tempDir, 'home'); + const statePath = join(ccsHome, '.ccs', 'cache', 'websearch-provider-state.json'); + const tracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + const html = ` + Cooldown title + Cooldown snippet + `.trim(); + + mkdirSync(join(ccsHome, '.ccs', 'cache'), { recursive: true }); + writeFileSync( + statePath, + JSON.stringify( + { + cooldowns: { + exa: { + until: Date.now() + 10 * 60 * 1000, + reason: 'quota_exhausted', + }, + }, + }, + null, + 2 + ), + 'utf8' + ); + writeFileSync( + preloadPath, + ` +const fs = require('fs'); +const requestLogPath = ${JSON.stringify(requestLogPath)}; +const html = ${JSON.stringify(html)}; +function record(url) { + const requests = fs.existsSync(requestLogPath) + ? JSON.parse(fs.readFileSync(requestLogPath, 'utf8')) + : []; + requests.push(String(url)); + fs.writeFileSync(requestLogPath, JSON.stringify(requests), 'utf8'); +} +global.fetch = async (url) => { + record(url); + return { + ok: true, + headers: { get: () => null }, + text: async () => html, + }; +}; + `.trimStart(), + 'utf8' + ); + + try { + const result = spawnSync('node', ['-r', preloadPath, hookPath], { + encoding: 'utf8', + input: JSON.stringify({ + tool_name: 'WebSearch', + tool_input: { query: 'btc price' }, + }), + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'cooldown-skip-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '1', + CCS_WEBSEARCH_EXA: '1', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + EXA_API_KEY: 'exa-test-key', + }, + }); + + expect(result.status).toBe(0); + const output = JSON.parse(result.stdout.trim()) as HookOutput; + expect(output.hookSpecificOutput.additionalContext).toContain('Provider: DuckDuckGo'); + + const requests = JSON.parse(readFileSync(requestLogPath, 'utf8')) as string[]; + expect(requests.some((url) => url.includes('api.exa.ai'))).toBe(false); + + const traceEvents = readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_cooldown_skip' && + event.providerId === 'exa' && + event.cooldownReason === 'quota_exhausted' + ) + ).toBe(true); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + it('retries transient backend failures once before succeeding', () => { + const tempDir = mkdtempSync(join(tmpdir(), 'websearch-hook-retry-')); + const preloadPath = join(tempDir, 'mock-fetch.cjs'); + const requestLogPath = join(tempDir, 'requests.json'); + const ccsHome = join(tempDir, 'home'); + const tracePath = join(ccsHome, '.ccs', 'logs', 'websearch-trace.jsonl'); + + writeFileSync( + preloadPath, + ` +const fs = require('fs'); +const requestLogPath = ${JSON.stringify(requestLogPath)}; +let exaAttempts = 0; +function record(url) { + const requests = fs.existsSync(requestLogPath) + ? JSON.parse(fs.readFileSync(requestLogPath, 'utf8')) + : []; + requests.push(String(url)); + fs.writeFileSync(requestLogPath, JSON.stringify(requests), 'utf8'); +} +global.fetch = async (url) => { + const resolvedUrl = String(url); + record(resolvedUrl); + exaAttempts += 1; + if (exaAttempts === 1) { + return { + ok: false, + status: 503, + headers: { get: () => null }, + text: async () => 'service unavailable', + }; + } + return { + ok: true, + headers: { get: () => null }, + json: async () => ({ + results: [ + { + title: 'Exa title', + url: 'https://example.com/exa', + text: 'Exa snippet', + }, + ], + }), + }; +}; + `.trimStart(), + 'utf8' + ); + + try { + const result = spawnSync('node', ['-r', preloadPath, hookPath], { + encoding: 'utf8', + input: JSON.stringify({ + tool_name: 'WebSearch', + tool_input: { query: 'btc price' }, + }), + env: { + ...process.env, + CCS_HOME: ccsHome, + CCS_WEBSEARCH_TRACE: '1', + CCS_WEBSEARCH_TRACE_LAUNCH_ID: 'transient-retry-test', + CCS_WEBSEARCH_TRACE_LAUNCHER: 'unit-test', + CCS_WEBSEARCH_ENABLED: '1', + CCS_WEBSEARCH_SKIP: '0', + CCS_WEBSEARCH_BRAVE: '0', + CCS_WEBSEARCH_DUCKDUCKGO: '0', + CCS_WEBSEARCH_EXA: '1', + CCS_WEBSEARCH_GEMINI: '0', + CCS_WEBSEARCH_GROK: '0', + CCS_WEBSEARCH_OPENCODE: '0', + CCS_WEBSEARCH_TAVILY: '0', + EXA_API_KEY: 'exa-test-key', + }, + }); + + expect(result.status).toBe(0); + const output = JSON.parse(result.stdout.trim()) as HookOutput; + expect(output.hookSpecificOutput.additionalContext).toContain('Provider: Exa'); + + const requests = JSON.parse(readFileSync(requestLogPath, 'utf8')) as string[]; + expect(requests.filter((url) => url.includes('api.exa.ai'))).toHaveLength(2); + + const traceEvents = readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_retry_scheduled' && + event.providerId === 'exa' && + event.reason === 'transient_failure' + ) + ).toBe(true); + expect( + traceEvents.some( + (event) => + event.event === 'websearch_provider_success' && event.providerId === 'exa' + ) + ).toBe(true); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } + }); }); diff --git a/tests/unit/instance-manager-mcp-sync.test.ts b/tests/unit/instance-manager-mcp-sync.test.ts index 8eec45a7..27d5dbd0 100644 --- a/tests/unit/instance-manager-mcp-sync.test.ts +++ b/tests/unit/instance-manager-mcp-sync.test.ts @@ -144,6 +144,68 @@ describe('InstanceManager MCP sync', () => { }); }); + it('repairs managed ccs-websearch entries from global config while preserving other instance MCP overrides', () => { + fs.writeFileSync( + path.join(tempRoot, '.claude.json'), + JSON.stringify( + { + mcpServers: { + 'ccs-websearch': { + type: 'stdio', + command: 'node', + args: ['/global/server.cjs'], + env: {}, + }, + shared: { command: 'global-shared' }, + }, + }, + null, + 2 + ), + 'utf8' + ); + + const manager = new InstanceManager(); + const instancePath = manager.getInstancePath('work'); + fs.mkdirSync(instancePath, { recursive: true }); + fs.writeFileSync( + path.join(instancePath, '.claude.json'), + JSON.stringify( + { + mcpServers: { + 'ccs-websearch': { + type: 'stdio', + command: 'node', + args: ['/old/server.cjs'], + env: {}, + }, + shared: { command: 'instance-shared' }, + instanceOnly: { command: 'instance-only' }, + }, + }, + null, + 2 + ), + 'utf8' + ); + + expect(manager.syncMcpServers(instancePath)).toBe(true); + + const instanceContent = readJson(path.join(instancePath, '.claude.json')) as { + mcpServers: Record; + }; + expect(instanceContent.mcpServers).toEqual({ + 'ccs-websearch': { + type: 'stdio', + command: 'node', + args: ['/global/server.cjs'], + env: {}, + }, + shared: { command: 'instance-shared' }, + instanceOnly: { command: 'instance-only' }, + }); + }); + it('logs warning when global MCP sync fails', () => { fs.writeFileSync(path.join(tempRoot, '.claude.json'), '{invalid-json', 'utf8'); const warnSpy = spyOn(console, 'warn').mockImplementation(() => {}); diff --git a/tests/unit/targets/settings-profile-websearch-launch.test.ts b/tests/unit/targets/settings-profile-websearch-launch.test.ts index 3753a218..6064575a 100644 --- a/tests/unit/targets/settings-profile-websearch-launch.test.ts +++ b/tests/unit/targets/settings-profile-websearch-launch.test.ts @@ -4,6 +4,8 @@ import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; +const STEERING_PROMPT_SNIPPET = 'prefer the CCS MCP tool WebSearch instead of Bash/curl/http fetches'; + interface RunResult { status: number | null; stdout: string; @@ -25,6 +27,15 @@ function runCcs(args: string[], env: NodeJS.ProcessEnv): RunResult { }; } +function readTraceEvents(tracePath: string): Array> { + return fs + .readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .filter((line) => line.length > 0) + .map((line) => JSON.parse(line) as Record); +} + describe('settings profile WebSearch launch', () => { let tmpHome = ''; let ccsDir = ''; @@ -92,7 +103,7 @@ exit 0 fs.rmSync(tmpHome, { recursive: true, force: true }); }); - it('fails before Claude launch when an enabled WebSearch hook cannot be prepared', () => { + it('fails before Claude launch when the local WebSearch tool runtime cannot be prepared', () => { if (process.platform === 'win32') return; fs.writeFileSync(path.join(ccsDir, 'hooks'), 'not-a-directory', 'utf8'); @@ -100,7 +111,19 @@ exit 0 const result = runCcs(['glm', 'smoke'], baseEnv); expect(result.status).toBe(1); - expect(result.stderr).toContain('could not prepare the profile hook for "glm"'); + expect(result.stderr).toContain('could not prepare the local WebSearch tool'); + expect(fs.existsSync(claudeArgsLogPath)).toBe(false); + }); + + it('fails before delegated headless launch when the local WebSearch tool runtime cannot be prepared', () => { + if (process.platform === 'win32') return; + + fs.writeFileSync(path.join(ccsDir, 'hooks'), 'not-a-directory', 'utf8'); + + const result = runCcs(['glm', '-p', 'smoke'], baseEnv); + + expect(result.status).toBe(1); + expect(result.stderr).toContain('could not prepare the local WebSearch tool'); expect(fs.existsSync(claudeArgsLogPath)).toBe(false); }); @@ -117,7 +140,42 @@ exit 0 const result = runCcs(['glm', 'smoke'], baseEnv); expect(result.status).toBe(0); - expect(result.stderr).not.toContain('could not prepare the profile hook for "glm"'); + expect(result.stderr).not.toContain('could not prepare the local WebSearch tool'); expect(fs.existsSync(claudeArgsLogPath)).toBe(true); + const launchedArgs = fs.readFileSync(claudeArgsLogPath, 'utf8'); + expect(launchedArgs).toContain('--disallowedTools'); + expect(launchedArgs).toContain('WebSearch'); + expect(launchedArgs).toContain('--append-system-prompt'); + expect(launchedArgs).toContain(STEERING_PROMPT_SNIPPET); + }); + + it('writes a source-side launch trace for settings profiles when tracing is enabled', () => { + if (process.platform === 'win32') return; + + const tracePath = path.join(ccsDir, 'logs', 'websearch-trace.jsonl'); + const result = runCcs(['glm', 'smoke'], { + ...baseEnv, + CCS_WEBSEARCH_TRACE: '1', + }); + + expect(result.status).toBe(0); + expect(fs.existsSync(tracePath)).toBe(true); + + const traceEvents = readTraceEvents(tracePath); + const launchEvent = traceEvents.find( + (event) => event.event === 'ccs_websearch_launch' + ) as + | { + launcher?: string; + nativeWebSearchDisallowed?: boolean; + steeringPromptApplied?: boolean; + settingsPath?: string; + } + | undefined; + + expect(launchEvent?.launcher).toBe('ccs.settings-profile'); + expect(launchEvent?.nativeWebSearchDisallowed).toBe(true); + expect(launchEvent?.steeringPromptApplied).toBe(true); + expect(launchEvent?.settingsPath).toBe(settingsPath); }); }); diff --git a/tests/unit/utils/claudecode-env-stripping.test.ts b/tests/unit/utils/claudecode-env-stripping.test.ts index 0d10ebe0..acd83eb7 100644 --- a/tests/unit/utils/claudecode-env-stripping.test.ts +++ b/tests/unit/utils/claudecode-env-stripping.test.ts @@ -21,12 +21,14 @@ type SpawnCall = { options: Record | undefined; }; +const STEERING_PROMPT_SNIPPET = 'prefer the CCS MCP tool WebSearch instead of Bash/curl/http fetches'; const spawnCalls: SpawnCall[] = []; const originalPlatform = process.platform; let baselineSigintListeners: Array<(...args: unknown[]) => void> = []; let baselineSigtermListeners: Array<(...args: unknown[]) => void> = []; let baselineSighupListeners: Array<(...args: unknown[]) => void> = []; let originalCcsHome: string | undefined; +let originalCcsClaudePath: string | undefined; let originalDisableAutoUpdater: string | undefined; const realSpawn = childProcess.spawn.bind(childProcess); const realSpawnSync = childProcess.spawnSync.bind(childProcess); @@ -150,6 +152,7 @@ describe('CLAUDECODE environment stripping', () => { spawnCalls.length = 0; process.env.CCS_QUIET = '1'; originalCcsHome = process.env.CCS_HOME; + originalCcsClaudePath = process.env.CCS_CLAUDE_PATH; originalDisableAutoUpdater = process.env.DISABLE_AUTOUPDATER; delete process.env.DISABLE_AUTOUPDATER; baselineSigintListeners = process.listeners('SIGINT'); @@ -162,8 +165,11 @@ describe('CLAUDECODE environment stripping', () => { delete process.env.CLAUDECODE; delete process.env.claudecode; delete process.env.CCS_QUIET; + delete process.env.CCS_WEBSEARCH_TRACE; if (originalCcsHome !== undefined) process.env.CCS_HOME = originalCcsHome; else delete process.env.CCS_HOME; + if (originalCcsClaudePath !== undefined) process.env.CCS_CLAUDE_PATH = originalCcsClaudePath; + else delete process.env.CCS_CLAUDE_PATH; if (originalDisableAutoUpdater !== undefined) { process.env.DISABLE_AUTOUPDATER = originalDisableAutoUpdater; } else { @@ -325,4 +331,76 @@ describe('CLAUDECODE environment stripping', () => { expect(Object.keys(env).map((k) => k.toUpperCase())).not.toContain('CLAUDECODE'); expect(env.DISABLE_AUTOUPDATER).toBe('1'); }); + + it('headless executor adds third-party WebSearch steering args and env', async () => { + writeConfigWithAutoUpdatePreference(false); + const ccsDir = path.join(process.env.CCS_HOME as string, '.ccs'); + fs.writeFileSync(path.join(ccsDir, 'glm.settings.json'), '{}\n', 'utf8'); + const projectDir = path.join(ccsDir, 'project'); + fs.mkdirSync(path.join(projectDir, '.claude'), { recursive: true }); + fs.writeFileSync( + path.join(projectDir, '.claude', 'settings.local.json'), + JSON.stringify( + { + permissions: { + deny: ['Bash', 'WebFetch'], + }, + }, + null, + 2 + ) + '\n', + 'utf8' + ); + process.env.CCS_CLAUDE_PATH = 'claude'; + + const result = await HeadlessExecutor.execute('glm', 'latest AI chip news', { + cwd: projectDir, + permissionMode: 'default', + timeout: 1000, + }); + + expect(result.success).toBe(true); + expect(spawnCalls.length).toBeGreaterThan(0); + const launch = spawnCalls[0]; + expect(launch.args).toContain('--disallowedTools'); + const disallowedToolsIndex = launch.args.indexOf('--disallowedTools'); + expect(disallowedToolsIndex).toBeGreaterThan(-1); + expect(launch.args[disallowedToolsIndex + 1]).toBe('Bash,WebFetch,WebSearch'); + expect(launch.args).toContain('--append-system-prompt'); + expect(launch.args.join(' ')).toContain(STEERING_PROMPT_SNIPPET); + const env = launch.options?.env as NodeJS.ProcessEnv; + expect(env.CCS_PROFILE_TYPE).toBe('settings'); + expect(env.CCS_WEBSEARCH_ENABLED || env.CCS_WEBSEARCH_SKIP).toBeDefined(); + const claudeUserConfig = JSON.parse( + fs.readFileSync(path.join(process.env.CCS_HOME as string, '.claude.json'), 'utf8') + ) as { + mcpServers?: Record; + }; + expect(claudeUserConfig.mcpServers?.['ccs-websearch']).toEqual({ + type: 'stdio', + command: 'node', + args: [path.join(ccsDir, 'mcp', 'ccs-websearch-server.cjs')], + env: {}, + }); + }); + + it('headless executor propagates a WebSearch trace launch id when tracing is enabled', async () => { + writeConfigWithAutoUpdatePreference(false); + const ccsDir = path.join(process.env.CCS_HOME as string, '.ccs'); + fs.writeFileSync(path.join(ccsDir, 'glm.settings.json'), '{}\n', 'utf8'); + process.env.CCS_CLAUDE_PATH = 'claude'; + process.env.CCS_WEBSEARCH_TRACE = '1'; + + const result = await HeadlessExecutor.execute('glm', 'latest AI chip news', { + permissionMode: 'default', + timeout: 1000, + }); + + expect(result.success).toBe(true); + expect(spawnCalls.length).toBeGreaterThan(0); + const env = spawnCalls[0].options?.env as NodeJS.ProcessEnv; + expect(env.CCS_WEBSEARCH_TRACE).toBe('1'); + expect(env.CCS_WEBSEARCH_TRACE_LAUNCH_ID).toBeString(); + expect(env.CCS_WEBSEARCH_TRACE_LAUNCHER).toBe('delegation.headless-executor'); + }); }); diff --git a/tests/unit/utils/websearch/claude-tool-args.test.ts b/tests/unit/utils/websearch/claude-tool-args.test.ts new file mode 100644 index 00000000..9f902b80 --- /dev/null +++ b/tests/unit/utils/websearch/claude-tool-args.test.ts @@ -0,0 +1,132 @@ +import { describe, expect, it } from 'bun:test'; +import { appendThirdPartyWebSearchToolArgs } from '../../../../src/utils/websearch/claude-tool-args'; + +const STEERING_PROMPT = + 'For web lookup or current-information requests, prefer the CCS MCP tool WebSearch instead of Bash/curl/http fetches. If the user explicitly wants shell commands, or WebSearch is unavailable or fails, you may fall back to Bash/network tools.'; + +describe('appendThirdPartyWebSearchToolArgs', () => { + it('appends native WebSearch suppression and steering prompt when no tool flags are present', () => { + expect(appendThirdPartyWebSearchToolArgs(['smoke'])).toEqual([ + 'smoke', + '--disallowedTools', + 'WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('does not append duplicate suppression or steering prompt when both are already present', () => { + expect( + appendThirdPartyWebSearchToolArgs([ + 'smoke', + '--disallowedTools', + 'WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]) + ).toEqual([ + 'smoke', + '--disallowedTools', + 'WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('detects comma-separated disallowed tool values', () => { + expect( + appendThirdPartyWebSearchToolArgs(['smoke', '--disallowedTools=Read,WebSearch']) + ).toEqual([ + 'smoke', + '--disallowedTools=Read,WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('merges WebSearch into an existing space-separated disallowed tool flag', () => { + expect(appendThirdPartyWebSearchToolArgs(['smoke', '--disallowedTools', 'Read'])).toEqual([ + 'smoke', + '--disallowedTools', + 'Read,WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('merges WebSearch into an existing equals-form disallowed tool flag', () => { + expect(appendThirdPartyWebSearchToolArgs(['smoke', '--disallowedTools=Read'])).toEqual([ + 'smoke', + '--disallowedTools=Read,WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('preserves user-supplied append-system-prompt values and adds the CCS steering hint once', () => { + expect( + appendThirdPartyWebSearchToolArgs([ + 'smoke', + '--append-system-prompt', + 'User-provided instruction', + ]) + ).toEqual([ + 'smoke', + '--append-system-prompt', + 'User-provided instruction', + '--disallowedTools', + 'WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('does not duplicate the steering prompt when it already exists in equals form', () => { + expect( + appendThirdPartyWebSearchToolArgs([ + 'smoke', + '--disallowedTools', + 'WebSearch', + `--append-system-prompt=${STEERING_PROMPT}`, + ]) + ).toEqual([ + 'smoke', + '--disallowedTools', + 'WebSearch', + `--append-system-prompt=${STEERING_PROMPT}`, + ]); + }); + + it('does not consume positional args after a disallowed-tools flag value', () => { + expect( + appendThirdPartyWebSearchToolArgs(['--disallowedTools', 'Read', 'latest AI news']) + ).toEqual([ + '--disallowedTools', + 'Read,WebSearch', + 'latest AI news', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); + + it('injects synthetic flags before an end-of-options marker', () => { + expect(appendThirdPartyWebSearchToolArgs(['--', 'latest AI news'])).toEqual([ + '--disallowedTools', + 'WebSearch', + '--append-system-prompt', + STEERING_PROMPT, + '--', + 'latest AI news', + ]); + }); + + it('inserts the WebSearch disallow value when the flag is present without one', () => { + expect(appendThirdPartyWebSearchToolArgs(['--disallowedTools', '--verbose'])).toEqual([ + '--disallowedTools', + 'WebSearch', + '--verbose', + '--append-system-prompt', + STEERING_PROMPT, + ]); + }); +}); diff --git a/tests/unit/utils/websearch/mcp-installer.test.ts b/tests/unit/utils/websearch/mcp-installer.test.ts new file mode 100644 index 00000000..5fb07d34 --- /dev/null +++ b/tests/unit/utils/websearch/mcp-installer.test.ts @@ -0,0 +1,213 @@ +import { afterEach, describe, expect, it, mock, spyOn } from 'bun:test'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { getHookPath } from '../../../../src/utils/websearch/hook-config'; +import { + ensureWebSearchMcp, + getWebSearchMcpServerName, + getWebSearchMcpServerPath, + uninstallWebSearchMcp, +} from '../../../../src/utils/websearch/mcp-installer'; + +describe('ensureWebSearchMcp', () => { + let tempHome: string | undefined; + let originalCcsHome: string | undefined; + + function setupTempHome(): string { + tempHome = fs.mkdtempSync(path.join(os.tmpdir(), 'ccs-websearch-mcp-')); + originalCcsHome = process.env.CCS_HOME; + process.env.CCS_HOME = tempHome; + return tempHome; + } + + function getCcsDir(): string { + if (!tempHome) { + throw new Error('tempHome not initialized'); + } + return path.join(tempHome, '.ccs'); + } + + function writeEnabledConfig(): void { + const ccsDir = getCcsDir(); + fs.mkdirSync(ccsDir, { recursive: true }); + fs.writeFileSync( + path.join(ccsDir, 'config.yaml'), + ['version: 12', 'websearch:', ' enabled: true', ' providers:', ' duckduckgo:', ' enabled: true', ''].join('\n'), + 'utf8' + ); + } + + function getManagedConfig() { + return { + type: 'stdio', + command: 'node', + args: [getWebSearchMcpServerPath()], + env: {}, + }; + } + + afterEach(() => { + mock.restore(); + + if (originalCcsHome !== undefined) { + process.env.CCS_HOME = originalCcsHome; + } else { + delete process.env.CCS_HOME; + } + + if (tempHome && fs.existsSync(tempHome)) { + fs.rmSync(tempHome, { recursive: true, force: true }); + } + + tempHome = undefined; + originalCcsHome = undefined; + }); + + it('installs the MCP server and preserves existing user mcpServers entries', () => { + setupTempHome(); + writeEnabledConfig(); + + const claudeUserConfigPath = path.join(tempHome as string, '.claude.json'); + fs.writeFileSync( + claudeUserConfigPath, + JSON.stringify( + { + mcpServers: { + existing: { command: 'uvx', args: ['some-server'] }, + }, + }, + null, + 2 + ) + '\n', + 'utf8' + ); + + expect(ensureWebSearchMcp()).toBe(true); + expect(fs.existsSync(getHookPath())).toBe(true); + expect(fs.existsSync(getWebSearchMcpServerPath())).toBe(true); + + const config = JSON.parse(fs.readFileSync(claudeUserConfigPath, 'utf8')) as { + mcpServers: Record; + }; + + expect(config.mcpServers.existing).toEqual({ command: 'uvx', args: ['some-server'] }); + expect(config.mcpServers[getWebSearchMcpServerName()]).toEqual(getManagedConfig()); + }); + + it('preserves the existing ~/.claude.json permissions when provisioning WebSearch MCP', () => { + setupTempHome(); + writeEnabledConfig(); + + const claudeUserConfigPath = path.join(tempHome as string, '.claude.json'); + fs.writeFileSync(claudeUserConfigPath, JSON.stringify({ existing: true }, null, 2) + '\n', { + encoding: 'utf8', + mode: 0o600, + }); + fs.chmodSync(claudeUserConfigPath, 0o600); + + expect(ensureWebSearchMcp()).toBe(true); + expect(fs.statSync(claudeUserConfigPath).mode & 0o777).toBe(0o600); + }); + + it('writes new ~/.claude.json with 0600 permissions', () => { + setupTempHome(); + writeEnabledConfig(); + + const claudeUserConfigPath = path.join(tempHome as string, '.claude.json'); + + expect(ensureWebSearchMcp()).toBe(true); + expect(fs.statSync(claudeUserConfigPath).mode & 0o777).toBe(0o600); + }); + + it('returns false and preserves malformed ~/.claude.json', () => { + setupTempHome(); + writeEnabledConfig(); + + const claudeUserConfigPath = path.join(tempHome as string, '.claude.json'); + fs.writeFileSync(claudeUserConfigPath, '{ invalid json', 'utf8'); + + expect(ensureWebSearchMcp()).toBe(false); + expect(fs.readFileSync(claudeUserConfigPath, 'utf8')).toBe('{ invalid json'); + }); + + it('removes the managed MCP runtime while preserving unrelated server entries', () => { + setupTempHome(); + writeEnabledConfig(); + + const claudeUserConfigPath = path.join(tempHome as string, '.claude.json'); + fs.writeFileSync( + claudeUserConfigPath, + JSON.stringify( + { + mcpServers: { + existing: { command: 'uvx', args: ['some-server'] }, + }, + }, + null, + 2 + ) + '\n', + 'utf8' + ); + + expect(ensureWebSearchMcp()).toBe(true); + + const instancePath = path.join(tempHome as string, '.ccs', 'instances', 'work'); + fs.mkdirSync(instancePath, { recursive: true }); + fs.writeFileSync( + path.join(instancePath, '.claude.json'), + JSON.stringify( + { + mcpServers: { + existing: { command: 'uvx', args: ['instance-server'] }, + [getWebSearchMcpServerName()]: { command: 'node', args: ['/tmp/override.cjs'] }, + }, + otherKey: 'keep-me', + }, + null, + 2 + ) + '\n', + 'utf8' + ); + + expect(uninstallWebSearchMcp()).toBe(true); + expect(fs.existsSync(getWebSearchMcpServerPath())).toBe(false); + + const globalConfig = JSON.parse(fs.readFileSync(claudeUserConfigPath, 'utf8')) as { + mcpServers: Record; + }; + expect(globalConfig.mcpServers).toEqual({ + existing: { command: 'uvx', args: ['some-server'] }, + }); + + const instanceConfig = JSON.parse( + fs.readFileSync(path.join(instancePath, '.claude.json'), 'utf8') + ) as { + otherKey: string; + mcpServers: Record; + }; + expect(instanceConfig.otherKey).toBe('keep-me'); + expect(instanceConfig.mcpServers).toEqual({ + existing: { command: 'uvx', args: ['instance-server'] }, + }); + }); + + it('falls back to copy-overwrite when rename is blocked during MCP server install', () => { + setupTempHome(); + writeEnabledConfig(); + + const realRenameSync = fs.renameSync; + const renameSpy = spyOn(fs, 'renameSync').mockImplementation((oldPath, newPath) => { + if (String(newPath) === getWebSearchMcpServerPath()) { + const error = new Error('busy') as NodeJS.ErrnoException; + error.code = 'EPERM'; + throw error; + } + return realRenameSync(oldPath, newPath); + }); + + expect(ensureWebSearchMcp()).toBe(true); + expect(renameSpy).toHaveBeenCalled(); + expect(fs.existsSync(getWebSearchMcpServerPath())).toBe(true); + }); +}); diff --git a/tests/unit/utils/websearch/status.test.ts b/tests/unit/utils/websearch/status.test.ts index 3bdd3c23..b696278d 100644 --- a/tests/unit/utils/websearch/status.test.ts +++ b/tests/unit/utils/websearch/status.test.ts @@ -1,5 +1,16 @@ -import { describe, expect, it } from 'bun:test'; -import { buildWebSearchReadiness } from '../../../../src/utils/websearch/status'; +import { describe, expect, it, spyOn } from 'bun:test'; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import * as geminiCli from '../../../../src/utils/websearch/gemini-cli'; +import * as grokCli from '../../../../src/utils/websearch/grok-cli'; +import * as opencodeCli from '../../../../src/utils/websearch/opencode-cli'; +import * as providerSecrets from '../../../../src/utils/websearch/provider-secrets'; +import * as unifiedConfigLoader from '../../../../src/config/unified-config-loader'; +import { + buildWebSearchReadiness, + getWebSearchCliProviders, +} from '../../../../src/utils/websearch/status'; import type { WebSearchCliInfo } from '../../../../src/utils/websearch/types'; function provider(overrides: Partial & Pick): WebSearchCliInfo { @@ -81,4 +92,103 @@ describe('websearch readiness', () => { expect(readiness.readiness).toBe('ready'); expect(readiness.message).toContain('Exa'); }); + + it('treats cooled-down providers as temporarily unavailable in readiness status', () => { + const tempHome = mkdtempSync(join(tmpdir(), 'websearch-status-cooldown-')); + const statePath = join(tempHome, '.ccs', 'cache', 'websearch-provider-state.json'); + const originalCcsHome = process.env.CCS_HOME; + + mkdirSync(join(tempHome, '.ccs', 'cache'), { recursive: true }); + writeFileSync( + statePath, + JSON.stringify( + { + cooldowns: { + exa: { + until: Date.now() + 10 * 60 * 1000, + reason: 'quota_exhausted', + }, + }, + }, + null, + 2 + ), + 'utf8' + ); + process.env.CCS_HOME = tempHome; + + const getConfigSpy = spyOn(unifiedConfigLoader, 'getWebSearchConfig').mockReturnValue({ + enabled: true, + providers: { + exa: { enabled: true, max_results: 5 }, + tavily: { enabled: false, max_results: 5 }, + duckduckgo: { enabled: false, max_results: 5 }, + brave: { enabled: false, max_results: 5 }, + gemini: { enabled: false }, + grok: { enabled: false }, + opencode: { enabled: false }, + }, + } as any); + const apiKeySpy = spyOn(providerSecrets, 'getWebSearchApiKeyStates').mockReturnValue({ + exa: { + envVar: 'EXA_API_KEY', + configured: true, + available: true, + source: 'process_env', + }, + tavily: { + envVar: 'TAVILY_API_KEY', + configured: false, + available: false, + source: 'none', + }, + brave: { + envVar: 'BRAVE_API_KEY', + configured: false, + available: false, + source: 'none', + }, + }); + const geminiStatusSpy = spyOn(geminiCli, 'getGeminiCliStatus').mockReturnValue({ + installed: false, + version: null, + } as any); + const geminiAuthSpy = spyOn(geminiCli, 'isGeminiAuthenticated').mockReturnValue(false); + const grokStatusSpy = spyOn(grokCli, 'getGrokCliStatus').mockReturnValue({ + installed: false, + version: null, + } as any); + const opencodeStatusSpy = spyOn(opencodeCli, 'getOpenCodeCliStatus').mockReturnValue({ + installed: false, + version: null, + } as any); + + try { + const providers = getWebSearchCliProviders(); + const exa = providers.find((provider) => provider.id === 'exa'); + + expect(exa?.enabled).toBe(true); + expect(exa?.available).toBe(false); + expect(exa?.detail).toContain('Cooling down'); + expect(exa?.detail).toContain('quota exhaustion'); + + const readiness = buildWebSearchReadiness(true, providers); + expect(readiness.readiness).toBe('needs_setup'); + expect(readiness.message).toContain('Cooling down'); + } finally { + getConfigSpy.mockRestore(); + apiKeySpy.mockRestore(); + geminiStatusSpy.mockRestore(); + geminiAuthSpy.mockRestore(); + grokStatusSpy.mockRestore(); + opencodeStatusSpy.mockRestore(); + + if (originalCcsHome === undefined) { + delete process.env.CCS_HOME; + } else { + process.env.CCS_HOME = originalCcsHome; + } + rmSync(tempHome, { recursive: true, force: true }); + } + }); });