mirror of
https://github.com/tiennm99/ccs.git
synced 2026-10-19 02:13:57 +00:00
docs(operations): refresh runtime contracts
This commit is contained in:
Tam Nhu Tran committed
1 parent
ebe1746459
commit
51e8062995
3 files changed
+244
-280
No files matched your search
+122
-130
@@ -1,96 +1,84 @@
|
||||
# WebSearch Configuration Guide
|
||||
|
||||
Last Updated: 2026-04-11
|
||||
CCS provides a local `WebSearch` MCP tool for managed Claude launches that use a
|
||||
third-party provider. Native Claude sessions keep Anthropic's native search
|
||||
behavior.
|
||||
|
||||
CCS provides automatic web search for third-party profiles that cannot access Anthropic's native WebSearch API.
|
||||
## Launch Contract
|
||||
|
||||
## How WebSearch Works
|
||||
For a third-party Claude launch, CCS:
|
||||
|
||||
### Native Claude Accounts
|
||||
1. suppresses the native `WebSearch` tool because the third-party backend cannot
|
||||
execute it;
|
||||
2. adds a short steering prompt that prefers the CCS MCP `WebSearch` tool;
|
||||
3. when WebSearch is enabled, installs the managed MCP server and adds the
|
||||
`ccs-websearch` entry to the applicable Claude configuration;
|
||||
4. searches enabled and ready providers in deterministic order.
|
||||
|
||||
Native Claude subscription accounts still use Anthropic's server-side WebSearch directly.
|
||||
Enabled launches fail closed if the MCP runtime or configuration cannot be
|
||||
prepared. This prevents a launch where native search is suppressed but the
|
||||
managed replacement is missing. When `websearch.enabled` is `false`, CCS skips
|
||||
MCP provisioning but still suppresses native WebSearch for third-party
|
||||
profiles; the model can use ordinary network or shell tools when allowed.
|
||||
|
||||
### Third-Party Profiles
|
||||
Claude subcommands that reject session flags are passed through without
|
||||
WebSearch argument injection.
|
||||
|
||||
Third-party profiles cannot execute Anthropic's server-side WebSearch because the tool never reaches their backend. CCS now handles that by provisioning a first-class local MCP tool when the managed runtime is available, suppressing native `WebSearch` for those launches, appending a short launch-time steering hint, and running real local search providers directly.
|
||||
Implementation sources:
|
||||
|
||||
## Architecture
|
||||
- [`src/utils/websearch/mcp-installer.ts`](../src/utils/websearch/mcp-installer.ts)
|
||||
- [`src/utils/websearch/claude-tool-args.ts`](../src/utils/websearch/claude-tool-args.ts)
|
||||
- [`src/dispatcher/flows/settings-flow.ts`](../src/dispatcher/flows/settings-flow.ts)
|
||||
- [`lib/mcp/ccs-websearch-server.cjs`](../lib/mcp/ccs-websearch-server.cjs)
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Claude Code CLI │
|
||||
│ │
|
||||
│ Search Request │
|
||||
│ │ │
|
||||
│ ├── Native Claude Account? → Anthropic WebSearch API │
|
||||
│ │ │
|
||||
│ └── Third-party Profile? → native WebSearch disabled │
|
||||
│ │ │
|
||||
│ ├── CCS MCP tool when ready │
|
||||
│ │ ccs-websearch.WebSearch │
|
||||
│ │ │ │
|
||||
│ │ ├── 1. Exa │
|
||||
│ │ ├── 2. Tavily │
|
||||
│ │ ├── 3. Brave │
|
||||
│ │ ├── 4. SearXNG │
|
||||
│ │ ├── 5. DuckDuckGo│
|
||||
│ │ └── 6. Legacy CLI│
|
||||
│ │ fallback │
|
||||
│ │ (Gemini/ │
|
||||
│ │ OpenCode/│
|
||||
│ │ Grok) │
|
||||
│ └── Bash/network fallback │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
## Provider Order, Runtime Eligibility, and Dashboard Status
|
||||
|
||||
## Why This Changed
|
||||
The managed runtime tries eligible providers sequentially:
|
||||
|
||||
The previous design asked another model CLI to perform web search and summarize the answer. A later compatibility path also depended on a denied native-tool hook. Both were brittle:
|
||||
1. Exa
|
||||
2. Tavily
|
||||
3. Brave Search
|
||||
4. SearXNG
|
||||
5. DuckDuckGo
|
||||
6. Antigravity CLI
|
||||
7. Gemini CLI compatibility fallback
|
||||
8. OpenCode
|
||||
9. Grok CLI
|
||||
|
||||
- CLI syntax changed upstream
|
||||
- auth state varied per tool
|
||||
- prompt/tool behavior drifted across releases
|
||||
- hook-shaped denial output produced awkward host UX
|
||||
The first successful provider wins. Temporarily failing providers can enter a
|
||||
bounded cooldown and be skipped on later calls. Runtime attempt eligibility is
|
||||
defined in
|
||||
[`lib/hooks/websearch-transformer.cjs`](../lib/hooks/websearch-transformer.cjs);
|
||||
do not duplicate that list in other architecture docs.
|
||||
|
||||
The new flow matches the `goclaw` model more closely: web search is treated as a first-class deterministic capability, not an LLM-to-LLM workaround or a denied native tool call.
|
||||
The transformer and dashboard answer related but different questions:
|
||||
|
||||
When provisioned, the managed MCP tool is exposed as `ccs-websearch.WebSearch`, not a generic `search` helper. That naming is deliberate: it gives Claude a tool that matches the native `WebSearch` concept more directly, which should reduce cases where the model reaches for ad hoc Bash or `curl` fetches instead.
|
||||
| Provider | Transformer attempts when | Dashboard reports available when |
|
||||
| --- | --- | --- |
|
||||
| Exa | Enabled and `EXA_API_KEY` is present. | Enabled and the key is available through the active process or enabled Global Env. |
|
||||
| Tavily | Enabled and `TAVILY_API_KEY` is present. | Enabled and the key is available through the active process or enabled Global Env. |
|
||||
| Brave Search | Enabled and `BRAVE_API_KEY` is present. | Enabled and the key is available through the active process or enabled Global Env. |
|
||||
| SearXNG | Enabled and a base URL is present. | Enabled with a valid normalized base URL. |
|
||||
| DuckDuckGo | Enabled. | Enabled. |
|
||||
| Antigravity | Enabled and the `agy` executable is available. | Enabled and the CLI is installed. |
|
||||
| Gemini compatibility | Enabled and the `gemini` executable is available. | Enabled, installed, and authenticated. |
|
||||
| OpenCode | Enabled and the `opencode` executable is available. | Enabled and the CLI is installed. |
|
||||
| Grok | Enabled and the `grok` executable is available. | Enabled, installed, and `GROK_API_KEY` is available. |
|
||||
|
||||
CCS also appends a third-party-only `--append-system-prompt` hint telling Claude to prefer that managed `WebSearch` tool for web lookups and current-information requests. This is soft steering only: if the user explicitly asks for shell commands, or the tool is unavailable, Claude can still fall back to Bash/network tools.
|
||||
That shared launch helper applies to normal third-party settings profiles, CLIProxy/Copilot-backed Claude launches, and CCS headless/delegation runs that execute through a settings profile.
|
||||
|
||||
`websearch.enabled: false` disables the managed local runtime, but CCS still suppresses Anthropic's native `WebSearch` on third-party profiles. That native tool cannot be satisfied by Exa, Tavily, Brave, DuckDuckGo, or other non-Anthropic backends, so CCS avoids sending a broken native-tool request and lets Claude fall back to normal shell/network tools instead.
|
||||
|
||||
## Providers
|
||||
|
||||
| Provider | Type | Setup | Default | Notes |
|
||||
|----------|------|-------|---------|-------|
|
||||
| Exa | HTTP API | `EXA_API_KEY` | No | High-quality API search with extracted content |
|
||||
| Tavily | HTTP API | `TAVILY_API_KEY` | No | Agent-oriented search API |
|
||||
| Brave Search | HTTP API | `BRAVE_API_KEY` | No | Cleaner snippets and metadata |
|
||||
| SearXNG | JSON API | `providers.searxng.url` | No | Self-hosted/public SearXNG backend via `/search?format=json` |
|
||||
| DuckDuckGo | HTML fetch | None | Yes | Built-in zero-setup fallback |
|
||||
| Antigravity (agy) | LLM CLI | `curl -fsSL https://antigravity.google/cli/install.sh \| bash` | No | Recommended LLM CLI fallback (Gemini CLI successor) |
|
||||
| Gemini CLI | LLM CLI | Deprecated, use Antigravity (agy) | No | Deprecated. Google retired the gemini CLI on 2026-06-18 |
|
||||
| OpenCode | LLM CLI | `curl -fsSL https://opencode.ai/install \| bash` | No | Optional compatibility fallback |
|
||||
| Grok CLI | LLM CLI | `npm i -g @vibe-kit/grok-cli` + `GROK_API_KEY` | No | Optional compatibility fallback |
|
||||
DuckDuckGo is the default zero-setup provider. API-backed and CLI fallback
|
||||
providers are disabled by default. Dashboard availability and setup guidance
|
||||
are computed separately by
|
||||
[`src/utils/websearch/status.ts`](../src/utils/websearch/status.ts); they do not
|
||||
change the transformer's attempt predicate.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Via Dashboard
|
||||
Use `ccs config` and open `Settings` → `WebSearch`, or edit the `websearch`
|
||||
section in the unified CCS configuration:
|
||||
|
||||
Open `ccs config` → `Settings` → `WebSearch`.
|
||||
|
||||
- Enable Exa, Tavily, Brave, SearXNG, or DuckDuckGo in the backend chain
|
||||
- Configure the SearXNG base URL (for example `https://search.example.com`) when SearXNG is enabled
|
||||
Do not include `/search`, embedded credentials, query parameters, or URL fragments. CCS appends `/search?format=json`.
|
||||
- Set or rotate Exa, Tavily, and Brave API keys directly inside each provider card
|
||||
- Saved keys are persisted in `global_env` and injected at runtime, so readiness updates from the same screen
|
||||
- Review whether any legacy fallback CLIs are still enabled in config
|
||||
|
||||
### Via Config File
|
||||
|
||||
Edit `~/.ccs/config.yaml`:
|
||||
The runtime schema supports Antigravity through `providers.agy`; the current
|
||||
dashboard editor does not expose that provider, so configure it in
|
||||
`config.yaml`.
|
||||
|
||||
```yaml
|
||||
websearch:
|
||||
@@ -129,80 +117,84 @@ websearch:
|
||||
timeout: 55
|
||||
```
|
||||
|
||||
Note: `enabled: false` stops provisioning the managed local `ccs-websearch.WebSearch` runtime. It does not re-enable Anthropic's native `WebSearch` for third-party backends.
|
||||
The schema is
|
||||
[`src/config/schemas/websearch.ts`](../src/config/schemas/websearch.ts), and
|
||||
defaults are in
|
||||
[`src/config/schemas/unified-config.ts`](../src/config/schemas/unified-config.ts).
|
||||
Deprecated top-level WebSearch fields remain load-compatible but must not be
|
||||
used for new configuration.
|
||||
|
||||
## Environment Variables
|
||||
### SearXNG URL
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `EXA_API_KEY` | Enables Exa when `providers.exa.enabled: true` |
|
||||
| `TAVILY_API_KEY` | Enables Tavily when `providers.tavily.enabled: true` |
|
||||
| `BRAVE_API_KEY` | Enables Brave Search when `providers.brave.enabled: true` |
|
||||
| `CCS_WEBSEARCH_SEARXNG_URL` | Runtime URL used when `providers.searxng.enabled: true` |
|
||||
| `CCS_WEBSEARCH_SEARXNG_MAX_RESULTS` | Optional runtime override for SearXNG result count (clamped 1..10) |
|
||||
| `GROK_API_KEY` | Required only for legacy Grok CLI fallback |
|
||||
| `CCS_WEBSEARCH_SKIP` | Disable the CCS local WebSearch runtime for the current process; third-party launches still keep native Anthropic `WebSearch` disabled |
|
||||
| `CCS_DEBUG` | Verbose WebSearch runtime logging |
|
||||
| `CCS_WEBSEARCH_TRACE` | Write opt-in JSONL trace records under `~/.ccs/logs/websearch-trace.jsonl` |
|
||||
| `CCS_WEBSEARCH_TRACE_FILE` | Override the trace file path (must stay inside `~/.ccs/`, your system temp directory, or `/var/log`) |
|
||||
Configure the instance base URL, for example
|
||||
`https://search.example.invalid`. Do not include `/search`, credentials, query
|
||||
parameters, or a URL fragment. CCS normalizes the base and calls the JSON search
|
||||
endpoint.
|
||||
|
||||
## Managed Runtime Files
|
||||
### Dashboard-managed API keys
|
||||
|
||||
- `~/.claude.json` → CCS manages `mcpServers.ccs-websearch`
|
||||
- `~/.ccs/mcp/ccs-websearch-server.cjs` → local MCP server binary
|
||||
- `~/.ccs/hooks/websearch-transformer.cjs` → shared provider runtime plus legacy compatibility fallback
|
||||
The dashboard stores supported provider keys in `global_env`. They are
|
||||
available to WebSearch only when global environment injection is enabled.
|
||||
Shell-provided environment values remain supported. Never place real keys in
|
||||
documentation, tests, or committed configuration.
|
||||
|
||||
## Troubleshooting
|
||||
## Managed Files
|
||||
|
||||
### WebSearch says "Ready (DuckDuckGo)"
|
||||
In the default user layout, CCS manages:
|
||||
|
||||
That is expected. DuckDuckGo is the default zero-setup backend.
|
||||
- `~/.claude.json` → `mcpServers.ccs-websearch`
|
||||
- `~/.ccs/mcp/ccs-websearch-server.cjs`
|
||||
- `~/.ccs/hooks/websearch-transformer.cjs`
|
||||
|
||||
### Exa, Tavily, or Brave is enabled but not ready
|
||||
CCS installation and test paths can differ because configuration helpers honor
|
||||
the active CCS and Claude home locations. Provisioning uses a lock and
|
||||
preserves unrelated MCP server entries. Malformed Claude configuration is not
|
||||
overwritten.
|
||||
|
||||
Set the matching API key in the WebSearch dashboard card, or export it in the environment that launches CCS, then refresh status:
|
||||
## Runtime Environment
|
||||
|
||||
```bash
|
||||
export EXA_API_KEY="your-api-key"
|
||||
# or: export TAVILY_API_KEY="your-api-key"
|
||||
# or: export BRAVE_API_KEY="your-api-key"
|
||||
ccs config
|
||||
```
|
||||
[`src/utils/websearch/hook-env.ts`](../src/utils/websearch/hook-env.ts) converts
|
||||
the resolved configuration into runtime environment values. Provider API keys
|
||||
remain environment inputs; boolean and limit variables describe provider
|
||||
selection.
|
||||
|
||||
If the dashboard says the key is stored but still not ready, check whether `Settings -> Global Env` is disabled. WebSearch reuses that injection path for dashboard-managed keys.
|
||||
Operational overrides:
|
||||
|
||||
### SearXNG is enabled but not ready
|
||||
| Variable | Effect |
|
||||
| --- | --- |
|
||||
| `CCS_DEBUG` | Enables verbose diagnostics and WebSearch trace collection. |
|
||||
| `CCS_WEBSEARCH_TRACE` | Enables WebSearch JSONL trace collection. |
|
||||
| `CCS_WEBSEARCH_TRACE_FILE` | Requests a trace path within an allowed CCS log, system temporary, or `/var/log` boundary. |
|
||||
|
||||
1. Confirm the configured base URL is valid (for example `https://search.example.com`)
|
||||
2. Confirm the instance exposes `GET /search?q=<query>&format=json`
|
||||
3. If the hook reports `SearXNG returned 403: format=json is disabled on this instance`, enable JSON format on that SearXNG deployment or switch to another backend
|
||||
An unsafe trace-file override is ignored. Trace writes are best effort and do
|
||||
not change launch or search results.
|
||||
|
||||
### I still want Gemini/OpenCode/Grok fallback
|
||||
## Diagnostics
|
||||
|
||||
Those providers remain supported, but they are no longer the primary path. Enable them explicitly in `config.yaml` if you want them as last-resort fallback.
|
||||
By default, trace records are written under
|
||||
`~/.ccs/logs/websearch-trace.jsonl`. They correlate launch preparation, MCP
|
||||
exposure, tool calls, provider attempts, provider success/failure, and session
|
||||
summaries.
|
||||
|
||||
### I need to see whether CCS exposed WebSearch or the model bypassed it
|
||||
Normal trace metadata uses a query fingerprint and length instead of the raw
|
||||
query. Provider-failure records can also include `error` detail returned by a
|
||||
provider implementation, including HTTP response excerpts or CLI stderr.
|
||||
That provider-generated detail is not guaranteed to exclude query text or
|
||||
other sensitive content unless the implementation redacts it. Treat the trace
|
||||
file as sensitive operational data.
|
||||
|
||||
Run the launch with `CCS_WEBSEARCH_TRACE=1` (or `CCS_DEBUG=1`). CCS writes a JSONL trace to `~/.ccs/logs/websearch-trace.jsonl` with:
|
||||
For delegated/headless sessions, a likely-bypass summary means the tool was
|
||||
exposed but no WebSearch call occurred and another allowed tool path was used.
|
||||
|
||||
1. source-side launch records from CCS (`ccs_websearch_launch`)
|
||||
2. MCP exposure and call records (`mcp_initialize`, `mcp_tools_list`, `mcp_tool_call_*`)
|
||||
3. provider attempt and winner records (`websearch_provider_attempt`, `websearch_provider_success`)
|
||||
4. session summaries (`mcp_session_summary`, and headless `headless_websearch_summary` when applicable)
|
||||
When search is unavailable:
|
||||
|
||||
Queries are fingerprinted (`queryHash`, `queryLength`) instead of logged raw by default. For headless/delegation runs, `headless_websearch_summary.likelyBypassed=true` means the MCP tool was exposed, no WebSearch call occurred, and Claude fell back to `Bash` or `WebFetch`.
|
||||
1. confirm `websearch.enabled` and at least one provider are enabled;
|
||||
2. check readiness in the dashboard;
|
||||
3. verify API keys or the SearXNG base URL without printing secrets;
|
||||
4. enable `CCS_WEBSEARCH_TRACE=1` for one launch;
|
||||
5. inspect provider attempt and cooldown events.
|
||||
|
||||
### WebSearch returns no results
|
||||
|
||||
1. Check `websearch.enabled: true`
|
||||
2. Keep DuckDuckGo enabled unless you have a strong reason to disable it
|
||||
3. If using Exa, Tavily, or Brave, verify the matching API key
|
||||
4. Run with `CCS_DEBUG=1` for runtime logs, or `CCS_WEBSEARCH_TRACE=1` for correlated launch/MCP/provider traces
|
||||
5. If DuckDuckGo returns a non-result HTML error, retry later or enable another provider. CCS now treats that as a provider failure instead of a false empty result.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- API keys entered from the dashboard are stored in `~/.ccs/config.yaml` under `global_env` and injected as environment variables at runtime
|
||||
- Shell-exported keys still work and are detected as external environment input
|
||||
- Never commit API keys to version control
|
||||
- Use the dashboard only on trusted machines, and protect `~/.ccs/config.yaml` with normal user-level filesystem permissions
|
||||
Focused coverage lives under
|
||||
[`tests/unit/utils/websearch/`](../tests/unit/utils/websearch/),
|
||||
[`tests/unit/hooks/`](../tests/unit/hooks/), and
|
||||
[`tests/unit/targets/settings-profile-websearch-launch.test.ts`](../tests/unit/targets/settings-profile-websearch-launch.test.ts).
|
||||
Reference in new issue
Block a user