feat(proxy): complete openai routing scope

2026-10-14 08:18:04 +00:00 · 2026-04-15 00:55:11 -04:00
1 parent b346693c3a
commit 17187c4abd
@@ -68,8 +68,22 @@ ccs proxy status
 ccs proxy stop
 ```

-`ccs proxy activate` prints the `ANTHROPIC_*` exports needed for a local
-Anthropic-compatible session against the running proxy.
+Useful variants:
+
+```bash
+ccs proxy start hf --host 127.0.0.1
+ccs proxy activate --fish
+```
+
+`ccs proxy activate` now prints the full local runtime contract:
+
+- `ANTHROPIC_BASE_URL`
+- `ANTHROPIC_AUTH_TOKEN`
+- `ANTHROPIC_MODEL` plus tier defaults when present
+- `DISABLE_TELEMETRY`
+- `DISABLE_COST_WARNINGS`
+- `API_TIMEOUT_MS`
+- `NO_PROXY`

 ## One Active Proxy Profile

@@ -82,6 +96,56 @@ The current runtime is a single local proxy daemon.
 This is intentional to avoid breaking an in-flight Claude session by swapping
 its upstream provider out from under it.

+## Request-Time Routing
+
+The proxy is no longer limited to the startup profile's default model.
+
+Supported request-time selectors:
+
+- `profile:model`
+  Example: `deepseek:deepseek-reasoner`
+- `profile`
+  Example: `openrouter`
+- plain model ids
+  Example: `deepseek-chat`
+
+Routing behavior:
+
+1. `profile:model` wins immediately.
+2. Scenario routing may override the active profile when configured.
+3. Plain model ids are matched against the configured OpenAI-compatible
+   profiles before falling back to the active profile.
+
+This means a Claude session launched through one compatible profile can still
+request another compatible profile/model when the proxy can resolve it safely.
+
+## Scenario Routing
+
+Scenario routing is now supported through `proxy.routing` in your CCS config.
+
+Example `~/.ccs/config.yaml`:
+
+```yaml
+proxy:
+  routing:
+    default: "deepseek:deepseek-chat"
+    background: "ollama:qwen2.5-coder:0.5b"
+    think: "deepseek:deepseek-reasoner"
+    longContext: "openrouter:google/gemini-2.5-pro"
+    longContextThreshold: 60000
+    webSearch: "openrouter:perplexity/sonar-pro"
+```
+
+Current scenario detection:
+
+- `background`: requested model contains `haiku`
+- `think`: Anthropic `thinking` is enabled
+- `longContext`: estimated request tokens exceed `longContextThreshold`
+- `webSearch`: tool list includes `web_search`
+- `default`: fallback selector when the above do not apply
+
+Routing decisions are logged through CCS structured logs.
+
 ## How Profile Detection Works

 CCS keeps these profiles in the normal API/settings-profile flow.
@@ -102,6 +166,79 @@ OpenAI-compatible endpoints such as:

 are routed through the local proxy for Claude-target launches.

+## Provider Setup
+
+### DeepSeek
+
+Use a settings profile whose env looks like:
+
+```json
+{
+  "env": {
+    "ANTHROPIC_BASE_URL": "https://api.deepseek.com/v1",
+    "ANTHROPIC_AUTH_TOKEN": "sk-...",
+    "ANTHROPIC_MODEL": "deepseek-chat",
+    "CCS_DROID_PROVIDER": "generic-chat-completion-api"
+  }
+}
+```
+
+Typical override target:
+
+- `deepseek:deepseek-reasoner`
+
+### OpenRouter
+
+```json
+{
+  "env": {
+    "ANTHROPIC_BASE_URL": "https://openrouter.ai/api/v1",
+    "ANTHROPIC_AUTH_TOKEN": "sk-or-...",
+    "ANTHROPIC_MODEL": "openai/gpt-4.1-mini",
+    "CCS_DROID_PROVIDER": "generic-chat-completion-api"
+  }
+}
+```
+
+Useful when you want:
+
+- model fan-out behind one provider profile
+- long-context or web-search scenario targets
+
+### Ollama / Local Gateways
+
+```json
+{
+  "env": {
+    "ANTHROPIC_BASE_URL": "http://127.0.0.1:11434",
+    "ANTHROPIC_AUTH_TOKEN": "ollama",
+    "ANTHROPIC_MODEL": "qwen3-coder",
+    "CCS_DROID_PROVIDER": "generic-chat-completion-api"
+  }
+}
+```
+
+For self-signed HTTPS gateways, add `CCS_OPENAI_PROXY_INSECURE=1`.
+
+### DashScope / Qwen Compatible Mode
+
+DashScope's compatible endpoint works even when older settings files still
+carry a stale Anthropic-style provider hint:
+
+```json
+{
+  "env": {
+    "ANTHROPIC_BASE_URL": "https://dashscope-us.aliyuncs.com/compatible-mode/v1",
+    "ANTHROPIC_AUTH_TOKEN": "sk-...",
+    "ANTHROPIC_MODEL": "qwen3.6-plus",
+    "CCS_DROID_PROVIDER": "anthropic"
+  }
+}
+```
+
+CCS now infers the OpenAI-compatible route from the base URL and does not let
+that stale provider hint block proxy routing.
+
 ## Self-Signed TLS

 If your upstream gateway uses a self-signed or privately issued certificate,
@@ -124,29 +261,50 @@ That flag is respected by both:

 - `ccs <profile>` with Claude target: auto-starts the local proxy when needed
 - `ccs proxy start <profile>`: starts the proxy explicitly
+- `GET /`: proxy info and bound profile details
 - `GET /health`: proxy liveness check
 - `GET /v1/models`: local view of the configured model mapping
 - `POST /v1/messages`: Anthropic-compatible request entrypoint

+## Troubleshooting
+
+### Missing or invalid local proxy token
+
+- Re-run `eval "$(ccs proxy activate)"`
+- Check `ccs proxy status` and confirm the expected profile is running
+
+### Self-signed or private CA upstream
+
+- Add `CCS_OPENAI_PROXY_INSECURE=1` to the profile settings
+- Restart the proxy after changing the setting
+
+### Port conflict on `3456`
+
+- Start with a fixed port: `ccs proxy start hf --port 3457`
+- Re-run `ccs proxy activate` after changing the port
+
+### Provider returns `429` or empty upstream output
+
+- CCS now preserves upstream rate-limit errors and retry headers
+- Empty or malformed provider JSON is returned as Anthropic-style `api_error`
+
+### Requests route to the wrong model/profile
+
+- Use an explicit selector such as `profile:model`
+- Review `proxy.routing` if scenario routing is enabled
+- Check CCS structured logs in `~/.ccs/logs/current.jsonl` for routing decisions
+
 ## Validation

 The shipped coverage includes:

 - unit tests for OpenAI-compatible profile detection
 - unit tests for Anthropic -> OpenAI request translation
+- unit tests for request-time profile/model routing and scenario routing
 - unit tests for multi-line SSE parsing
 - integration tests for `/v1/messages` request/response translation
+- integration tests for rate limits, empty upstream responses, timeout handling,
+  thinking/tool-call chunk streaming, and request-time routing
 - integration tests for daemon lifecycle and `/health` / `/v1/models`
 - e2e tests for `ccs proxy` lifecycle
 - e2e tests for `ccs <profile>` auto-routing through a mock upstream
-
-## Current Scope
-
-The current implementation focuses on the core routing path:
-
- local proxy lifecycle
- Anthropic/OpenAI request-response translation
- Claude-target settings profile auto-routing
-
-Scenario-based routing and token-count-driven model switching remain follow-up
-work if they are needed beyond the base provider-routing flow.
@@ -41,7 +41,7 @@ All major modularization work is complete. The codebase evolved from monolithic

 ### Recent Fixes

- **2026-04-14**: **#991** CCS now auto-routes Claude-target settings profiles that use OpenAI-compatible endpoints through a local Anthropic-compatible proxy instead of sending raw Anthropic `/v1/messages` traffic directly to chat-completions backends. The new `ccs proxy` command supports `start`, `status`, `activate`, and `stop`, the runtime exposes `/health` and `/v1/models`, the shared SSE parser now handles multi-line `data:` payloads, and the feature ships with dedicated unit, integration, and e2e coverage.
+- **2026-04-14**: **#991** CCS now auto-routes Claude-target settings profiles that use OpenAI-compatible endpoints through a local Anthropic-compatible proxy instead of sending raw Anthropic `/v1/messages` traffic directly to chat-completions backends. The `ccs proxy` command now supports `start`, `status`, `activate`, and `stop` with explicit host binding, shell-aware activation helpers, and a fuller local runtime env contract. The proxy surface now exposes `GET /`, `/health`, `/v1/models`, and `/v1/messages`, logs routing decisions into CCS structured logs, supports Anthropic image blocks plus request-time `profile:model` overrides, and adds config-driven scenario routing (`background`, `think`, `longContext`, `webSearch`) on top of the compatible-profile path. Coverage now includes request routing, rate-limit/timeout/empty-upstream failures, chunked tool-call streaming, and disconnect cleanup alongside the existing unit, integration, and e2e suites.
 - **2026-04-10**: **#765** `/providers` now includes a first-class Hugging Face preset for API Profiles. CCS exposes Hugging Face Inference Providers through the existing OpenAI-compatible profile flow with the official router endpoint `https://router.huggingface.co/v1`, a short `hf` default profile name, and `hf` preset alias support for both the dashboard chooser and `ccs api create --preset hf`.
 - **2026-04-10**: **#944** Image Analysis auth readiness no longer collapses to native Read when merged runtime-status dependency overrides include a missing initializer value. CCS now preserves default dependency functions when override entries are `undefined`, still reads token-backed auth status directly in the local readiness path, and includes regression coverage for the missing-initializer case that previously surfaced as `deps.initializeAccounts is not a function`.
 - **2026-04-10**: **#945** CCS now normalizes Gemini CLI and Antigravity tier signals around an explicit `free / pro / ultra / unknown` model, preserves raw tier ids such as `g1-pro-tier`, enriches Gemini quota responses with provider entitlement evidence, classifies `MODEL_CAPACITY_EXHAUSTED` separately from auth/entitlement failures, fixes the Antigravity CLI quota table so live quota-derived tiers no longer collapse back to stale `unknown`, adds Gemini tier ids to CLI quota output, extends Gemini Flash Lite grouping to cover `gemini-3.1-flash-lite-preview`, and allows Gemini account surfaces to render the same tier badge semantics as Antigravity.