mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-16 16:15:40 +00:00
feat(providers): support Codex OAuth pools with inherited routing defaults
* feat(auth): support named chatgpt oauth providers - add provider-scoped ChatGPT OAuth routes and CLI support - persist refresh tokens per provider and reject provider-type collisions - wire provider OAuth setup flows in the dashboard and setup UI Refs #448 * feat(agent): add chatgpt oauth account routing - add agent other_config routing for manual and round-robin selection - reuse routed provider resolution across resolver and pending loaders - add router, parser, and agent advanced dialog coverage for multi-account use Refs #448 * docs(api): describe chatgpt oauth routing - document named-provider ChatGPT OAuth auth routes - describe agent-side account routing and round-robin behavior - update OpenAPI agent config schema and provider type enum Refs #448 * fix(store): add missing agent key context helpers * feat(ui): clarify chatgpt oauth account setup and routing * docs(providers): align chatgpt oauth alias examples * feat(agent): add codex pool activity dashboard * fix(providers): harden codex oauth alias setup * feat(codex-pool): improve routing dashboard UX - redesign the Codex/OpenAI pool page around saved-pool checkpoints and live evidence - add clearer selection, attention, and recent-proof states for pool members - make the lower panels fill the remaining desktop viewport while staying responsive * fix(store): resolve context helper merge duplication * feat(oauth): add codex pool quota and observation APIs - add quota inspection and observation endpoints for ChatGPT Subscription (OAuth) providers - teach codex routing to surface pool activity, observation metadata, and quota-aware readiness - extend tests and HTTP docs/OpenAPI for the new pool monitoring flows * feat(web): add codex pool quota monitor and controls - add provider quota fetching, readiness badges, and live routing evidence on the account pool page - redesign pool setup and activity panels for multi-account management with localized copy updates - keep the live monitor internally scrollable and compact the account cards for better viewport fit * fix(web): clarify pool routing labels - rename the recent request badge from Direct to Selected - restore compact quota bars in the live pool cards * feat(codex-pool): add runtime health dashboard - derive per-provider success and failure health from routed Codex traces - surface routing, quota, and recent request evidence in the pool UI - align provider alias guidance and owner access with the dashboard role model * docs(auth): document tenant scoping and key roles * fix(auth): harden tenant and codex pool access control * fix(providers): align codex pool runtime defaults * feat(ui): tighten codex pool responsive layout * feat(chatgpt-oauth): refine codex pool management UX * feat(chatgpt-oauth): surface quota bars on provider pages - add compact quota bars to Codex provider rows and provider detail - fetch quota only for ready visible provider rows and ready detail aliases - fix managed-member detail visibility and tighten provider locale copy
This commit is contained in:
Kai (Tam Nhu) Tran authored and GitHub committed
1 parent
e183b459c9
commit
30708ae79d
113 files changed
+11816
-536
No files matched your search
@@ -627,6 +627,53 @@ Codex provider reports `SupportsThinking() = true`, allowing thinking_level to b
|
||||
|
||||
Tracks prompt, completion, and total tokens. `CacheCreationTokens` and `CacheReadTokens` are supported for prompt caching if available.
|
||||
|
||||
### Provider-Level Defaults + Agent Overrides
|
||||
|
||||
Multiple authenticated `chatgpt_oauth` providers can coexist in one tenant. Each provider name is one OpenAI Codex OAuth alias. Pool membership is authoritative at the provider layer: one alias owns the reusable pool, while member aliases stay leaf accounts.
|
||||
|
||||
Provider default example:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "openai-codex",
|
||||
"provider_type": "chatgpt_oauth",
|
||||
"settings": {
|
||||
"codex_pool": {
|
||||
"strategy": "round_robin",
|
||||
"extra_provider_names": ["codex-work"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Agent override example:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "openai-codex",
|
||||
"other_config": {
|
||||
"chatgpt_oauth_routing": {
|
||||
"override_mode": "custom",
|
||||
"strategy": "round_robin"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Routing behavior:
|
||||
- The main `provider` field remains the preferred/default account.
|
||||
- Provider aliases are arbitrary. `openai-codex` and `codex-work` are examples, not required prefixes.
|
||||
- `settings.codex_pool.extra_provider_names` is the authoritative membership list for that pool owner.
|
||||
- A provider listed in another pool cannot also manage its own pool.
|
||||
- `override_mode: "inherit"` uses the primary provider's `settings.codex_pool`.
|
||||
- `override_mode: "custom"` is limited to routing behavior for that provider-owned pool.
|
||||
- `primary_first` keeps the preferred account fixed. When saved as a custom override with no extra names, it disables the pool for that agent and keeps the agent on the primary account only.
|
||||
- `round_robin` rotates requests across the preferred account plus the provider-owned extra authenticated OpenAI Codex OAuth accounts.
|
||||
- `priority_order` tries the preferred account first, then drains the provider-owned extra accounts in order.
|
||||
- Retryable upstream failures can fall through to the next eligible OpenAI Codex OAuth account in the same request.
|
||||
- Explicit provider names remain explicit. OAuth auth/logout is still provider-scoped.
|
||||
- Runtime observability for one agent is available at `GET /v1/agents/{id}/codex-pool-activity`, which exposes recent routed traces plus per-alias health derived from those traces.
|
||||
|
||||
---
|
||||
|
||||
## 14. File Reference
|
||||
@@ -652,6 +699,7 @@ Tracks prompt, completion, and total tokens. `CacheCreationTokens` and `CacheRea
|
||||
| `internal/providers/codex.go` | CodexProvider: OAuth-based ChatGPT Responses API |
|
||||
| `internal/providers/codex_build.go` | Codex request builder: message formatting, phase handling |
|
||||
| `internal/providers/codex_types.go` | Codex request/response types and OAuth token management |
|
||||
| `internal/providers/chatgpt_oauth_router.go` | Agent-side routing across multiple authenticated OpenAI Codex OAuth providers |
|
||||
| `internal/providers/dashscope.go` | DashScope provider: OpenAI-compat wrapper with thinking budget, tools+streaming fallback |
|
||||
| `internal/providers/acp_provider.go` | ACPProvider: orchestrates ACP-compatible agent subprocesses |
|
||||
| `internal/providers/acp/types.go` | ACP protocol types: InitializeRequest, SessionUpdate, ContentBlock, etc. |
|
||||
|
||||
@@ -138,6 +138,82 @@ POST /v1/agents/{id}/wake
|
||||
|
||||
Response: `{content, run_id, usage?}`. Used by orchestrators (n8n, Paperclip) to trigger agent runs.
|
||||
|
||||
### Codex/OpenAI OAuth Routing in `other_config`
|
||||
|
||||
For agents whose main `provider` is a `chatgpt_oauth` provider, `other_config.chatgpt_oauth_routing`
|
||||
can override or inherit routing behavior while keeping the main `provider` field as the preferred/default account alias.
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "openai-codex",
|
||||
"model": "gpt-5.4",
|
||||
"other_config": {
|
||||
"chatgpt_oauth_routing": {
|
||||
"override_mode": "custom",
|
||||
"strategy": "round_robin"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Provider settings may define reusable `settings.codex_pool` defaults for a primary alias.
|
||||
- `settings.codex_pool.extra_provider_names` is the authoritative membership list for that pool owner.
|
||||
- A provider listed in another pool cannot also manage its own pool.
|
||||
- `override_mode: "inherit"` tells the agent to follow those provider defaults.
|
||||
- `override_mode: "custom"` stores an agent-local routing override for that provider-owned pool.
|
||||
- `strategy: "primary_first"` keeps the main `provider` as the preferred account. When saved as a custom override with no extra names, it disables pooling for that agent.
|
||||
- Provider aliases are arbitrary. `openai-codex`, `codex-work`, and `codex-team` are examples, not required prefixes.
|
||||
- `strategy: "round_robin"` rotates requests across the main provider plus the provider-owned extra authenticated OpenAI Codex OAuth providers.
|
||||
- `strategy: "priority_order"` tries the main provider first, then drains the provider-owned extra providers in order.
|
||||
- Retryable upstream failures can fall through to the next eligible OpenAI Codex OAuth provider in the same request.
|
||||
- Only enabled and authenticated `chatgpt_oauth` providers participate.
|
||||
- Provider-scoped auth remains unchanged: `cmd/auth` and `/v1/auth/chatgpt/{provider}/*` still operate on explicit providers.
|
||||
|
||||
Provider-level defaults example:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "openai-codex",
|
||||
"provider_type": "chatgpt_oauth",
|
||||
"settings": {
|
||||
"codex_pool": {
|
||||
"strategy": "round_robin",
|
||||
"extra_provider_names": ["codex-work", "codex-team"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Codex Pool Activity
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/v1/agents/{id}/codex-pool-activity` | Summarize recent Codex/OpenAI OAuth pool usage for one agent |
|
||||
|
||||
Query parameters:
|
||||
- `limit` optional, defaults to `18`, max `50`
|
||||
|
||||
Response fields:
|
||||
- `strategy`: effective routing strategy (`primary_first`, `round_robin`, or `priority_order`)
|
||||
- `pool_providers`: configured primary + extra provider aliases in pool order
|
||||
- `stats_sample_size`: number of recent routed `llm_call` spans used to derive runtime health. The server derives health from `max(limit, 120)` recent spans even when `recent_requests` is still capped by the requested `limit`.
|
||||
- `provider_counts`: per-alias routing evidence:
|
||||
- `request_count`: backward-compatible count of direct selections
|
||||
- `direct_selection_count`: times the router selected that alias first
|
||||
- `failover_serve_count`: times that alias only served as failover
|
||||
- `success_count`, `failure_count`: trace-backed runtime outcomes attributed to that alias. Success is attributed to the alias that actually served the request. On successful failover, earlier attempted aliases receive failures. On terminal error, every attempted alias receives a failure.
|
||||
- `consecutive_failures`: current newest-first failure streak from recent trace evidence
|
||||
- `success_rate`, `health_score`, `health_state`: additive runtime health summary. `health_score` is heuristic, but the stable bands are `idle` when there are no recent outcomes, `critical` at 3+ consecutive failures or score `< 40`, `degraded` below `80`, otherwise `healthy`
|
||||
- `last_selected_at`, `last_failover_at`, `last_used_at`, `last_success_at`, `last_failure_at`: latest timestamps for each evidence type
|
||||
- `recent_requests`: recent routed Codex calls:
|
||||
- `span_id`, `trace_id`, `started_at`, `status`, `duration_ms`, `model`
|
||||
- `selected_provider`: alias chosen first by the router
|
||||
- `provider_name`: alias that actually served the request. This can be empty on terminal failures where no alias completed the call.
|
||||
- `attempt_count`, `used_failover`, `failover_providers`
|
||||
|
||||
Use `direct_selection_count` plus the `selected_provider` sequence to verify real round-robin behavior. A provider with `failover_serve_count > 0` and `direct_selection_count = 0` was only observed as a rescue target, not as a confirmed round-robin selection.
|
||||
|
||||
---
|
||||
|
||||
## 5. Skills
|
||||
@@ -652,11 +728,87 @@ Admin-only endpoints for managing gateway API keys. See [20 — API Keys & Auth]
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/v1/auth/chatgpt/{provider}/status` | Check ChatGPT OAuth status for a provider |
|
||||
| `GET` | `/v1/auth/chatgpt/{provider}/quota` | Fetch Codex/OpenAI quota state for a provider |
|
||||
| `POST` | `/v1/auth/chatgpt/{provider}/start` | Start ChatGPT OAuth flow for a provider |
|
||||
| `POST` | `/v1/auth/chatgpt/{provider}/callback` | Manual callback handler for a provider |
|
||||
| `POST` | `/v1/auth/chatgpt/{provider}/logout` | Revoke ChatGPT OAuth token for a provider |
|
||||
| `GET` | `/v1/auth/openai/status` | Check OpenAI auth status |
|
||||
| `GET` | `/v1/auth/openai/quota` | Fetch quota state for the default `openai-codex` provider |
|
||||
| `POST` | `/v1/auth/openai/start` | Start OAuth flow |
|
||||
| `POST` | `/v1/auth/openai/callback` | Manual callback handler |
|
||||
| `POST` | `/v1/auth/openai/logout` | Revoke token |
|
||||
|
||||
Legacy `/v1/auth/openai/*` routes remain as compatibility aliases for the default `openai-codex` OpenAI Codex OAuth provider.
|
||||
|
||||
### Provider Quota Response
|
||||
|
||||
`GET /v1/auth/chatgpt/{provider}/quota` and `GET /v1/auth/openai/quota` always return a provider-scoped quota envelope.
|
||||
|
||||
Success payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider_name": "openai-codex",
|
||||
"success": true,
|
||||
"plan_type": "team",
|
||||
"windows": [
|
||||
{
|
||||
"label": "Primary",
|
||||
"used_percent": 24,
|
||||
"remaining_percent": 76,
|
||||
"reset_after_seconds": 3600,
|
||||
"reset_at": "2026-03-24T20:15:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Secondary",
|
||||
"used_percent": 38,
|
||||
"remaining_percent": 62,
|
||||
"reset_after_seconds": 604800,
|
||||
"reset_at": "2026-03-31T19:15:00Z"
|
||||
}
|
||||
],
|
||||
"core_usage": {
|
||||
"five_hour": {
|
||||
"label": "Primary",
|
||||
"remaining_percent": 76,
|
||||
"reset_after_seconds": 3600,
|
||||
"reset_at": "2026-03-24T20:15:00Z"
|
||||
},
|
||||
"weekly": {
|
||||
"label": "Secondary",
|
||||
"remaining_percent": 62,
|
||||
"reset_after_seconds": 604800,
|
||||
"reset_at": "2026-03-31T19:15:00Z"
|
||||
}
|
||||
},
|
||||
"last_updated": "2026-03-24T19:15:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Failure payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider_name": "openai-codex",
|
||||
"success": false,
|
||||
"windows": [],
|
||||
"error": "Quota metadata is missing for this account.",
|
||||
"error_code": "missing_account_id",
|
||||
"action_hint": "Sign in again so GoClaw can restore the ChatGPT account workspace metadata.",
|
||||
"last_updated": "2026-03-24T19:15:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Invalid provider slugs return `400`.
|
||||
- Missing provider still returns `404`, and provider type conflicts still return `409`.
|
||||
- Missing quota metadata, expired workspace access, upstream `402`, upstream `403`, and upstream `429` return `200` with a structured failure payload so the dashboard can render actionable state inline.
|
||||
- `needs_reauth`, `is_forbidden`, and `retryable` are boolean hints for UI/state-machine handling.
|
||||
- `error_code` can be `missing_account_id`, `reauth_required`, `payment_required`, `quota_api_forbidden`, `quota_endpoint_not_found`, `rate_limited`, `provider_unavailable`, `network_timeout`, `network_error`, `quota_request_failed`, or `unknown_upstream_error`.
|
||||
- Failure payloads still include `windows: []` so clients can treat the envelope consistently.
|
||||
- `core_usage.five_hour` and `core_usage.weekly` are derived from upstream windows. When labels drift, GoClaw falls back to shortest-reset and longest-reset usage windows.
|
||||
|
||||
---
|
||||
|
||||
## 26. System
|
||||
|
||||
@@ -97,10 +97,10 @@ The derived role is then used by the `PolicyEngine.CanAccess()` method to gate R
|
||||
|
||||
GoClaw tries authentication methods in this priority order:
|
||||
|
||||
1. **Gateway token** (exact match via constant-time comparison) → `RoleAdmin`
|
||||
1. **Gateway token** (exact match via constant-time comparison) → `RoleAdmin` or `RoleOwner` for configured owner IDs
|
||||
2. **API key** (SHA-256 hash lookup in `api_keys` table) → role from scopes
|
||||
3. **Browser pairing** (sender ID must be paired with "browser" device type) → `RoleOperator` (HTTP only; requires `X-GoClaw-Sender-Id` header)
|
||||
4. **No auth configured** (backward compatibility: if no gateway token is set) → `RoleOperator`
|
||||
4. **No auth configured** (backward compatibility: if no gateway token is set) → full-access dev mode
|
||||
5. **No valid auth found** → `401 Unauthorized`
|
||||
|
||||
### HTTP Request Flow
|
||||
@@ -116,7 +116,7 @@ flowchart TD
|
||||
G -->|Yes| H[Derive role from scopes]
|
||||
G -->|No| I{Gateway token configured?}
|
||||
I -->|Yes| J[401 Unauthorized]
|
||||
I -->|No| K[RoleOperator - backward compat]
|
||||
I -->|No| K[Full-access backward compat]
|
||||
C -->|Check paired device| L{Device paired?}
|
||||
L -->|Yes| M[RoleOperator]
|
||||
L -->|No| J
|
||||
@@ -151,11 +151,20 @@ On successful API key authentication, `last_used_at` is updated asynchronously (
|
||||
- **Bearer token**: `Authorization: Bearer <token>` — checked first for gateway token or API key
|
||||
- **User ID**: `X-GoClaw-User-Id: <user-id>` — optional external user identifier (max 255 chars)
|
||||
- **Browser pairing**: `X-GoClaw-Sender-Id: <sender-id>` — identifies a previously-paired browser device
|
||||
- **Tenant scope**: `X-GoClaw-Tenant-Id: <tenant-uuid-or-slug>` — owner/system-key scope narrowing; non-owner gateway token and browser-pairing callers must already belong to the requested tenant
|
||||
- **Locale**: `Accept-Language` — user's preferred language (en, vi, zh; default: en)
|
||||
|
||||
### Tenant Scope Rules
|
||||
|
||||
- **Gateway token + owner user ID**: may narrow to any tenant via `X-GoClaw-Tenant-Id`
|
||||
- **Gateway token + non-owner user ID**: may only use `X-GoClaw-Tenant-Id` for a tenant where that user already has membership; otherwise auth fails
|
||||
- **Browser pairing**: same tenant-membership rule as non-owner gateway-token HTTP requests
|
||||
- **Tenant-bound API key**: always stays bound to its stored `tenant_id`; request headers cannot move it
|
||||
- **System-level API key** (`tenant_id = NULL`): keeps its scope-derived role (`admin`, `operator`, or `viewer`) and may narrow requests to a tenant via `X-GoClaw-Tenant-Id`, but it does **not** become `owner`
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
If no gateway token is configured (`gateway.token` is empty in `config.json`), all unauthenticated requests are granted `RoleOperator` access. This enables self-hosted deployments without strict authentication. Once a gateway token is configured, all requests must authenticate or use browser pairing.
|
||||
If no gateway token is configured (`gateway.token` is empty in `config.json`), unauthenticated requests run in backward-compatibility full-access mode. This enables self-hosted deployments without strict authentication. Once a gateway token is configured, all requests must authenticate or use browser pairing.
|
||||
|
||||
---
|
||||
|
||||
@@ -205,7 +214,7 @@ The partial index on `key_hash` ensures only active (non-revoked) keys are searc
|
||||
| `api_keys.create` | Create key |
|
||||
| `api_keys.revoke` | Revoke key |
|
||||
|
||||
All API key management operations require admin access (gateway token or API key with `operator.admin` scope).
|
||||
All API key management operations require admin access (gateway token or API key with `operator.admin` scope). Owner callers are accepted because owner is a superset of admin.
|
||||
|
||||
### Create Request
|
||||
|
||||
|
||||
@@ -154,10 +154,10 @@ GoClaw determines the tenant from the credentials used to connect:
|
||||
| Credential | Tenant Resolution | Use Case |
|
||||
|------------|-------------------|----------|
|
||||
| **Gateway token** + owner user ID | All tenants (cross-tenant) | System administration |
|
||||
| **Gateway token** + non-owner user ID | User's tenant membership | Dashboard users |
|
||||
| **Gateway token** + non-owner user ID | Master tenant by default, or a membership-validated `X-GoClaw-Tenant-Id` / `tenant_id` hint | Dashboard users |
|
||||
| **API key** (tenant-bound) | Auto from key's `tenant_id` | Normal SaaS integration |
|
||||
| **API key** (system-level) + `X-GoClaw-Tenant-Id` | Header value (UUID or slug) | Cross-tenant admin tools |
|
||||
| **Browser pairing** | Paired tenant | Dashboard operators |
|
||||
| **API key** (system-level) + `X-GoClaw-Tenant-Id` | Header value (UUID or slug), while keeping the key's original role | Cross-tenant tools |
|
||||
| **Browser pairing** | Master tenant by default, or a membership-validated tenant hint | Dashboard operators |
|
||||
| **No credentials** | Master tenant | Dev/single-user mode |
|
||||
|
||||
**Owner IDs:** Configured via `GOCLAW_OWNER_IDS` env var (comma-separated). Only owners get cross-tenant access with the gateway token. Default: `system`.
|
||||
|
||||
Reference in new issue
Block a user