Files
goclaw/internal/providers/openai_endpoints.go
T
Plateau Nguyen 0e994f959c feat(providers): explicit prompt cache for DashScope/Qwen (#1127)
* feat(providers): explicit prompt cache for DashScope/Qwen

Extend OpenAI-compat path with Anthropic-style cache_control:ephemeral
inline blocks for Alibaba DashScope endpoints (provider_type=bailian or
URL contains "dashscope"). Verified live: qwen3.6-plus / qwen3.5-plus /
qwen3-coder-plus all return 99.7% cache hit rate on 6K-token prefix,
yielding ~89% token cost reduction on cached prefix (90% Alibaba discount).

- isDashScope(): 3-source detection (URL + providerType + name) handles
  reverse-proxied endpoints; covers both "dashscope" and "bailian"
- buildRequestBody wraps system content with SplitSystemPromptForCache
  (exported from anthropic_request.go) using <!-- GOCLAW_CACHE_BOUNDARY -->
- Tool prefix cache: cache_control on last tool definition, with 4-marker
  budget guard
- Parse cache_creation_input_tokens from prompt_tokens_details into
  Usage.CacheCreationTokens; propagates via existing span metadata writers
- Runtime escape hatch: GOCLAW_DISABLE_DASHSCOPE_CACHE=true
- Live integration smoke test (build tag integration, env-gated)

* chore(providers): polish per PR #1127 review

- Use strings.Repeat instead of custom repeat() helper in smoke test
- Clarify BuildRequestBodyForTest is test-only, not public API

* feat(providers): enable thinking for Qwen 3.7/3.6, observe cache in smoke test

Qwen3.7-plus and qwen3.6-plus support deep thinking but were missing from
dashscopeThinkingModels, so enable_thinking/thinking_budget was silently
skipped. Add both to the whitelist and test.

Add qwen3.7-plus to the cache smoke test as cache-optional: Alibaba's
context-cache doc does not yet list 3.7/3.6 for explicit cache, and the
cache wrap is a safe no-op when unsupported, so a no-cache result is logged
rather than failed to avoid a flaky live assertion.

Claude-Session: https://claude.ai/code/session_01X3jkrc7N3ar8ZGzUJWyNS5

* fix(providers): preserve DashScope detection for proxy routes
2026-06-19 10:59:02 +07:00

71 lines
2.7 KiB
Go

package providers
import "strings"
// isOpenAINativeEndpoint returns true for endpoints confirmed to be native OpenAI
// infrastructure that accepts the "developer" message role.
// Azure OpenAI, proxies, and other OpenAI-compatible backends only support "system".
// Matching OpenClaw TS: model-compat.ts → isOpenAINativeEndpoint().
func isOpenAINativeEndpoint(apiBase string) bool {
// Extract hostname from the API base URL.
lower := strings.ToLower(apiBase)
return strings.Contains(lower, "api.openai.com")
}
// isFireworksEndpoint returns true for Fireworks AI endpoints.
// Fireworks requires stream=true for max_tokens > 4096.
func (p *OpenAIProvider) isFireworksEndpoint() bool {
return strings.Contains(strings.ToLower(p.apiBase), "fireworks.ai")
}
// isTogetherEndpoint returns true for Together AI inference hosts.
// Together rejects some OpenAI extensions (e.g. stream_options, reasoning_effort) with HTTP 400.
// Uses URL, provider_type, and name so reverse-proxied Together endpoints are also detected.
func (p *OpenAIProvider) isTogetherEndpoint() bool {
b := strings.ToLower(p.apiBase)
if strings.Contains(b, "together.xyz") || strings.Contains(b, "together.ai") {
return true
}
if strings.Contains(strings.ToLower(strings.TrimSpace(p.providerType)), "together") {
return true
}
if strings.Contains(strings.ToLower(p.name), "together") {
return true
}
return false
}
// isDashScopeAPIBase returns true for Alibaba DashScope OpenAI-compatible endpoints.
func isDashScopeAPIBase(apiBase string) bool {
return strings.Contains(strings.ToLower(apiBase), "dashscope")
}
// dashScopePassthroughKeys is true when enable_thinking / thinking_budget may be added to the JSON body.
// Uses the same DashScope/Bailian route detection as prompt-cache wrapping.
func (p *OpenAIProvider) dashScopePassthroughKeys() bool {
return p.isDashScope()
}
// isDashScope returns true when this provider routes requests to DashScope/Bailian
// (supports cache_control:ephemeral wire format - verified live 2026-05-08).
// Uses 3-source detection (URL + providerType + name) to handle reverse-proxied
// DashScope endpoints. Includes "bailian" because live qwen-richard provider has
// provider_type=bailian.
//
// Used by buildRequestBody to wrap system content with Anthropic-style
// cache_control blocks for prompt caching (90% discount on cached prefix tokens).
func (p *OpenAIProvider) isDashScope() bool {
if isDashScopeAPIBase(p.apiBase) {
return true
}
pt := strings.ToLower(strings.TrimSpace(p.providerType))
if strings.Contains(pt, "dashscope") || strings.Contains(pt, "bailian") {
return true
}
name := strings.ToLower(p.name)
if strings.Contains(name, "dashscope") || strings.Contains(name, "bailian") {
return true
}
return false
}