2 Commits
Author SHA1 Message Date
ntduc bb58e03352 fix(audio): resolve provider options and honor STT timeout 2026-07-22 20:48:12 +07:00
Bruno ClermontandBruno Clermont cc9956c1f9 feat(audio): openai_compat TTS/STT provider for self-hosted endpoints (#1447)
* feat(audio): add openai_compat TTS/STT provider for self-hosted endpoints

Adds a provider pair speaking the OpenAI audio wire format against any
compatible endpoint (gpu-manager, Speaches, vLLM, LocalAI, llama.cpp,
Ollama), so a self-hosted engine no longer needs a bespoke HTTP shim
implementing goclaw's proprietary /transcribe_audio contract.

Kept distinct from the openai package on purpose. Both speak the same
wire format, but "openai" carries api.openai.com semantics, and
audio.IsVoiceCompatible applies OpenAI's voice allowlist to any provider
by that name — FilterVoiceForProvider then silently substitutes "alloy"
for anything outside it. A self-hosted voice ID such as Piper's
"fr_FR-gilles-low" would be replaced with no error and no log line, and
the caller would hear the wrong language. Providers without validation
rules pass through untouched, which is the correct behaviour for an
engine whose voice namespace is its own.

Other deliberate departures from the openai package:

- api_base is required. There is no public endpoint to fall back to, and
  defaulting to api.openai.com would send self-hosted traffic to a vendor.
- An empty api_key omits the Authorization header rather than sending an
  empty Bearer token; self-hosted engines commonly have no auth.
- Empty voice/model are omitted from the request instead of sent blank,
  since some engines resolve the model from the requested voice.
- The multipart filename is never blank: several engines type the audio
  by extension rather than Content-Type and reject what they cannot type.

Endpoint error bodies are preserved in the returned error so a format
mismatch (e.g. an engine that cannot encode mp3) is diagnosable.

* feat(audio): wire openai_compat TTS/STT provider into gateway config

Adds tts.openai_compat config block and registers both providers at
startup. api_base is the enable switch rather than an API key, since
self-hosted endpoints commonly have no auth and there is no vendor
default to fall back to.

The STT chain is now built from the providers that actually registered
instead of being hardcoded to {elevenlabs, proxy}. Transcribe skips
unregistered names with a warning on every call, so a static chain
naming absent providers is log noise on the hot path. openai_compat goes
first when present: it is an explicit operator choice and keeps audio on
the local network. "proxy" stays last, as BridgeLegacySTT registers it
later per channel.

Behaviour is unchanged when tts.openai_compat.api_base is unset: the
chain resolves to {elevenlabs, proxy} exactly as before.

---------

Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com>
2026-07-17 22:51:16 +07:00