feat(tts): wire tenant timeout + fix Gemini text-only 400

- HTTP synthesize + test-connection now read tenant tts.timeout_ms
  (default 120s, was hardcoded 15s/10s). Gemini client default also
  bumped 30s→120s so both layers align when tenant config unset.
- Inline prefix "Speak naturally: " prepended to single-voice text;
  multi-speaker transcripts pass through unchanged.
- ErrTextOnlyResponse sentinel for 400 "text generation" bodies;
  single-voice retries once with stronger prefix. Narrowed needle
  list avoids false positives on unrelated 400s.
- SynthesizeWithFallbackAdapted now returns errors.Join so sentinel
  survives fallback chain; HTTP 422 mapping + locale-translated
  ForLLM in agent tool (EN/VI/ZH catalogs).
- Default Gemini model bumped to gemini-3.1-flash-tts-preview.
This commit is contained in:
viettranx committed 2026-04-23 08:31:53 +07:00
1 parent 4d6ebe9c58
commit 04a9938f4f
24 files changed
+891 -28

No files matched your search

+2 -1
View File
@@ -100,7 +100,7 @@ old clients see flat keys; new clients see the full params blob.
### Gemini Specifics
- Models: `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` (preview).
- Models: `gemini-3.1-flash-tts-preview` (default), `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` (preview).
- Multi-speaker: up to 2 simultaneous speakers, each with distinct voice + name annotation.
- Audio tags: inline `<say-as>` / style directives via bracketed prompts.
- Sentinel errors: `ErrInvalidVoice`, `ErrInvalidModel`, `ErrSpeakerLimit` → HTTP 422 with i18n message.
@@ -143,6 +143,7 @@ Native `image_generation` support in the Codex provider (`POST /codex/responses`
## Key Conventions
- **Store layer:** Interface-based; PG (`store/pg/`) + SQLite (`store/sqlitestore/`). Raw SQL, `$1/$2` params.
- **Session token display:** v3 compaction now uses dynamic max_tokens; session token display reads from `sessions.metadata.last_prompt_tokens`.
- **Context propagation:** `store.WithLocale`, `store.WithUserID`, `store.WithTenantID`, etc.
- **Security logs:** `slog.Warn("security.*")` for all security events.
- **SSRF prevention:** `validateProviderURL()` in `internal/http/tts_validate.go`.
+23
View File
@@ -37,6 +37,29 @@ Implementation is evidence-backed against the native ChatGPT Responses API event
## 2026-04-20
### TTS: timeout tenant-config + Gemini text-only 400 fix
**Features & Fixes**
- **Tenant-config timeout:** HTTP `/v1/tts/synthesize` and `/v1/tts/test-connection` now read `tts.timeout_ms` from system_configs (default 120s, was hardcoded 15s/10s). Gemini client default bumped 30s→120s for end-to-end alignment.
- **Gemini text-only error recovery:** Gemini preview models occasionally emit 400 "text generation" responses. Fixed by: (1) prepending inline prefix `"Speak naturally: "` to single-voice synthesis (multi-speaker untouched), (2) 1-retry with stronger prefix `"Read the following text aloud without translating, commenting, or modifying: "`, (3) new sentinel `gemini.ErrTextOnlyResponse` preserved through fallback chain via `errors.Join`.
- **Error UX:** HTTP returns 422 with localized `MsgTtsGeminiTextOnly` message. Agent TTS tool branches on sentinel to emit locale-translated ForLLM response.
- **Model default:** Gemini default model bumped `gemini-2.5-flash-preview-tts` → `gemini-3.1-flash-tts-preview` for higher stability.
- **UI bounds:** TTS timeout input now has `max=300000` (5 min).
**i18n**
- New key `MsgTtsGeminiTextOnly` in EN/VI/ZH catalogs for HTTP 422 + agent-tool ForLLM mapping.
**Code**
- `internal/audio/tts.go` — read tenant timeout in synthesize handlers.
- `internal/audio/gemini/` — inline prefix logic, retry budget, text-only sentinel.
- `internal/tools/tts.go` — agent-tool i18n branching on sentinel.
- `internal/http/methods/tts.go` — HTTP 422 error mapping.
---
### Tools: `send_file` — explicit workspace file delivery
**Features**
+1 -1
View File
@@ -323,9 +323,9 @@ For each `ParamSchema`, add:
Gemini TTS uses preview models only (as of 2026-04):
- `gemini-3.1-flash-tts-preview` (**default** — higher Elo, more stable)
- `gemini-2.5-flash-preview-tts`
- `gemini-2.5-pro-preview-tts`
- `gemini-3.1-flash-tts-preview`
The frontend displays a "Preview" badge (i18n key `tts.gemini.previewBadge`).