mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-14 20:17:20 +00:00
feat(tts): wire tenant timeout + fix Gemini text-only 400
- HTTP synthesize + test-connection now read tenant tts.timeout_ms (default 120s, was hardcoded 15s/10s). Gemini client default also bumped 30s→120s so both layers align when tenant config unset. - Inline prefix "Speak naturally: " prepended to single-voice text; multi-speaker transcripts pass through unchanged. - ErrTextOnlyResponse sentinel for 400 "text generation" bodies; single-voice retries once with stronger prefix. Narrowed needle list avoids false positives on unrelated 400s. - SynthesizeWithFallbackAdapted now returns errors.Join so sentinel survives fallback chain; HTTP 422 mapping + locale-translated ForLLM in agent tool (EN/VI/ZH catalogs). - Default Gemini model bumped to gemini-3.1-flash-tts-preview.
This commit is contained in:
1 parent
4d6ebe9c58
commit
04a9938f4f
24 files changed
+891
-28
No files matched your search
@@ -100,7 +100,7 @@ old clients see flat keys; new clients see the full params blob.
|
||||
|
||||
### Gemini Specifics
|
||||
|
||||
- Models: `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` (preview).
|
||||
- Models: `gemini-3.1-flash-tts-preview` (default), `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` (preview).
|
||||
- Multi-speaker: up to 2 simultaneous speakers, each with distinct voice + name annotation.
|
||||
- Audio tags: inline `<say-as>` / style directives via bracketed prompts.
|
||||
- Sentinel errors: `ErrInvalidVoice`, `ErrInvalidModel`, `ErrSpeakerLimit` → HTTP 422 with i18n message.
|
||||
@@ -143,6 +143,7 @@ Native `image_generation` support in the Codex provider (`POST /codex/responses`
|
||||
## Key Conventions
|
||||
|
||||
- **Store layer:** Interface-based; PG (`store/pg/`) + SQLite (`store/sqlitestore/`). Raw SQL, `$1/$2` params.
|
||||
- **Session token display:** v3 compaction now uses dynamic max_tokens; session token display reads from `sessions.metadata.last_prompt_tokens`.
|
||||
- **Context propagation:** `store.WithLocale`, `store.WithUserID`, `store.WithTenantID`, etc.
|
||||
- **Security logs:** `slog.Warn("security.*")` for all security events.
|
||||
- **SSRF prevention:** `validateProviderURL()` in `internal/http/tts_validate.go`.
|
||||
|
||||
@@ -37,6 +37,29 @@ Implementation is evidence-backed against the native ChatGPT Responses API event
|
||||
|
||||
## 2026-04-20
|
||||
|
||||
### TTS: timeout tenant-config + Gemini text-only 400 fix
|
||||
|
||||
**Features & Fixes**
|
||||
|
||||
- **Tenant-config timeout:** HTTP `/v1/tts/synthesize` and `/v1/tts/test-connection` now read `tts.timeout_ms` from system_configs (default 120s, was hardcoded 15s/10s). Gemini client default bumped 30s→120s for end-to-end alignment.
|
||||
- **Gemini text-only error recovery:** Gemini preview models occasionally emit 400 "text generation" responses. Fixed by: (1) prepending inline prefix `"Speak naturally: "` to single-voice synthesis (multi-speaker untouched), (2) 1-retry with stronger prefix `"Read the following text aloud without translating, commenting, or modifying: "`, (3) new sentinel `gemini.ErrTextOnlyResponse` preserved through fallback chain via `errors.Join`.
|
||||
- **Error UX:** HTTP returns 422 with localized `MsgTtsGeminiTextOnly` message. Agent TTS tool branches on sentinel to emit locale-translated ForLLM response.
|
||||
- **Model default:** Gemini default model bumped `gemini-2.5-flash-preview-tts` → `gemini-3.1-flash-tts-preview` for higher stability.
|
||||
- **UI bounds:** TTS timeout input now has `max=300000` (5 min).
|
||||
|
||||
**i18n**
|
||||
|
||||
- New key `MsgTtsGeminiTextOnly` in EN/VI/ZH catalogs for HTTP 422 + agent-tool ForLLM mapping.
|
||||
|
||||
**Code**
|
||||
|
||||
- `internal/audio/tts.go` — read tenant timeout in synthesize handlers.
|
||||
- `internal/audio/gemini/` — inline prefix logic, retry budget, text-only sentinel.
|
||||
- `internal/tools/tts.go` — agent-tool i18n branching on sentinel.
|
||||
- `internal/http/methods/tts.go` — HTTP 422 error mapping.
|
||||
|
||||
---
|
||||
|
||||
### Tools: `send_file` — explicit workspace file delivery
|
||||
|
||||
**Features**
|
||||
|
||||
@@ -323,9 +323,9 @@ For each `ParamSchema`, add:
|
||||
|
||||
Gemini TTS uses preview models only (as of 2026-04):
|
||||
|
||||
- `gemini-3.1-flash-tts-preview` (**default** — higher Elo, more stable)
|
||||
- `gemini-2.5-flash-preview-tts`
|
||||
- `gemini-2.5-pro-preview-tts`
|
||||
- `gemini-3.1-flash-tts-preview`
|
||||
|
||||
The frontend displays a "Preview" badge (i18n key `tts.gemini.previewBadge`).
|
||||
|
||||
|
||||
Reference in new issue
Block a user