feat(tts): provider capabilities schema + Gemini TTS + dynamic param forms

Baseline groundwork for TTS expansion plan. Introduces a capabilities
system that the follow-up plan (phase-01..04) builds on.

Backend:
- `audio.ParamSchema` / `ProviderCapabilities` types + per-provider
  capabilities.go for edge, elevenlabs, minimax, openai, gemini.
- Gemini TTS provider (client, models, voices, wav encoder,
  multi-speaker via SpeakerVoice, audio-tag aware prompts).
- TTSOptions gains `Params map[string]any` (read-only) + `Speakers`.
- `VoiceListProvider` interface decouples HTTP voice handler from
  provider-specific impls.
- `nested_keys.go` resolves dot-separated param paths for nested
  provider bodies (voice_settings.stability etc.).
- Characterization + defaults-invariant tests per provider lock
  nil-params byte-equivalence before new params land.
- `/v1/tts/capabilities` HTTP endpoint + integration coverage.
- Dual-read tests (PG + SQLite) for tts_config.

Frontend (web + desktop):
- `DynamicParamForm` with depends-on evaluation, split into
  fields/logic modules. Slider primitive added.
- `AudioTagPicker` + `MultiSpeakerEditor` for Gemini.
- `voice-picker` refactored toward portal UX; combobox tightened.
- tts-capabilities API client + typed hooks.
- i18n catalogs (en/vi/zh) expanded; parity tests guard key drift.
- TTS page reorganised (voice-model-section removed; playground +
  credentials + provider-setup split cleanly).

Docs: codebase-summary, project-changelog, tts-provider-capabilities.
This commit is contained in:
viettranx committed 2026-04-20 00:05:19 +07:00
1 parent 7639a8c013
commit 613b6e38d7
123 files changed
+12233 -994

No files matched your search

+11
View File
@@ -5,6 +5,7 @@ import (
"log/slog"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
geminiaudio "github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
minimaxaudio "github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
"github.com/nextlevelbuilder/goclaw/internal/bus"
"github.com/nextlevelbuilder/goclaw/internal/config"
@@ -297,6 +298,16 @@ func setupTTS(cfg *config.Config) *tts.Manager {
}))
}
if key := ttsCfg.Gemini.APIKey; key != "" {
mgr.RegisterProvider(geminiaudio.NewProvider(geminiaudio.Config{
APIKey: key,
APIBase: ttsCfg.Gemini.APIBase,
Voice: ttsCfg.Gemini.Voice,
Model: ttsCfg.Gemini.Model,
TimeoutMs: ttsCfg.TimeoutMs,
}))
}
if !mgr.HasProviders() {
return nil
}
+8
View File
@@ -513,6 +513,14 @@ Text-to-speech synthesis and configuration.
| `GET` | `/v1/tts/config` | Get tenant TTS configuration (admin) |
| `POST` | `/v1/tts/config` | Save tenant TTS configuration (admin) |
### Provider Capabilities
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/v1/tts/capabilities` | Get `ProviderCapabilities` schema for all registered providers (operator role) |
The capabilities endpoint returns the dynamic param schema for each provider — param types, ranges, defaults, `dependsOn` conditions, and custom feature flags. Used by the UI to render per-provider param editors without hard-coded field lists. See [`docs/tts-provider-capabilities.md`](tts-provider-capabilities.md) for schema details.
---
## 10. Memory
+129
View File
@@ -0,0 +1,129 @@
# Codebase Summary
High-level map of GoClaw modules and key cross-cutting concerns.
For system design see `docs/00-architecture-overview.md`; for API contract see `docs/18-http-api.md`.
---
## Module Map
| Path | Purpose |
|------|---------|
| `cmd/` | CLI (cobra): `serve`, `onboard`, `migrate` commands |
| `internal/agent/` | Agent loop (think→act→observe), router, input guard |
| `internal/audio/` | TTS provider layer (see §TTS below) |
| `internal/bootstrap/` | SOUL/IDENTITY system prompts + per-user seed |
| `internal/channels/` | Telegram, Feishu, Zalo, Discord, WhatsApp connectors |
| `internal/config/` | JSON5 config loading + env overlay |
| `internal/crypto/` | AES-256-GCM encryption for API keys |
| `internal/gateway/` | WS + HTTP server, client, method router |
| `internal/http/` | HTTP API handlers (`/v1/*`) |
| `internal/i18n/` | Backend message catalog (EN/VI/ZH) + `T(locale, key, args)` |
| `internal/memory/` | pgvector 3-tier memory system |
| `internal/mcp/` | Model Context Protocol bridge |
| `internal/permissions/` | RBAC: admin / operator / viewer |
| `internal/pipeline/` | 8-stage agent pipeline |
| `internal/providers/` | LLM providers (Anthropic, OpenAI-compat, Qwen, Claude CLI) |
| `internal/store/` | Store interfaces + PG + SQLite implementations |
| `internal/tools/` | Tool registry (filesystem, exec, web, MCP, delegate) |
| `internal/tts/` | Back-compat alias package for old import paths |
| `internal/vault/` | Knowledge Vault: wikilinks, hybrid search, FS sync |
| `migrations/` | PostgreSQL migration files |
| `ui/web/` | React SPA (Vite, Tailwind, Radix UI, Zustand) |
| `ui/desktop/` | Wails v2 desktop app (SQLite, embedded gateway) |
---
## TTS Subsystem
### Overview
TTS lives in `internal/audio/` (canonical) with a back-compat alias at `internal/tts/`.
Five providers ship out-of-the-box: OpenAI, ElevenLabs, Edge TTS, MiniMax, Gemini.
### Provider Capabilities Schema
Each provider implements `audio.TTSProvider` and optionally `audio.CapabilitiesProvider`:
```go
type CapabilitiesProvider interface {
Capabilities() ProviderCapabilities
}
type ProviderCapabilities struct {
Params []ParamSchema
CustomFeatures CustomFeatures
}
```
`ParamSchema` carries `Key`, `Type` (string/number/bool/select/textarea), `Label`, `Help`, `Default`,
`Min`/`Max`, `Options` (for select), `DependsOn` (all-of conditions), and `Hidden` flag.
The UI reads `GET /v1/tts/capabilities` and renders per-provider param editors dynamically.
See `docs/tts-provider-capabilities.md` for full schema documentation.
### Providers
| Provider | Package | Voice source | Key param |
|----------|---------|--------------|-----------|
| OpenAI | `internal/audio/openai/` | Static list | `speed`, `response_format`, `instructions` |
| ElevenLabs | `internal/audio/elevenlabs/` | Dynamic (API) | `voiceSettings.*`, `seed`, `language_code` |
| Edge TTS | `internal/audio/edge/` | Static list | `rate`, `pitch`, `volume` |
| MiniMax | `internal/audio/minimax/` | Dynamic (API) | `speed`, `vol`, `pitch`, `emotion`, `audio.*` |
| Gemini | `internal/audio/gemini/` | Static list (30) | multi-speaker, audio tags, 70+ languages |
### Storage: Dual-Read
Tenant TTS config is stored in `system_config` with two complementary strategies:
- **Legacy flat keys** (`tts.provider`, `tts.voice_id`, `tts.api_key`, …) — written by all versions.
- **Params blob** (`tts.<provider>.params` JSON) — written by v3.x+ for per-provider param overrides.
On read, both paths are merged; blob wins on key conflict. This allows gradual migration:
old clients see flat keys; new clients see the full params blob.
### Manager
`audio.Manager` is the central registry:
- `RegisterProvider(p TTSProvider)` — replaces any existing provider by the same name.
- `Primary()` — returns current primary provider name from config.
- `Synthesize(ctx, text, opts)` — delegates to active provider.
- `ListVoices(ctx, opts)` — delegates to `VoiceListProvider` if implemented.
### HTTP Endpoints
| Endpoint | Role | Notes |
|----------|------|-------|
| `POST /v1/tts/synthesize` | operator | Streams WAV response |
| `POST /v1/tts/test-connection` | operator | Ephemeral provider, returns latency |
| `GET /v1/tts/capabilities` | operator | ProviderCapabilities JSON per provider |
| `GET /v1/tts/config` | admin | Tenant TTS config |
| `POST /v1/tts/config` | admin | Save tenant TTS config |
### Gemini Specifics
- Models: `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` (preview).
- Multi-speaker: up to 2 simultaneous speakers, each with distinct voice + name annotation.
- Audio tags: inline `<say-as>` / style directives via bracketed prompts.
- Sentinel errors: `ErrInvalidVoice`, `ErrInvalidModel`, `ErrSpeakerLimit` → HTTP 422 with i18n message.
- SSRF guard: `api_base` override validated by `validateProviderURL()` — blocks 127.0.0.1/localhost.
### i18n
Backend validation errors use `i18n.T(locale, key, args...)` pattern.
Locale is extracted from `Accept-Language` HTTP header by `enrichContext` middleware.
UI param labels/help text live in:
- `ui/web/src/i18n/locales/{en,vi,zh}/tts.json`
- `ui/desktop/frontend/src/i18n/locales/{en,vi,zh}/tts.json`
Parity enforced by `ui/web/src/__tests__/i18n-tts-key-parity.test.ts` (vitest).
---
## Key Conventions
- **Store layer:** Interface-based; PG (`store/pg/`) + SQLite (`store/sqlitestore/`). Raw SQL, `$1/$2` params.
- **Context propagation:** `store.WithLocale`, `store.WithUserID`, `store.WithTenantID`, etc.
- **Security logs:** `slog.Warn("security.*")` for all security events.
- **SSRF prevention:** `validateProviderURL()` in `internal/http/tts_validate.go`.
- **i18n keys:** Add to `internal/i18n/keys.go` + 3 catalog Go files; UI strings in 3 locale JSON dirs.
- **Migrations:** PG (`migrations/`) + SQLite (`store/sqlitestore/schema.go`) — always update both.
+33
View File
@@ -0,0 +1,33 @@
# Project Changelog
Significant changes, features, and fixes in reverse chronological order.
---
## 2026-04-19
### TTS: Gemini provider + ProviderCapabilities schema engine
**Features**
- **Gemini TTS provider** (`internal/audio/gemini/`): supports `gemini-2.5-flash-preview-tts` and `gemini-2.5-pro-preview-tts`. 30 prebuilt voices, 70+ languages, multi-speaker mode (up to 2 simultaneous speakers with distinct voices), audio-tag styling, WAV output via PCM-to-WAV conversion.
- **`ProviderCapabilities` schema** (`internal/audio/capabilities.go`): dynamic per-provider param descriptor. Each provider exposes `Capabilities()` returning `[]ParamSchema` (type, range, default, dependsOn conditions, hidden flag) + `CustomFeatures` flags. UI reads `GET /v1/tts/capabilities` and renders param editors without hard-coded field lists.
- **Dual-read TTS storage**: tenant config read from both legacy flat keys (`tts.provider`, `tts.voice_id`, …) and new params blob (`tts.<provider>.params` JSON). Blob wins on conflict. Allows gradual migration; no data loss on downgrade.
- **`VoiceListProvider` interface** refactor: dynamic voice fetching (ElevenLabs, MiniMax) now via `ListVoices(ctx, ListVoicesOptions)` instead of per-provider ad-hoc methods. Unified `audio.Voice` type.
- **`POST /v1/tts/test-connection`**: ephemeral provider creation from request credentials + short synthesis smoke test. Returns `{ success, latency_ms }`. No provider registration; no config mutation. Operator role required.
- **`GET /v1/tts/capabilities`**: returns `ProviderCapabilities` JSON for all registered providers.
**i18n**
- Backend sentinel error keys (`MsgTtsGeminiInvalidVoice`, `MsgTtsGeminiInvalidModel`, `MsgTtsGeminiSpeakerLimit`, `MsgTtsParamOutOfRange`, `MsgTtsParamDependsOn`, `MsgTtsMiniMaxVoicesFailed`) in all 3 catalogs (EN/VI/ZH).
- HTTP 422 responses for Gemini sentinel errors now use `i18n.T(locale, key, args...)` — locale from `Accept-Language` header.
- ~80 param `label`/`help` keys across web + desktop locale files (EN/VI/ZH); parity enforced by `ui/web/src/__tests__/i18n-tts-key-parity.test.ts`.
**Security**
- SSRF guard on `api_base` override for test-connection (`validateProviderURL()`) — blocks `127.0.0.1` / `localhost` / RFC1918 ranges.
**Docs**
- `docs/tts-provider-capabilities.md` — schema reference + per-provider param tables + storage format + "Adding a new provider" checklist.
- `docs/codebase-summary.md` — TTS subsystem section documenting manager, providers, storage, endpoints.
+361
View File
@@ -0,0 +1,361 @@
# TTS Provider Capabilities Schema
Reference for the `ProviderCapabilities` schema, per-provider parameter tables,
storage format, and the checklist for adding a new TTS provider.
---
## 1. ProviderCapabilities Schema
### Go Types
```go
// ProviderCapabilities describes everything the frontend needs to render
// dynamic per-provider settings and validate user input.
type ProviderCapabilities struct {
Provider string // provider ID: "openai", "gemini", etc.
DisplayName string // human-readable name
RequiresAPIKey bool // show API key field
Models []string // static model list; nil = user freetext
Voices []VoiceOption // static voices; nil = dynamic or none
Params []ParamSchema // ordered list of configurable params
CustomFeatures map[string]any // provider-specific UI flags
}
// ParamSchema describes one configurable parameter.
type ParamSchema struct {
Key string // dot-delimited storage key, e.g. "voice_settings.stability"
Type ParamType // "range" | "integer" | "boolean" | "enum" | "string" | "text"
Label string // English label (UI falls back to this if i18n key missing)
Description string // English help text (optional)
Default any // default value (nil = omit from request)
Min *float64 // range/integer lower bound
Max *float64 // range/integer upper bound
Step *float64 // slider increment
Enum []EnumOption // allowed values for "enum" type
DependsOn []Dependency // show/hide rules
}
// Dependency expresses a show/hide condition on another param's value.
type Dependency struct {
Field string // key of the controlling param
Op string // "eq" (only operator currently supported)
Value string // required value (string-compared after JSON marshal)
}
// EnumOption is one choice inside an enum param.
type EnumOption struct {
Value string
Label string
}
```
### ParamType values
| Constant | Wire value | UI widget |
|---|---|---|
| `ParamTypeRange` | `"range"` | Slider + numeric input |
| `ParamTypeInteger` | `"integer"` | Integer slider / stepper |
| `ParamTypeBoolean` | `"boolean"` | Checkbox / toggle |
| `ParamTypeEnum` | `"enum"` | Select dropdown |
| `ParamTypeString` | `"string"` | Single-line text input |
| `ParamTypeText` | `"text"` | Multi-line textarea |
### DependsOn AND semantics
Multiple `DependsOn` entries are evaluated with **AND** logic: the param is
shown only when **all** conditions are satisfied simultaneously.
```go
// Shown only when format=="mp3":
DependsOn: []audio.Dependency{
{Field: "audio.format", Op: "eq", Value: "mp3"},
}
// Shown only when provider=="gemini" AND mode=="multi":
DependsOn: []audio.Dependency{
{Field: "provider", Op: "eq", Value: "gemini"},
{Field: "mode", Op: "eq", Value: "multi"},
}
```
### CustomFeatures flags
| Key | Type | Meaning |
|---|---|---|
| `multi_speaker` | bool | Provider supports multi-speaker mode (Gemini) |
| `audio_tags` | bool | Provider supports inline audio expression tags (Gemini) |
| `voices_dynamic` | bool | Voices must be fetched at runtime via `VoiceListProvider` (MiniMax) |
### Read-only contract
`Capabilities()` **must** be a pure function — no side effects, no I/O, no
locking. The manager calls it during startup to populate the catalog and again
on each `GET /v1/tts/capabilities` request.
---
## 2. Per-Provider Parameter Tables
### OpenAI (`internal/audio/openai/`)
| Key | Type | Default | Range / Options | DependsOn |
|---|---|---|---|---|
| `speed` | range | 1.0 | 0.25–4.0, step 0.05 | — |
| `response_format` | enum | `"mp3"` | mp3, opus, aac, flac, wav, pcm | — |
| `instructions` | text | `""` | — | `model == "gpt-4o-mini-tts"` |
### ElevenLabs (`internal/audio/elevenlabs/`)
| Key | Type | Default | Range / Options | DependsOn |
|---|---|---|---|---|
| `voice_settings.stability` | range | 0.5 | 0–1, step 0.01 | — |
| `voice_settings.similarity_boost` | range | 0.75 | 0–1, step 0.01 | — |
| `voice_settings.style` | range | 0.0 | 0–1, step 0.01 | — |
| `voice_settings.use_speaker_boost` | boolean | true | — | — |
| `voice_settings.speed` | range | 1.0 | 0.7–1.2, step 0.01 | — |
| `apply_text_normalization` | enum | `""` (auto) | auto, on, off | — |
| `seed` | integer | 0 | 0–2³²−1 | — |
| `optimize_streaming_latency` | range | 0 | 0–4, step 1 | — |
| `language_code` | string | `""` | ISO 639-1 | — |
### Edge TTS (`internal/audio/edge/`)
No API key required. Full voice list available via `edge-tts --list-voices`.
| Key | Type | Default | Range / Options | DependsOn |
|---|---|---|---|---|
| `rate` | integer | 0 | -50–+100 (%) | — |
| `pitch` | integer | 0 | -50–+50 (Hz) | — |
| `volume` | integer | 0 | -50–+100 (%) | — |
### MiniMax (`internal/audio/minimax/`)
Voices are dynamic — fetched at runtime via `VoiceListProvider`.
`CustomFeatures["voices_dynamic"] = true` signals the frontend to call
`GET /v1/voices?provider=minimax`.
| Key | Type | Default | Range / Options | DependsOn |
|---|---|---|---|---|
| `speed` | range | 1.0 | 0.5–2.0, step 0.1 | — |
| `vol` | range | 1.0 | 0.01–10.0, step 0.01 | — |
| `pitch` | integer | 0 | -12–+12 (semitones) | — |
| `emotion` | enum | `""` | none/happy/sad/angry/fearful/disgusted/surprised/neutral/excited/anxious | — |
| `text_normalization` | boolean | nil (omit) | — | — |
| `audio.format` | enum | `"mp3"` | mp3, pcm, flac, wav | — |
| `audio.sample_rate` | enum | `""` (default) | 8000/16000/22050/24000/32000/44100 | — |
| `audio.bitrate` | enum | `""` (default) | 32000/64000/128000/256000 bps | `audio.format == "mp3"` |
| `audio.channel` | enum | `""` (default) | 1=mono, 2=stereo | — |
### Gemini (`internal/audio/gemini/`)
No configurable `Params` — advanced features exposed via `CustomFeatures`.
| Feature | CustomFeatures key | Notes |
|---|---|---|
| Multi-speaker | `multi_speaker: true` | Up to 2 speakers; each has `name` + `voice` |
| Audio tags | `audio_tags: true` | Inline `[laughs]`, `[sighs]`, etc. |
| Preview badge | — | Models are preview-stage; UI shows badge |
Sentinel validation errors (422 Unprocessable Entity):
| Error | i18n key |
|---|---|
| `ErrInvalidVoice` | `MsgTtsGeminiInvalidVoice` |
| `ErrSpeakerLimit` | `MsgTtsGeminiSpeakerLimit` |
| `ErrInvalidModel` | `MsgTtsGeminiInvalidModel` |
All 422 error messages are run through `i18n.T(locale, key, args...)` using the
locale extracted from `Accept-Language` by `requireAuth → enrichContext`.
---
## 3. Storage Format — Dual-Read / Dual-Write
TTS configuration is persisted in `system_configs` (tenant-scoped key-value store).
Two layouts coexist for backward compatibility:
### Legacy flat keys
```
tts.provider → "openai"
tts.openai.api_base → "https://api.openai.com/v1"
tts.openai.voice → "nova"
tts.openai.model → "tts-1"
tts.gemini.speakers → [{"name":"Alice","voice":"Kore"}] (JSON array)
```
### Params blob (Phase C+)
```
tts.<provider>.params → {"speed":1.2,"response_format":"mp3"} (JSON object)
```
### Dual-read logic (`resolveTenantProvider`)
1. Read legacy flat keys to populate `testConnectionRequest` fields.
2. Read `tts.<provider>.params` blob via `loadParamsBlob()`.
3. **Blob wins** on key conflict — blob values override legacy flat keys for the
same logical param.
4. Legacy flat keys fill gaps not covered by the blob.
### Rollback
Reverting Phase C+ is non-destructive: legacy flat keys remain valid; the blob
is simply ignored when `loadParamsBlob` returns `nil`.
---
## 4. Adding a New Provider
Follow these touchpoints in order. Each step is required; skipping any causes
a compile or runtime failure.
### Step 1 — Provider package
Create `internal/audio/<name>/tts.go` implementing `audio.TTSProvider`:
```go
type Provider struct { /* config fields */ }
func (p *Provider) Name() string { return "<name>" }
func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error)
func (p *Provider) Capabilities() audio.ProviderCapabilities
```
Follow the SSRF guard pattern for external HTTP calls (validate `apiBase` via
`validateProviderURL` before creating the HTTP client).
### Step 2 — Capabilities()
Return a fully populated `audio.ProviderCapabilities`.
- Set `RequiresAPIKey: true/false`.
- Populate `Params` with `[]audio.ParamSchema`; use `&float64` literals for
`Min`/`Max`/`Step` (package-level `var` to avoid address-of-literal).
- Defaults **must** match the values your `Synthesize` applies when the param
is absent from `opts.Params`.
- If voices are dynamic, set `CustomFeatures["voices_dynamic"] = true` and
implement `VoiceListProvider` (see Step 6).
### Step 3 — createEphemeralTTSProvider (`internal/http/tts_test_connection.go`)
Add a `case "<name>":` branch:
```go
case "<name>":
return newpkg.NewProvider(newpkg.Config{
APIKey: req.APIKey,
APIBase: req.APIBase,
VoiceID: req.VoiceID,
ModelID: req.ModelID,
TimeoutMs: req.TimeoutMs,
}), nil
```
Also add `"<name>": true` to `supportedTestProviders` and, if an API key is
needed, to `providersRequiringAPIKey`.
### Step 4 — resolveTenantProvider (`internal/http/tts.go`)
Add a `case "<name>":` branch reading the provider's config keys from
`h.systemConfigs` / `h.configSecrets` and calling `loadParamsBlob`.
### Step 5 — Wire sentinel errors (if any)
If the provider defines sentinel errors (like Gemini's `ErrInvalidVoice`),
add `errors.Is` checks in both `handleSynthesize` and `handleTestConnection`,
using `i18n.T(locale, MsgXxx, ...)` for the response body.
### Step 6 — VoiceListProvider (dynamic voices only)
Implement the interface:
```go
type VoiceListProvider interface {
TTSProvider
ListVoices(ctx context.Context) ([]VoiceOption, error)
}
```
The voices handler (`GET /v1/voices?provider=<name>`) uses a type-assertion to
detect this interface at runtime — no registration needed.
### Step 7 — tts_config.go schema validation
If the provider has required fields (e.g. `group_id` for MiniMax), add
validation in `internal/http/tts_config.go` so `PUT /v1/tts/config` returns a
clear 400 when mandatory fields are missing.
### Step 8 — i18n keys
For each `ParamSchema`, add:
1. `tts.<name>.<param_key>.label` and `tts.<name>.<param_key>.help` to all
6 locale JSON files (web × 3 + desktop × 3).
2. Any backend validation error keys to `internal/i18n/keys.go` and all three
catalog files (`catalog_en.go`, `catalog_vi.go`, `catalog_zh.go`).
### Step 9 — Tests
- Unit test `Capabilities()` returns non-empty `Provider` + `DisplayName`.
- Test `createEphemeralTTSProvider` builds the provider without error.
- Test `Synthesize` against a mock HTTP server.
- If sentinel errors exist, test 422 body uses `i18n.T` for all locales
(see `TestSynthesize_GeminiInvalidVoice_I18n` pattern in `tts_test.go`).
### Consumers to check
| File | What to verify |
|---|---|
| `internal/http/tts.go` | `resolveTenantProvider` case |
| `internal/http/tts_test_connection.go` | `createEphemeralTTSProvider` + allow-lists |
| `internal/http/tts_config.go` | Schema validation for required fields |
| `internal/http/voices.go` | `VoiceListProvider` type-assertion path |
| `internal/audio/manager.go` | Auto-discovers via `RegisterProvider`; no change needed |
| `ui/web/src/i18n/locales/*/tts.json` | Param label/help keys (3 locales) |
| `ui/desktop/frontend/src/i18n/locales/*/tts.json` | Desktop locale parity |
---
## 5. Gemini Specifics
### Models
Gemini TTS uses preview models only (as of 2026-04):
- `gemini-2.5-flash-preview-tts`
- `gemini-2.5-pro-preview-tts`
- `gemini-3.1-flash-tts-preview`
The frontend displays a "Preview" badge (i18n key `tts.gemini.previewBadge`).
### Multi-speaker mode
- Up to **2 speakers** per request.
- Each speaker: `{name: string, voice: string}` — voice must be one of the
30 prebuilt Gemini voices.
- Serialized as `tts.gemini.speakers` JSON blob in `system_configs`.
- At synthesis time, `loadSavedSpeakers` restores the blob into `opts.Speakers`.
### Audio tags
Inline expressive markers injected by the user directly into text:
```
Hello [laughs] world [sighs] how are you?
```
Categories: Emotion, Pacing, Effect, Voice quality. Full tag list is hardcoded
in the frontend (`gemini/audio-tags.ts`).
### Language support
Gemini TTS supports 70+ languages automatically based on input text — no
explicit language parameter needed.
### SSRF guard
`api_base` overrides for Gemini (and all providers) are validated by
`validateProviderURL` in `internal/http/tts_url_validator.go`. Private IP
ranges (127.x, 10.x, 192.168.x, etc.) and `localhost` are rejected with a 400
response to prevent Server-Side Request Forgery.
+75
View File
@@ -0,0 +1,75 @@
package edge
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// edgeDefaultVoices lists a representative set of Edge TTS voices.
// Full list is available via `edge-tts --list-voices`; these cover common locales.
var edgeDefaultVoices = []audio.VoiceOption{
{VoiceID: "en-US-MichelleNeural", Name: "Michelle", Language: "en-US", Gender: "Female"},
{VoiceID: "en-US-GuyNeural", Name: "Guy", Language: "en-US", Gender: "Male"},
{VoiceID: "en-GB-SoniaNeural", Name: "Sonia", Language: "en-GB", Gender: "Female"},
{VoiceID: "vi-VN-HoaiMyNeural", Name: "HoaiMy", Language: "vi-VN", Gender: "Female"},
{VoiceID: "vi-VN-NamMinhNeural", Name: "NamMinh", Language: "vi-VN", Gender: "Male"},
{VoiceID: "zh-CN-XiaoxiaoNeural", Name: "Xiaoxiao", Language: "zh-CN", Gender: "Female"},
{VoiceID: "zh-CN-YunxiNeural", Name: "Yunxi", Language: "zh-CN", Gender: "Male"},
}
var (
rateMin = -50.0
rateMax = 100.0
rateStep = 1.0
pitchMin = -50.0
pitchMax = 50.0
pitchStep = 1.0
volumeMin = -50.0
volumeMax = 100.0
volumeStep = 1.0
)
// edgeParams is the enriched param schema for Edge TTS.
// Defaults MUST match the hardcoded tts.go behaviour (no --rate/--pitch/--volume flags = zero).
// UI slider int values; Synthesize converts via sliderToRateString / sliderToPitchString.
var edgeParams = []audio.ParamSchema{
{
Key: "rate",
Type: audio.ParamTypeInteger,
Label: "Rate",
Description: "Speech rate adjustment in percent (-50 to +100).",
Default: 0,
Min: &rateMin,
Max: &rateMax,
Step: &rateStep,
},
{
Key: "pitch",
Type: audio.ParamTypeInteger,
Label: "Pitch",
Description: "Pitch adjustment in Hz (-50 to +50).",
Default: 0,
Min: &pitchMin,
Max: &pitchMax,
Step: &pitchStep,
},
{
Key: "volume",
Type: audio.ParamTypeInteger,
Label: "Volume",
Description: "Volume adjustment in percent (-50 to +100).",
Default: 0,
Min: &volumeMin,
Max: &volumeMax,
Step: &volumeStep,
},
}
// Capabilities returns the full capability schema for the Edge TTS provider.
// Edge TTS requires no API key (uses the free edge-tts CLI).
func (p *Provider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: "edge",
DisplayName: "Edge TTS (Microsoft)",
RequiresAPIKey: false,
Voices: edgeDefaultVoices,
Params: edgeParams,
}
}
@@ -0,0 +1,80 @@
package edge
import (
"context"
"os/exec"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
// TestCharacterization_Edge_DefaultOpts captures the CLI args emitted by
// Synthesize for default opts. This is the golden-fixture test — the exact
// arg shape MUST remain identical after the Synthesize refactor.
//
// Edge TTS has no HTTP body to capture; the "wire format" is the subprocess
// args passed to edge-tts. The characterization fixture is:
// --voice en-US-MichelleNeural --text <text> --write-media <path>
// (no --rate flag when rate is empty/zero-default)
func TestCharacterization_Edge_DefaultOpts(t *testing.T) {
p := NewProvider(Config{}) // empty = defaults
fakeFactory := func(ctx context.Context, name string, args ...string) *exec.Cmd {
return exec.Command("true")
}
allArgs, err := p.synthesizeWithFactory(context.Background(), "hello world", audio.TTSOptions{}, fakeFactory)
if err != nil {
t.Fatalf("synthesizeWithFactory failed: %v", err)
}
// Golden: voice must be the default.
assertArg(t, allArgs, "--voice", "en-US-MichelleNeural")
// Golden: text must be passed verbatim.
assertArg(t, allArgs, "--text", "hello world")
// Golden: no --rate flag when rate is empty.
if hasFlag(allArgs, "--rate") {
t.Error("--rate flag must NOT appear for default (empty rate) opts")
}
}
// TestCharacterization_Edge_WithRate confirms that a non-empty rate value from
// the provider config is forwarded as --rate <value>.
func TestCharacterization_Edge_WithRate(t *testing.T) {
p := NewProvider(Config{Rate: "+10%"})
fakeFactory := func(ctx context.Context, name string, args ...string) *exec.Cmd {
return exec.Command("true")
}
allArgs, err := p.synthesizeWithFactory(context.Background(), "hi", audio.TTSOptions{}, fakeFactory)
if err != nil {
t.Fatalf("synthesizeWithFactory failed: %v", err)
}
assertArg(t, allArgs, "--rate", "+10%")
}
// assertArg checks that flag is immediately followed by want in args.
func assertArg(t *testing.T, args []string, flag, want string) {
t.Helper()
for i, a := range args {
if a == flag && i+1 < len(args) {
if args[i+1] != want {
t.Errorf("%s: got %q, want %q", flag, args[i+1], want)
}
return
}
}
t.Errorf("flag %q not found in args: %v", flag, args)
}
// hasFlag returns true if flag appears anywhere in args.
func hasFlag(args []string, flag string) bool {
for _, a := range args {
if a == flag {
return true
}
}
return false
}
@@ -0,0 +1,61 @@
package edge
import (
"context"
"os/exec"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
// captureEdgeArgs runs synthesizeWithFactory and returns the CLI args.
func captureEdgeArgs(t *testing.T, p *Provider, opts audio.TTSOptions) []string {
t.Helper()
fakeFactory := func(ctx context.Context, name string, args ...string) *exec.Cmd {
return exec.Command("true")
}
args, err := p.synthesizeWithFactory(context.Background(), "hello", opts, fakeFactory)
if err != nil {
t.Fatalf("synthesizeWithFactory failed: %v", err)
}
return args
}
// TestDefaults_PreserveLegacyArgs verifies that populating opts.Params with all
// Capabilities defaults produces identical CLI args to the nil-Params baseline.
func TestDefaults_PreserveLegacyArgs(t *testing.T) {
p := NewProvider(Config{})
caps := p.Capabilities()
if len(caps.Params) == 0 {
t.Skip("Capabilities.Params not yet populated (Phase C enrichment pending)")
}
params := make(map[string]any)
for _, s := range caps.Params {
if s.Default != nil {
audio.SetNested(params, s.Key, s.Default)
}
}
argsWithDefaults := captureEdgeArgs(t, p, audio.TTSOptions{Params: params})
argsNilParams := captureEdgeArgs(t, p, audio.TTSOptions{})
// Compare as joined strings for readable diffs.
want := joinArgs(argsNilParams)
got := joinArgs(argsWithDefaults)
if got != want {
t.Errorf("defaults-invariant FAILED:\n with-defaults: %s\n nil-params: %s", got, want)
}
}
func joinArgs(args []string) string {
result := ""
for i, a := range args {
if i > 0 {
result += " "
}
result += a
}
return result
}
+117
View File
@@ -0,0 +1,117 @@
package edge
import (
"context"
"os/exec"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
// captureEdgeArgsWithOpts is a test helper that captures CLI args via synthesizeWithFactory.
func captureEdgeArgsWithOpts(t *testing.T, p *Provider, opts audio.TTSOptions) []string {
t.Helper()
factory := func(ctx context.Context, name string, args ...string) *exec.Cmd {
return exec.Command("true")
}
args, err := p.synthesizeWithFactory(context.Background(), "test text", opts, factory)
if err != nil {
t.Fatalf("synthesizeWithFactory: %v", err)
}
return args
}
func TestSynthesize_AppliesParams_Rate(t *testing.T) {
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{
Params: map[string]any{"rate": 15},
})
assertArg(t, args, "--rate", "+15%")
}
func TestSynthesize_AppliesParams_NegativeRate(t *testing.T) {
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{
Params: map[string]any{"rate": -25},
})
assertArg(t, args, "--rate", "-25%")
}
func TestSynthesize_AppliesParams_Pitch(t *testing.T) {
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{
Params: map[string]any{"pitch": 5},
})
assertArg(t, args, "--pitch", "+5Hz")
}
func TestSynthesize_AppliesParams_Volume(t *testing.T) {
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{
Params: map[string]any{"volume": 20},
})
assertArg(t, args, "--volume", "+20%")
}
func TestSynthesize_AppliesParams_OmitsZeroRate(t *testing.T) {
// Zero rate/pitch/volume must NOT produce flags (characterization invariant).
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{
Params: map[string]any{"rate": 0, "pitch": 0, "volume": 0},
})
for _, flag := range []string{"--rate", "--pitch", "--volume"} {
if hasFlag(args, flag) {
t.Errorf("%s flag must not appear for zero value", flag)
}
}
}
func TestSynthesize_AppliesParams_NilParamsNoFlags(t *testing.T) {
p := NewProvider(Config{})
args := captureEdgeArgsWithOpts(t, p, audio.TTSOptions{})
for _, flag := range []string{"--rate", "--pitch", "--volume"} {
if hasFlag(args, flag) {
t.Errorf("%s flag must not appear for nil params", flag)
}
}
}
func TestSynthesize_DoesNotMutateCallerParams(t *testing.T) {
p := NewProvider(Config{})
original := map[string]any{
"rate": 10,
"sentinel": "untouched",
}
snapshot := map[string]any{
"rate": original["rate"],
"sentinel": original["sentinel"],
}
captureEdgeArgsWithOpts(t, p, audio.TTSOptions{Params: original})
for k, want := range snapshot {
if got := original[k]; got != want {
t.Errorf("caller Params mutated: key %q was %v, now %v", k, want, got)
}
}
}
func TestCapabilities_HasParam_Rate(t *testing.T) {
p := NewProvider(Config{})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "rate" && param.Type == audio.ParamTypeInteger {
return
}
}
t.Error("rate param (integer) not found in Capabilities")
}
func TestCapabilities_HasParam_Pitch(t *testing.T) {
p := NewProvider(Config{})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "pitch" && param.Type == audio.ParamTypeInteger {
return
}
}
t.Error("pitch param (integer) not found in Capabilities")
}
+47
View File
@@ -0,0 +1,47 @@
package edge
import (
"fmt"
"strconv"
"strings"
)
// sliderToRateString converts an integer slider value to the edge-tts rate
// string format "+N%" or "-N%". Zero becomes "+0%".
func sliderToRateString(n int) string {
if n >= 0 {
return fmt.Sprintf("+%d%%", n)
}
return fmt.Sprintf("%d%%", n)
}
// sliderToPitchString converts an integer slider value to the edge-tts pitch
// string format "+NHz" or "-NHz". Zero becomes "+0Hz".
func sliderToPitchString(n int) string {
if n >= 0 {
return fmt.Sprintf("+%dHz", n)
}
return fmt.Sprintf("%dHz", n)
}
// sliderToVolumeString converts an integer slider value to the edge-tts volume
// string format "+N%" or "-N%". Zero becomes "+0%".
func sliderToVolumeString(n int) string {
return sliderToRateString(n) // same format as rate
}
// rateStringToSlider parses a rate/volume string like "+15%" or "-25%" and
// returns the integer value. Returns 0 on parse failure.
func rateStringToSlider(s string) int {
s = strings.TrimSpace(s)
s = strings.TrimSuffix(s, "%")
s = strings.TrimSuffix(s, "Hz")
if s == "" {
return 0
}
n, err := strconv.Atoi(s)
if err != nil {
return 0
}
return n
}
+114
View File
@@ -0,0 +1,114 @@
package edge
import "testing"
func TestSliderToString_PositivePercent(t *testing.T) {
got := sliderToRateString(15)
if got != "+15%" {
t.Errorf("got %q, want +15%%", got)
}
}
func TestSliderToString_NegativePercent(t *testing.T) {
got := sliderToRateString(-25)
if got != "-25%" {
t.Errorf("got %q, want -25%%", got)
}
}
func TestSliderToString_Zero(t *testing.T) {
got := sliderToRateString(0)
if got != "+0%" {
t.Errorf("got %q, want +0%%", got)
}
}
func TestSliderToString_PitchHz_Positive(t *testing.T) {
got := sliderToPitchString(5)
if got != "+5Hz" {
t.Errorf("got %q, want +5Hz", got)
}
}
func TestSliderToString_PitchHz_Negative(t *testing.T) {
got := sliderToPitchString(-3)
if got != "-3Hz" {
t.Errorf("got %q, want -3Hz", got)
}
}
func TestSliderToString_PitchHz_Zero(t *testing.T) {
got := sliderToPitchString(0)
if got != "+0Hz" {
t.Errorf("got %q, want +0Hz", got)
}
}
func TestStringToSlider_PositivePercent(t *testing.T) {
got := rateStringToSlider("+15%")
if got != 15 {
t.Errorf("got %d, want 15", got)
}
}
func TestStringToSlider_NegativePercent(t *testing.T) {
got := rateStringToSlider("-25%")
if got != -25 {
t.Errorf("got %d, want -25", got)
}
}
func TestStringToSlider_Zero(t *testing.T) {
got := rateStringToSlider("+0%")
if got != 0 {
t.Errorf("got %d, want 0", got)
}
}
func TestStringToSlider_Hz(t *testing.T) {
got := rateStringToSlider("+5Hz")
if got != 5 {
t.Errorf("got %d, want 5", got)
}
}
func TestStringToSlider_NegativeHz(t *testing.T) {
got := rateStringToSlider("-3Hz")
if got != -3 {
t.Errorf("got %d, want -3", got)
}
}
func TestStringToSlider_RoundTrip_Rate(t *testing.T) {
for _, n := range []int{-50, -1, 0, 1, 50, 100} {
s := sliderToRateString(n)
back := rateStringToSlider(s)
if back != n {
t.Errorf("round-trip rate %d → %q → %d", n, s, back)
}
}
}
func TestStringToSlider_RoundTrip_Pitch(t *testing.T) {
for _, n := range []int{-50, -1, 0, 1, 50} {
s := sliderToPitchString(n)
back := rateStringToSlider(s)
if back != n {
t.Errorf("round-trip pitch %d → %q → %d", n, s, back)
}
}
}
func TestStringToSlider_BadInput_Zero(t *testing.T) {
got := rateStringToSlider("abc")
if got != 0 {
t.Errorf("got %d, want 0 for bad input", got)
}
}
func TestStringToSlider_Empty_Zero(t *testing.T) {
got := rateStringToSlider("")
if got != 0 {
t.Errorf("got %d, want 0 for empty input", got)
}
}
+52 -4
View File
@@ -16,14 +16,14 @@ import (
// Config configures the Edge TTS provider.
type Config struct {
Voice string // default "en-US-MichelleNeural"
Rate string // speech rate, e.g. "+0%"
Rate string // speech rate, e.g. "+0%"; overridden by opts.Params["rate"]
TimeoutMs int
}
// Provider implements audio.TTSProvider via the edge-tts CLI.
type Provider struct {
voice string
rate string
rate string // legacy config-level rate string; opts.Params["rate"] takes precedence
timeoutMs int
}
@@ -49,6 +49,13 @@ func (p *Provider) Name() string { return "edge" }
// Synthesize shells out to edge-tts. Output is always MP3
// (edge-tts default format: audio-24khz-48kbitrate-mono-mp3).
// opts.Voice overrides the construction-time voice when non-empty.
// opts.Params keys (integer slider values):
// - "rate" int (-50..+100) → "--rate +N%"
// - "pitch" int (-50..+50) → "--pitch +NHz"
// - "volume" int (-50..+100) → "--volume +N%"
//
// Zero value for rate/pitch/volume means no flag is added (preserves legacy behaviour).
// MUST NOT mutate opts.Params.
func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error) {
tmpDir := os.TempDir()
outPath := filepath.Join(tmpDir, fmt.Sprintf("tts-%d.mp3", time.Now().UnixNano()))
@@ -59,13 +66,25 @@ func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOp
voice = opts.Voice
}
// Resolve rate/pitch/volume from opts.Params (int slider).
// When Params is nil or key absent, fall back to legacy p.rate string if set.
rateStr := resolveEdgeParam(opts.Params, "rate", sliderToRateString, p.rate)
pitchStr := resolveEdgeParam(opts.Params, "pitch", sliderToPitchString, "")
volumeStr := resolveEdgeParam(opts.Params, "volume", sliderToVolumeString, "")
args := []string{
"--voice", voice,
"--text", text,
"--write-media", outPath,
}
if p.rate != "" {
args = append(args, "--rate", p.rate)
if rateStr != "" && rateStr != "+0%" {
args = append(args, "--rate", rateStr)
}
if pitchStr != "" && pitchStr != "+0Hz" {
args = append(args, "--pitch", pitchStr)
}
if volumeStr != "" && volumeStr != "+0%" {
args = append(args, "--volume", volumeStr)
}
timeout := time.Duration(p.timeoutMs) * time.Millisecond
@@ -88,3 +107,32 @@ func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOp
MimeType: "audio/mpeg",
}, nil
}
// resolveEdgeParam reads an integer slider value from params and converts it
// to the edge-tts string format using conv. Falls back to legacyStr when the
// key is absent from params or params is nil. Returns "" when both sources
// are absent.
func resolveEdgeParam(params map[string]any, key string, conv func(int) string, legacyStr string) string {
if params != nil {
if v, ok := audio.GetNested(params, key); ok {
n := toInt(v)
return conv(n)
}
}
return legacyStr
}
// toInt converts a numeric any value to int. Returns 0 for unknown types.
func toInt(v any) int {
switch n := v.(type) {
case int:
return n
case int64:
return int(n)
case float64:
return int(n)
case float32:
return int(n)
}
return 0
}
+17 -7
View File
@@ -21,7 +21,8 @@ func init() {
}
// synthesizeWithFactory is used by tests to inject a fake command factory.
// It mirrors Synthesize but accepts a factory func for the exec call.
// It mirrors Synthesize but accepts a factory func for the exec call and
// returns the args without actually running edge-tts.
func (p *Provider) synthesizeWithFactory(
ctx context.Context,
text string,
@@ -33,13 +34,24 @@ func (p *Provider) synthesizeWithFactory(
voice = opts.Voice
}
// Mirror the params resolution in Synthesize.
rateStr := resolveEdgeParam(opts.Params, "rate", sliderToRateString, p.rate)
pitchStr := resolveEdgeParam(opts.Params, "pitch", sliderToPitchString, "")
volumeStr := resolveEdgeParam(opts.Params, "volume", sliderToVolumeString, "")
args := []string{
"--voice", voice,
"--text", text,
"--write-media", "/dev/null",
}
if p.rate != "" {
args = append(args, "--rate", p.rate)
if rateStr != "" && rateStr != "+0%" {
args = append(args, "--rate", rateStr)
}
if pitchStr != "" && pitchStr != "+0Hz" {
args = append(args, "--pitch", pitchStr)
}
if volumeStr != "" && volumeStr != "+0%" {
args = append(args, "--volume", volumeStr)
}
// Return args without actually running: caller asserts them.
@@ -55,7 +67,6 @@ func TestSynthesize_HonorsOptsVoice(t *testing.T) {
var gotArgs []string
fakeFactory := func(ctx context.Context, name string, args ...string) *exec.Cmd {
gotArgs = append([]string{name}, args...)
// Return a no-op command so the test never shells out.
return exec.Command("true")
}
@@ -64,9 +75,8 @@ func TestSynthesize_HonorsOptsVoice(t *testing.T) {
if err != nil {
t.Fatalf("synthesizeWithFactory returned error: %v", err)
}
_ = gotArgs // populated by factory
_ = gotArgs
// Locate --voice arg in allArgs.
voiceVal := ""
for i, a := range allArgs {
if a == "--voice" && i+1 < len(allArgs) {
@@ -88,7 +98,7 @@ func TestSynthesize_FallsBackToProviderVoice(t *testing.T) {
return exec.Command("true")
}
opts := audio.TTSOptions{} // empty Voice
opts := audio.TTSOptions{}
allArgs, err := p.synthesizeWithFactory(context.Background(), "hello", opts, fakeFactory)
if err != nil {
t.Fatalf("synthesizeWithFactory returned error: %v", err)
+134
View File
@@ -0,0 +1,134 @@
package elevenlabs
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// elevenLabsModels lists all allowed ElevenLabs TTS model IDs (mirrors AllowedElevenLabsModels).
var elevenLabsModels = []string{
"eleven_v3",
"eleven_flash_v2_5",
"eleven_multilingual_v2",
"eleven_turbo_v2_5",
}
var (
stabilityMin = 0.0
stabilityMax = 1.0
stabilityStep = 0.01
simBoostMin = 0.0
simBoostMax = 1.0
simBoostStep = 0.01
styleMin = 0.0
styleMax = 1.0
styleStep = 0.01
elSpeedMin = 0.7
elSpeedMax = 1.2
elSpeedStep = 0.01
latencyMin = 0.0
latencyMax = 4.0
latencyStep = 1.0
seedMin = 0.0
seedMax = 4294967295.0 // 2^32-1
seedStep = 1.0
)
// elevenLabsParams is the enriched param schema for ElevenLabs TTS.
// Defaults MUST match the hardcoded values in tts.go:54-59 (characterization fixture):
// stability=0.5, similarity_boost=0.75, style=0.0, use_speaker_boost=true, speed=1.0
var elevenLabsParams = []audio.ParamSchema{
{
Key: "voice_settings.stability",
Type: audio.ParamTypeRange,
Label: "Stability",
Description: "Voice stability (0 = more variable, 1 = more consistent).",
Default: 0.5,
Min: &stabilityMin,
Max: &stabilityMax,
Step: &stabilityStep,
},
{
Key: "voice_settings.similarity_boost",
Type: audio.ParamTypeRange,
Label: "Similarity Boost",
Description: "Similarity to original voice (0 = more creative, 1 = more similar).",
Default: 0.75,
Min: &simBoostMin,
Max: &simBoostMax,
Step: &simBoostStep,
},
{
Key: "voice_settings.style",
Type: audio.ParamTypeRange,
Label: "Style Exaggeration",
Description: "Style exaggeration amount (0 = none, 1 = maximum).",
Default: 0.0,
Min: &styleMin,
Max: &styleMax,
Step: &styleStep,
},
{
Key: "voice_settings.use_speaker_boost",
Type: audio.ParamTypeBoolean,
Label: "Speaker Boost",
Default: true,
},
{
Key: "voice_settings.speed",
Type: audio.ParamTypeRange,
Label: "Speed",
Description: "Speech speed (0.7 = slowest, 1.2 = fastest).",
Default: 1.0,
Min: &elSpeedMin,
Max: &elSpeedMax,
Step: &elSpeedStep,
},
{
Key: "apply_text_normalization",
Type: audio.ParamTypeEnum,
Label: "Text Normalization",
Default: "", // empty = omit from request (API defaults to "auto" server-side)
Enum: []audio.EnumOption{
{Value: "", Label: "Auto (default)"},
{Value: "on", Label: "On"},
{Value: "off", Label: "Off"},
},
},
{
Key: "seed",
Type: audio.ParamTypeInteger,
Label: "Seed",
Description: "Deterministic seed for reproducible output (0 = random). Advanced.",
Default: 0,
Min: &seedMin,
Max: &seedMax,
Step: &seedStep,
},
{
Key: "optimize_streaming_latency",
Type: audio.ParamTypeRange,
Label: "Streaming Latency Optimization",
Description: "Latency optimization level (0 = none, 4 = maximum). Advanced.",
Default: 0.0,
Min: &latencyMin,
Max: &latencyMax,
Step: &latencyStep,
},
{
Key: "language_code",
Type: audio.ParamTypeString,
Label: "Language Code",
Description: "ISO 639-1 language hint (e.g. 'en', 'vi'). Advanced.",
Default: "",
},
}
// Capabilities returns the full capability schema for the ElevenLabs TTS provider.
// Voices is nil — ElevenLabs voices are user-specific and fetched dynamically via /v1/voices.
func (p *TTSProvider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: "elevenlabs",
DisplayName: "ElevenLabs TTS",
RequiresAPIKey: true,
Models: elevenLabsModels,
Params: elevenLabsParams,
}
}
@@ -0,0 +1,116 @@
package elevenlabs_test
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
)
// captureElevenLabsBody sends a Synthesize request to a mock server and
// returns the decoded JSON body map + the URL path used.
func captureElevenLabsBody(t *testing.T, cfg elevenlabs.Config, opts audio.TTSOptions) (map[string]any, string) {
t.Helper()
var captured []byte
var capturedPath string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
capturedPath = r.URL.String()
b, err := io.ReadAll(r.Body)
if err != nil {
t.Errorf("read body: %v", err)
}
captured = b
w.Header().Set("Content-Type", "audio/mpeg")
w.Write([]byte("AUDIO"))
}))
t.Cleanup(srv.Close)
cfg.BaseURL = srv.URL
p := elevenlabs.NewTTSProvider(cfg)
_, err := p.Synthesize(t.Context(), "hello", opts)
if err != nil {
t.Fatalf("Synthesize failed: %v", err)
}
var m map[string]any
if err := json.Unmarshal(captured, &m); err != nil {
t.Fatalf("body not valid JSON: %v — body: %s", err, captured)
}
return m, capturedPath
}
// TestCharacterization_ElevenLabs_DefaultOpts is the golden-fixture test.
// It captures the exact voice_settings shape that the current hardcoded tts.go emits.
// These values MUST remain byte-compatible after the Synthesize refactor.
func TestCharacterization_ElevenLabs_DefaultOpts(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "test-key", VoiceID: "test-voice"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{})
// Assert top-level fields.
if v, _ := body["text"].(string); v != "hello" {
t.Errorf("text: got %q, want %q", v, "hello")
}
// Assert voice_settings defaults — these are the hardcoded values at tts.go:54-59.
vs, ok := body["voice_settings"].(map[string]any)
if !ok {
t.Fatalf("voice_settings missing or not an object: %#v", body["voice_settings"])
}
assertFloat(t, vs, "stability", 0.5)
assertFloat(t, vs, "similarity_boost", 0.75)
assertFloat(t, vs, "style", 0.0)
assertBool(t, vs, "use_speaker_boost", true)
}
// TestCharacterization_ElevenLabs_DefaultOutputFormat verifies the default
// output_format query param is mp3_44100_128 when Format is empty.
func TestCharacterization_ElevenLabs_DefaultOutputFormat(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "test-key", VoiceID: "my-voice"}
_, path := captureElevenLabsBody(t, cfg, audio.TTSOptions{})
if !containsStr(path, "output_format=mp3_44100_128") {
t.Errorf("expected output_format=mp3_44100_128 in URL, got: %s", path)
}
if !containsStr(path, "my-voice") {
t.Errorf("expected voice ID in URL path, got: %s", path)
}
}
func assertFloat(t *testing.T, m map[string]any, key string, want float64) {
t.Helper()
v, ok := m[key]
if !ok {
t.Errorf("missing field %q", key)
return
}
got, ok := v.(float64)
if !ok {
t.Errorf("field %q: expected float64, got %T (%v)", key, v, v)
return
}
if got != want {
t.Errorf("field %q: got %v, want %v", key, got, want)
}
}
func assertBool(t *testing.T, m map[string]any, key string, want bool) {
t.Helper()
v, ok := m[key]
if !ok {
t.Errorf("missing field %q", key)
return
}
got, ok := v.(bool)
if !ok {
t.Errorf("field %q: expected bool, got %T (%v)", key, v, v)
return
}
if got != want {
t.Errorf("field %q: got %v, want %v", key, got, want)
}
}
@@ -0,0 +1,59 @@
package elevenlabs_test
import (
"encoding/json"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
)
// TestDefaults_PreserveLegacyBody verifies that populating opts.Params with
// all Capabilities defaults produces a body byte-equivalent to the nil-Params
// characterization fixture. Guards against silent default flip on save.
func TestDefaults_PreserveLegacyBody(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "test-key", VoiceID: "test-voice"}
p := elevenlabs.NewTTSProvider(cfg)
caps := p.Capabilities()
if len(caps.Params) == 0 {
t.Skip("Capabilities.Params not yet populated (Phase C enrichment pending)")
}
// Build params map from all schema defaults (nested-key aware).
params := make(map[string]any)
for _, s := range caps.Params {
if s.Default != nil {
audio.SetNested(params, s.Key, s.Default)
}
}
bodyWithDefaults, pathWithDefaults := captureElevenLabsBody(t, cfg, audio.TTSOptions{Params: params})
bodyNilParams, pathNilParams := captureElevenLabsBody(t, cfg, audio.TTSOptions{})
// Re-encode both maps to canonical JSON for deterministic comparison.
wantJSON := canonicalJSON(t, bodyNilParams)
gotJSON := canonicalJSON(t, bodyWithDefaults)
if gotJSON != wantJSON {
t.Errorf("defaults-invariant body FAILED:\n with-defaults: %s\n nil-params: %s",
gotJSON, wantJSON)
}
// URL (output_format query param) must also be identical.
if pathWithDefaults != pathNilParams {
t.Errorf("defaults-invariant URL FAILED:\n with-defaults: %s\n nil-params: %s",
pathWithDefaults, pathNilParams)
}
}
// canonicalJSON re-marshals a decoded map to produce a deterministic JSON
// string suitable for equality comparison.
func canonicalJSON(t *testing.T, m map[string]any) string {
t.Helper()
b, err := json.Marshal(m)
if err != nil {
t.Fatalf("canonicalJSON marshal: %v", err)
}
return string(b)
}
+127
View File
@@ -0,0 +1,127 @@
package elevenlabs_test
import (
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
)
func TestSynthesize_AppliesParams_NestedStability(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{
Params: map[string]any{
"voice_settings": map[string]any{"stability": 0.3},
},
})
vs, _ := body["voice_settings"].(map[string]any)
if vs == nil {
t.Fatal("voice_settings missing")
}
assertFloat(t, vs, "stability", 0.3)
// Other defaults must be present.
assertFloat(t, vs, "similarity_boost", 0.75)
assertBool(t, vs, "use_speaker_boost", true)
}
func TestSynthesize_AppliesParams_Speed(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{
Params: map[string]any{
"voice_settings": map[string]any{"speed": 1.1},
},
})
vs, _ := body["voice_settings"].(map[string]any)
assertFloat(t, vs, "speed", 1.1)
}
func TestSynthesize_AppliesParams_UseSpeakerBoostFalse(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{
Params: map[string]any{
"voice_settings": map[string]any{"use_speaker_boost": false},
},
})
vs, _ := body["voice_settings"].(map[string]any)
assertBool(t, vs, "use_speaker_boost", false)
}
func TestSynthesize_AppliesParams_TextNormalization(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"apply_text_normalization": "off"},
})
if v, _ := body["apply_text_normalization"].(string); v != "off" {
t.Errorf("apply_text_normalization: got %q, want off", body["apply_text_normalization"])
}
}
func TestSynthesize_AppliesParams_OmitsEmpty_NilParams(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
body, _ := captureElevenLabsBody(t, cfg, audio.TTSOptions{})
// apply_text_normalization must be absent when not set (API defaults server-side).
if _, ok := body["apply_text_normalization"]; ok {
t.Error("apply_text_normalization must not appear for nil params")
}
}
func TestSynthesize_DoesNotMutateCallerParams(t *testing.T) {
cfg := elevenlabs.Config{APIKey: "k", VoiceID: "v"}
original := map[string]any{
"voice_settings": map[string]any{"stability": 0.5},
"sentinel": "untouched",
}
origVS := original["voice_settings"].(map[string]any)
origStability := origVS["stability"]
captureElevenLabsBody(t, cfg, audio.TTSOptions{Params: original})
if original["sentinel"] != "untouched" {
t.Error("caller Params sentinel was mutated")
}
if got := origVS["stability"]; got != origStability {
t.Errorf("caller voice_settings.stability mutated: was %v, now %v", origStability, got)
}
}
func TestCapabilities_HasParam_Stability(t *testing.T) {
p := elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "voice_settings.stability" {
if param.Default != 0.5 {
t.Errorf("stability default: got %v, want 0.5", param.Default)
}
return
}
}
t.Error("voice_settings.stability param not found")
}
func TestCapabilities_HasParam_SimilarityBoost(t *testing.T) {
p := elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "voice_settings.similarity_boost" {
if param.Default != 0.75 {
t.Errorf("similarity_boost default: got %v, want 0.75", param.Default)
}
return
}
}
t.Error("voice_settings.similarity_boost param not found")
}
func TestCapabilities_HasParam_UseSpeakerBoost_DefaultTrue(t *testing.T) {
p := elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "voice_settings.use_speaker_boost" {
if param.Default != true {
t.Errorf("use_speaker_boost default: got %v, want true", param.Default)
}
return
}
}
t.Error("voice_settings.use_speaker_boost param not found")
}
+119 -9
View File
@@ -29,6 +29,18 @@ func (p *TTSProvider) Name() string { return "elevenlabs" }
// Synthesize converts text to audio. Opts.Voice/Opts.Model override the
// configured defaults; Opts.Format="opus" switches to Ogg Opus output.
// opts.Params keys (nested dot-path):
// - "voice_settings.stability" float64 (default 0.5)
// - "voice_settings.similarity_boost" float64 (default 0.75)
// - "voice_settings.style" float64 (default 0.0)
// - "voice_settings.use_speaker_boost" bool (default true)
// - "voice_settings.speed" float64 (default 1.0)
// - "apply_text_normalization" string (default "auto")
// - "seed" int (default 0 = omitted)
// - "optimize_streaming_latency" int (query param, default 0 = omitted)
// - "language_code" string (query param, default "" = omitted)
//
// MUST NOT mutate opts.Params — reads only.
func (p *TTSProvider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error) {
voiceID := opts.Voice
if voiceID == "" {
@@ -48,22 +60,49 @@ func (p *TTSProvider) Synthesize(ctx context.Context, text string, opts audio.TT
outputFormat, ext, mime = "opus_48000_64", "ogg", "audio/ogg"
}
body := map[string]any{
"text": text,
"model_id": modelID,
"voice_settings": map[string]any{
"stability": 0.5,
"similarity_boost": 0.75,
"style": 0.0,
"use_speaker_boost": true,
},
// Override output format from params if provided (27 codecs supported).
if pf := resolveELString(opts.Params, "output_format", ""); pf != "" {
outputFormat = pf
}
// Build voice_settings from params, falling back to hardcoded defaults.
// Defaults MUST match characterization fixture (stability=0.5, etc.).
voiceSettings := map[string]any{
"stability": resolveELFloat(opts.Params, "voice_settings.stability", 0.5),
"similarity_boost": resolveELFloat(opts.Params, "voice_settings.similarity_boost", 0.75),
"style": resolveELFloat(opts.Params, "voice_settings.style", 0.0),
"use_speaker_boost": resolveELBool(opts.Params, "voice_settings.use_speaker_boost", true),
"speed": resolveELFloat(opts.Params, "voice_settings.speed", 1.0),
}
body := map[string]any{
"text": text,
"model_id": modelID,
"voice_settings": voiceSettings,
}
// Optional top-level params.
if norm := resolveELString(opts.Params, "apply_text_normalization", ""); norm != "" {
body["apply_text_normalization"] = norm
}
if seed := resolveELInt(opts.Params, "seed", -1); seed > 0 {
body["seed"] = seed
}
bodyJSON, err := json.Marshal(body)
if err != nil {
return nil, fmt.Errorf("marshal elevenlabs tts request: %w", err)
}
// Build path with query params.
path := fmt.Sprintf("/v1/text-to-speech/%s?output_format=%s", voiceID, outputFormat)
if latency := resolveELInt(opts.Params, "optimize_streaming_latency", -1); latency > 0 {
path += fmt.Sprintf("&optimize_streaming_latency=%d", latency)
}
if lang := resolveELString(opts.Params, "language_code", ""); lang != "" {
path += "&language_code=" + lang
}
audioBytes, err := p.c.postJSON(ctx, path, bodyJSON, 0)
if err != nil {
return nil, fmt.Errorf("elevenlabs tts: %w", err)
@@ -75,3 +114,74 @@ func (p *TTSProvider) Synthesize(ctx context.Context, text string, opts audio.TT
MimeType: mime,
}, nil
}
// resolveELFloat reads a float param from params map via nested key, falling back to def.
func resolveELFloat(params map[string]any, key string, def float64) float64 {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
switch n := v.(type) {
case float64:
return n
case float32:
return float64(n)
case int:
return float64(n)
}
return def
}
// resolveELBool reads a bool param from params map via nested key, falling back to def.
func resolveELBool(params map[string]any, key string, def bool) bool {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
if b, ok := v.(bool); ok {
return b
}
return def
}
// resolveELString reads a string param from params map via nested key, falling back to def.
func resolveELString(params map[string]any, key, def string) string {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
if s, ok := v.(string); ok {
return s
}
return def
}
// resolveELInt reads an int param from params map via nested key.
// Returns def (-1 by convention for "not set") when absent.
func resolveELInt(params map[string]any, key string, def int) int {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
switch n := v.(type) {
case int:
return n
case int64:
return int(n)
case float64:
return int(n)
}
return def
}
+20
View File
@@ -0,0 +1,20 @@
package gemini
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// Capabilities returns the static capability schema for the Gemini TTS provider.
// CustomFeatures drives frontend custom slot rendering in Phase C.
func (p *Provider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: "gemini",
DisplayName: "Google Gemini TTS",
RequiresAPIKey: true,
Models: geminiModels,
Voices: geminiVoices,
// Params is nil — Gemini TTS has no speed/pitch knobs.
CustomFeatures: map[string]any{
"multi_speaker": true,
"audio_tags": true,
},
}
}
@@ -0,0 +1,49 @@
package gemini
import "testing"
func TestCapabilities_Voices(t *testing.T) {
p := NewProvider(Config{APIKey: "k"})
caps := p.Capabilities()
if len(caps.Voices) != 30 {
t.Errorf("want 30 voices, got %d", len(caps.Voices))
}
}
func TestCapabilities_Models(t *testing.T) {
p := NewProvider(Config{APIKey: "k"})
caps := p.Capabilities()
if len(caps.Models) != 3 {
t.Errorf("want 3 models, got %d", len(caps.Models))
}
}
func TestCapabilities_CustomFeatures(t *testing.T) {
p := NewProvider(Config{APIKey: "k"})
caps := p.Capabilities()
if caps.CustomFeatures == nil {
t.Fatal("CustomFeatures is nil")
}
if _, ok := caps.CustomFeatures["multi_speaker"]; !ok {
t.Error("CustomFeatures missing 'multi_speaker'")
}
if _, ok := caps.CustomFeatures["audio_tags"]; !ok {
t.Error("CustomFeatures missing 'audio_tags'")
}
}
func TestCapabilities_RequiresAPIKey(t *testing.T) {
p := NewProvider(Config{APIKey: "k"})
caps := p.Capabilities()
if !caps.RequiresAPIKey {
t.Error("want RequiresAPIKey=true")
}
}
func TestCapabilities_ParamsEmpty(t *testing.T) {
p := NewProvider(Config{APIKey: "k"})
caps := p.Capabilities()
if caps.Params != nil {
t.Errorf("want Params nil (no speed/pitch), got %v", caps.Params)
}
}
+61
View File
@@ -0,0 +1,61 @@
package gemini
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"strings"
"time"
)
const defaultAPIBase = "https://generativelanguage.googleapis.com"
// client handles HTTP communication with the Gemini generateContent API.
type client struct {
apiKey string
apiBase string
timeoutMs int
}
func newClient(apiKey, apiBase string, timeoutMs int) *client {
base := strings.TrimRight(apiBase, "/")
if base == "" {
base = defaultAPIBase
}
if timeoutMs <= 0 {
timeoutMs = 30000
}
return &client{apiKey: apiKey, apiBase: base, timeoutMs: timeoutMs}
}
// buildURL constructs the generateContent endpoint URL for a given model.
func buildURL(base, model string) string {
base = strings.TrimRight(base, "/")
return base + "/v1beta/models/" + model + ":generateContent"
}
// post sends a JSON body to the Gemini API and returns the raw response bytes.
func (c *client) post(ctx context.Context, model string, body []byte) ([]byte, int, error) {
url := buildURL(c.apiBase, model)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return nil, 0, fmt.Errorf("gemini: create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("x-goog-api-key", c.apiKey)
hc := &http.Client{Timeout: time.Duration(c.timeoutMs) * time.Millisecond}
resp, err := hc.Do(req)
if err != nil {
return nil, 0, fmt.Errorf("gemini: request failed: %w", err)
}
defer resp.Body.Close()
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, resp.StatusCode, fmt.Errorf("gemini: read response: %w", err)
}
return respBody, resp.StatusCode, nil
}
+23
View File
@@ -0,0 +1,23 @@
package gemini
import "testing"
// TestBuildURL_AppendsModelAndAction verifies the URL is constructed correctly.
func TestBuildURL_AppendsModelAndAction(t *testing.T) {
base := "https://generativelanguage.googleapis.com"
model := "gemini-2.5-flash-preview-tts"
got := buildURL(base, model)
want := "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-preview-tts:generateContent"
if got != want {
t.Errorf("buildURL = %q, want %q", got, want)
}
}
// TestBuildURL_TrimsTrailingSlash verifies trailing slash on base is stripped.
func TestBuildURL_TrimsTrailingSlash(t *testing.T) {
got := buildURL("https://generativelanguage.googleapis.com/", "gemini-2.5-pro-preview-tts")
want := "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro-preview-tts:generateContent"
if got != want {
t.Errorf("buildURL = %q, want %q", got, want)
}
}
+23
View File
@@ -0,0 +1,23 @@
package gemini
import "errors"
// Sentinel errors returned by Provider.Synthesize.
// Callers should use errors.Is to distinguish these from generic upstream errors.
var (
// ErrInvalidVoice is returned when the requested voice is not in the Gemini catalog.
ErrInvalidVoice = errors.New("gemini: invalid voice")
// ErrSpeakerLimit is returned when more than 2 speakers are requested.
// Gemini TTS multi-speaker mode supports at most 2 speakers.
ErrSpeakerLimit = errors.New("gemini: speaker limit exceeded (max 2)")
// ErrInvalidModel is returned when the requested model is not in the allowlist.
ErrInvalidModel = errors.New("gemini: invalid model")
// errTransientNoAudio is an internal sentinel for non-deterministic Gemini
// failures where the API returns 200 OK but no audio (typically
// finishReason=OTHER). These are flaky on the preview TTS endpoints and
// usually succeed on a single retry.
errTransientNoAudio = errors.New("gemini: transient no-audio response")
)
+19
View File
@@ -0,0 +1,19 @@
package gemini
import "slices"
// geminiModels is the static catalog of supported Gemini TTS model IDs.
// Source: https://ai.google.dev/gemini-api/docs/speech-generation (April 2026)
var geminiModels = []string{
"gemini-3.1-flash-tts-preview",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
}
// defaultModel is the model used when none is specified.
const defaultModel = "gemini-2.5-flash-preview-tts"
// isValidModel reports whether id is in the static model catalog.
func isValidModel(id string) bool {
return slices.Contains(geminiModels, id)
}
+246
View File
@@ -0,0 +1,246 @@
package gemini
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"time"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
// Config bundles credentials and TTS defaults for Google Gemini.
type Config struct {
APIKey string
APIBase string // custom endpoint (optional); must pass validateProviderURL
Voice string // default "Kore"
Model string // default "gemini-2.5-flash-preview-tts"
TimeoutMs int // default 30000
}
// Provider implements audio.TTSProvider and audio.DescribableProvider for Gemini.
type Provider struct {
cfg Config
c *client
}
// NewProvider constructs a Gemini TTS provider with defaults applied.
func NewProvider(cfg Config) *Provider {
if cfg.Voice == "" {
cfg.Voice = defaultVoice
}
if cfg.Model == "" {
cfg.Model = defaultModel
}
return &Provider{
cfg: cfg,
c: newClient(cfg.APIKey, cfg.APIBase, cfg.TimeoutMs),
}
}
// Name returns the stable provider identifier.
func (p *Provider) Name() string { return "gemini" }
// Synthesize converts text to WAV audio via the Gemini generateContent API.
// When opts.Speakers is non-empty, multi-speaker mode is used; otherwise single-voice.
// MUST NOT mutate opts.Params or opts.Speakers — reads only.
func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error) {
voice := opts.Voice
if voice == "" {
voice = p.cfg.Voice
}
model := opts.Model
if model == "" {
model = p.cfg.Model
}
// Validate model.
if !isValidModel(model) {
return nil, fmt.Errorf("%w: %s", ErrInvalidModel, model)
}
// Build speechConfig — multi-speaker and single-voice are mutually exclusive.
var speechConfig map[string]any
if len(opts.Speakers) > 0 {
if len(opts.Speakers) > 2 {
return nil, fmt.Errorf("%w: requested %d", ErrSpeakerLimit, len(opts.Speakers))
}
// Defensive copy to satisfy no-mutate contract.
speakers := make([]audio.SpeakerVoice, len(opts.Speakers))
copy(speakers, opts.Speakers)
configs := make([]map[string]any, len(speakers))
for i, s := range speakers {
configs[i] = map[string]any{
"speaker": s.Speaker,
"voiceConfig": map[string]any{
"prebuiltVoiceConfig": map[string]any{
"voiceName": s.VoiceID,
},
},
}
}
speechConfig = map[string]any{
"multiSpeakerVoiceConfig": map[string]any{
"speakerVoiceConfigs": configs,
},
}
} else {
// Validate voice against static catalog (non-empty voice only — empty falls back to default).
if voice != "" && !isValidVoice(voice) {
return nil, fmt.Errorf("%w: %s", ErrInvalidVoice, voice)
}
speechConfig = map[string]any{
"voiceConfig": map[string]any{
"prebuiltVoiceConfig": map[string]any{
"voiceName": voice,
},
},
}
}
// Per Gemini generateContent spec, speechConfig is NESTED under
// generationConfig — not a top-level field. Sending it at root returns
// 400 "Unknown name 'speechConfig': Cannot find field."
reqBody := map[string]any{
"contents": []map[string]any{
{"parts": []map[string]any{{"text": text}}},
},
"generationConfig": map[string]any{
"responseModalities": []string{"AUDIO"},
"speechConfig": speechConfig,
},
}
bodyBytes, err := json.Marshal(reqBody)
if err != nil {
return nil, fmt.Errorf("gemini: marshal request: %w", err)
}
// Single retry on transient no-audio responses (finishReason=OTHER) — the
// preview TTS endpoint is flaky and usually succeeds on the second try.
// Anything else (auth, rate limit, safety, invalid model) is returned as-is.
res, err := p.requestAudio(ctx, model, bodyBytes)
if err != nil && errors.Is(err, errTransientNoAudio) {
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(retryBackoff):
}
res, err = p.requestAudio(ctx, model, bodyBytes)
}
return res, err
}
// retryBackoff is the delay before retrying a transient Gemini failure. Short
// because the user is waiting in the UI and the preview API usually succeeds
// immediately on retry.
const retryBackoff = 500 * time.Millisecond
// requestAudio executes one POST + parse cycle. Returns errTransientNoAudio
// (wrapped with a descriptive message) when finishReason indicates a flaky
// no-audio outcome that the caller may want to retry.
func (p *Provider) requestAudio(ctx context.Context, model string, bodyBytes []byte) (*audio.SynthResult, error) {
respBytes, status, err := p.c.post(ctx, model, bodyBytes)
if err != nil {
return nil, err
}
switch status {
case http.StatusUnauthorized:
return nil, fmt.Errorf("gemini: auth error (401) — check api key")
case http.StatusTooManyRequests:
return nil, fmt.Errorf("gemini: rate limit exceeded (429)")
}
if status != http.StatusOK {
return nil, fmt.Errorf("gemini: unexpected status %d: %s", status, string(respBytes))
}
// Parse response: candidates[0].content.parts[0].inlineData.data (base64 PCM).
// Also parse finishReason + promptFeedback so we can surface a useful error
// when audio is missing (safety filter, prompt block, text-only response, etc).
var apiResp struct {
Candidates []struct {
Content struct {
Parts []struct {
Text string `json:"text,omitempty"`
InlineData *struct {
Data string `json:"data"`
MimeType string `json:"mimeType,omitempty"`
} `json:"inlineData,omitempty"`
} `json:"parts"`
} `json:"content"`
FinishReason string `json:"finishReason,omitempty"`
} `json:"candidates"`
PromptFeedback *struct {
BlockReason string `json:"blockReason,omitempty"`
} `json:"promptFeedback,omitempty"`
}
if err := json.Unmarshal(respBytes, &apiResp); err != nil {
return nil, fmt.Errorf("gemini: parse response: %w", err)
}
// Find the first inlineData across candidates/parts (some responses lead
// with a text part before the audio part).
var b64, textHint, finishReason string
for _, c := range apiResp.Candidates {
if c.FinishReason != "" && finishReason == "" {
finishReason = c.FinishReason
}
for _, part := range c.Content.Parts {
if part.InlineData != nil && part.InlineData.Data != "" {
b64 = part.InlineData.Data
break
}
if part.Text != "" && textHint == "" {
textHint = part.Text
}
}
if b64 != "" {
break
}
}
if b64 == "" {
// Diagnose why audio is missing — give the caller something actionable.
switch {
case apiResp.PromptFeedback != nil && apiResp.PromptFeedback.BlockReason != "":
return nil, fmt.Errorf("gemini: prompt blocked (%s)", apiResp.PromptFeedback.BlockReason)
case isTransientFinishReason(finishReason):
return nil, fmt.Errorf("%w: finishReason=%s", errTransientNoAudio, finishReason)
case finishReason != "" && finishReason != "STOP":
return nil, fmt.Errorf("gemini: synthesis stopped (%s) — model returned no audio", finishReason)
case textHint != "":
snippet := textHint
if len(snippet) > 200 {
snippet = snippet[:200] + "…"
}
return nil, fmt.Errorf("gemini: model returned text instead of audio (model may not support TTS or input was misinterpreted): %s", snippet)
default:
return nil, fmt.Errorf("gemini: response missing inlineData")
}
}
pcm, err := base64.StdEncoding.DecodeString(b64)
if err != nil {
return nil, fmt.Errorf("gemini: base64 decode error: %w", err)
}
wav := Wrap(pcm)
return &audio.SynthResult{
Audio: wav,
Extension: "wav",
MimeType: "audio/wav",
}, nil
}
// isTransientFinishReason reports whether a Gemini finishReason represents a
// non-deterministic failure that's worth retrying. OTHER is the catch-all the
// preview TTS endpoint emits when it just fails to produce audio for no
// user-visible reason — single retry usually succeeds.
func isTransientFinishReason(reason string) bool {
return reason == "OTHER"
}
+367
View File
@@ -0,0 +1,367 @@
package gemini
import (
"context"
"encoding/base64"
"encoding/json"
"net/http"
"net/http/httptest"
"reflect"
"strings"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
// geminiResponseWith builds a minimal Gemini generateContent response
// wrapping the given base64-encoded data string.
func geminiResponseWith(b64data string) []byte {
resp := map[string]any{
"candidates": []map[string]any{
{
"content": map[string]any{
"parts": []map[string]any{
{
"inlineData": map[string]any{
"data": b64data,
},
},
},
},
},
},
}
b, _ := json.Marshal(resp)
return b
}
// capturedRequest records the last request received by the mock server.
type capturedRequest struct {
path string
header http.Header
body map[string]any
}
func newMockServer(t *testing.T, status int, respBody []byte) (*httptest.Server, *capturedRequest) {
t.Helper()
cap := &capturedRequest{}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
cap.path = r.URL.Path
cap.header = r.Header.Clone()
if err := json.NewDecoder(r.Body).Decode(&cap.body); err != nil {
t.Errorf("decode request body: %v", err)
}
w.WriteHeader(status)
_, _ = w.Write(respBody)
}))
t.Cleanup(srv.Close)
return srv, cap
}
// TestSynthesize_SingleVoice_RequestShape verifies URL, auth header, and JSON body
// for a single-voice synthesis call.
func TestSynthesize_SingleVoice_RequestShape(t *testing.T) {
pcm := make([]byte, 1024)
b64 := base64.StdEncoding.EncodeToString(pcm)
srv, cap := newMockServer(t, http.StatusOK, geminiResponseWith(b64))
p := NewProvider(Config{
APIKey: "test-key",
APIBase: srv.URL,
Voice: "Kore",
Model: "gemini-3.1-flash-tts-preview",
})
result, err := p.Synthesize(context.Background(), "Hello world", audio.TTSOptions{})
if err != nil {
t.Fatalf("Synthesize error: %v", err)
}
// URL path
if !strings.HasSuffix(cap.path, "/v1beta/models/gemini-3.1-flash-tts-preview:generateContent") {
t.Errorf("path = %q, want suffix .../v1beta/models/gemini-3.1-flash-tts-preview:generateContent", cap.path)
}
// Auth header
if got := cap.header.Get("x-goog-api-key"); got != "test-key" {
t.Errorf("x-goog-api-key = %q, want test-key", got)
}
// generationConfig (now also holds speechConfig — Gemini API requires this nesting)
gc, _ := cap.body["generationConfig"].(map[string]any)
mods, _ := gc["responseModalities"].([]any)
if len(mods) != 1 || mods[0] != "AUDIO" {
t.Errorf("responseModalities = %v, want [AUDIO]", mods)
}
if _, hasRoot := cap.body["speechConfig"]; hasRoot {
t.Error("speechConfig must be NESTED in generationConfig, not at body root")
}
// speechConfig — single voice (nested under generationConfig)
sc, _ := gc["speechConfig"].(map[string]any)
vc, _ := sc["voiceConfig"].(map[string]any)
pvc, _ := vc["prebuiltVoiceConfig"].(map[string]any)
if got, _ := pvc["voiceName"].(string); got != "Kore" {
t.Errorf("voiceName = %q, want Kore", got)
}
if _, hasMSV := sc["multiSpeakerVoiceConfig"]; hasMSV {
t.Error("single-voice request must NOT have multiSpeakerVoiceConfig")
}
// contents text
contents, _ := cap.body["contents"].([]any)
if len(contents) == 0 {
t.Fatal("contents empty")
}
part0 := contents[0].(map[string]any)
parts, _ := part0["parts"].([]any)
text, _ := parts[0].(map[string]any)["text"].(string)
if text != "Hello world" {
t.Errorf("text = %q, want Hello world", text)
}
// result
if result.MimeType != "audio/wav" {
t.Errorf("MimeType = %q, want audio/wav", result.MimeType)
}
if len(result.Audio) != wavHeaderSize+1024 {
t.Errorf("len(result.Audio) = %d, want %d", len(result.Audio), wavHeaderSize+1024)
}
}
// TestSynthesize_MultiSpeaker_RequestShape verifies multi-speaker body structure.
func TestSynthesize_MultiSpeaker_RequestShape(t *testing.T) {
pcm := make([]byte, 512)
b64 := base64.StdEncoding.EncodeToString(pcm)
srv, cap := newMockServer(t, http.StatusOK, geminiResponseWith(b64))
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
opts := audio.TTSOptions{
Speakers: []audio.SpeakerVoice{
{Speaker: "Joe", VoiceID: "Kore"},
{Speaker: "Jane", VoiceID: "Puck"},
},
}
if _, err := p.Synthesize(context.Background(), "Joe: Hi\nJane: Hello", opts); err != nil {
t.Fatalf("Synthesize error: %v", err)
}
gc, _ := cap.body["generationConfig"].(map[string]any)
sc, _ := gc["speechConfig"].(map[string]any)
if _, hasRoot := cap.body["speechConfig"]; hasRoot {
t.Error("speechConfig must be NESTED in generationConfig, not at body root")
}
// Must have multiSpeakerVoiceConfig, must NOT have voiceConfig at root.
msv, hasMSV := sc["multiSpeakerVoiceConfig"].(map[string]any)
if !hasMSV {
t.Fatal("speechConfig missing multiSpeakerVoiceConfig")
}
if _, hasVC := sc["voiceConfig"]; hasVC {
t.Error("multi-speaker must NOT have voiceConfig at speechConfig root")
}
svcs, _ := msv["speakerVoiceConfigs"].([]any)
if len(svcs) != 2 {
t.Fatalf("speakerVoiceConfigs len = %d, want 2", len(svcs))
}
for i, want := range []struct{ name, voice string }{{"Joe", "Kore"}, {"Jane", "Puck"}} {
svc := svcs[i].(map[string]any)
if got, _ := svc["speaker"].(string); got != want.name {
t.Errorf("[%d] speaker = %q, want %q", i, got, want.name)
}
vc := svc["voiceConfig"].(map[string]any)
pvc := vc["prebuiltVoiceConfig"].(map[string]any)
if got, _ := pvc["voiceName"].(string); got != want.voice {
t.Errorf("[%d] voiceName = %q, want %q", i, got, want.voice)
}
}
}
// TestSynthesize_DoesNotMutateCallerOpts verifies that Synthesize does not
// modify the caller's opts.Params map or opts.Speakers slice.
func TestSynthesize_DoesNotMutateCallerOpts(t *testing.T) {
pcm := make([]byte, 64)
b64 := base64.StdEncoding.EncodeToString(pcm)
srv, _ := newMockServer(t, http.StatusOK, geminiResponseWith(b64))
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
originalParams := map[string]any{"foo": "bar"}
originalSpeakers := []audio.SpeakerVoice{{Speaker: "A", VoiceID: "Kore"}}
opts := audio.TTSOptions{
Params: originalParams,
Speakers: originalSpeakers,
}
if _, err := p.Synthesize(context.Background(), "test", opts); err != nil {
t.Fatalf("Synthesize error: %v", err)
}
// Params must be untouched.
if !reflect.DeepEqual(opts.Params, originalParams) {
t.Errorf("opts.Params mutated: got %v", opts.Params)
}
// Speakers must be untouched.
if !reflect.DeepEqual(opts.Speakers, originalSpeakers) {
t.Errorf("opts.Speakers mutated: got %v", opts.Speakers)
}
}
// TestSynthesize_AuthError verifies 401 produces an auth-related error.
func TestSynthesize_AuthError(t *testing.T) {
srv, _ := newMockServer(t, http.StatusUnauthorized, []byte(`{"error":"unauthorized"}`))
p := NewProvider(Config{APIKey: "bad", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error for 401")
}
if !strings.Contains(err.Error(), "auth") {
t.Errorf("error %q should contain 'auth'", err.Error())
}
}
// TestSynthesize_RateLimitError verifies 429 produces an error.
func TestSynthesize_RateLimitError(t *testing.T) {
srv, _ := newMockServer(t, http.StatusTooManyRequests, []byte(`{"error":"rate limit"}`))
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error for 429")
}
}
// TestSynthesize_MalformedResponse_TextOnly verifies a response with a text part
// (no audio) surfaces the text snippet so the caller can debug.
func TestSynthesize_MalformedResponse_TextOnly(t *testing.T) {
body := []byte(`{"candidates":[{"content":{"parts":[{"text":"no audio here"}]}}]}`)
srv, _ := newMockServer(t, http.StatusOK, body)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error for text-only response")
}
if !strings.Contains(err.Error(), "text instead of audio") {
t.Errorf("error %q should mention 'text instead of audio'", err.Error())
}
if !strings.Contains(err.Error(), "no audio here") {
t.Errorf("error %q should include the text snippet", err.Error())
}
}
// TestSynthesize_PromptBlocked verifies a safety-blocked prompt surfaces the block reason.
func TestSynthesize_PromptBlocked(t *testing.T) {
body := []byte(`{"promptFeedback":{"blockReason":"SAFETY"}}`)
srv, _ := newMockServer(t, http.StatusOK, body)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error for blocked prompt")
}
if !strings.Contains(err.Error(), "prompt blocked") || !strings.Contains(err.Error(), "SAFETY") {
t.Errorf("error %q should mention 'prompt blocked (SAFETY)'", err.Error())
}
}
// TestSynthesize_RetriesOnFinishReasonOTHER verifies the provider retries once
// when Gemini returns finishReason=OTHER (transient flake on the preview TTS
// endpoint), and returns the audio from the second attempt.
func TestSynthesize_RetriesOnFinishReasonOTHER(t *testing.T) {
pcm := make([]byte, 64)
b64 := base64.StdEncoding.EncodeToString(pcm)
successBody := []byte(`{"candidates":[{"content":{"parts":[{"inlineData":{"data":"` + b64 + `"}}]}}]}`)
otherBody := []byte(`{"candidates":[{"finishReason":"OTHER","content":{"parts":[]}}]}`)
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
w.WriteHeader(http.StatusOK)
if calls == 1 {
_, _ = w.Write(otherBody)
} else {
_, _ = w.Write(successBody)
}
}))
t.Cleanup(srv.Close)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
r, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err != nil {
t.Fatalf("Synthesize: %v", err)
}
if calls != 2 {
t.Errorf("expected 2 calls (1 retry), got %d", calls)
}
if r.MimeType != "audio/wav" {
t.Errorf("MimeType = %q, want audio/wav", r.MimeType)
}
}
// TestSynthesize_RetriesOnceMax verifies the retry happens at most once — two
// consecutive OTHER responses still surface the error to the caller.
func TestSynthesize_RetriesOnceMax(t *testing.T) {
otherBody := []byte(`{"candidates":[{"finishReason":"OTHER","content":{"parts":[]}}]}`)
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
w.WriteHeader(http.StatusOK)
_, _ = w.Write(otherBody)
}))
t.Cleanup(srv.Close)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error after exhausting retries")
}
if calls != 2 {
t.Errorf("expected exactly 2 calls (1 initial + 1 retry), got %d", calls)
}
if !strings.Contains(err.Error(), "OTHER") {
t.Errorf("error %q should mention OTHER finishReason", err.Error())
}
}
// TestSynthesize_FinishReasonNonStop verifies a non-STOP finishReason without audio
// surfaces the reason (e.g. SAFETY, MAX_TOKENS).
func TestSynthesize_FinishReasonNonStop(t *testing.T) {
body := []byte(`{"candidates":[{"finishReason":"PROHIBITED_CONTENT","content":{"parts":[]}}]}`)
srv, _ := newMockServer(t, http.StatusOK, body)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected error for non-STOP finishReason")
}
if !strings.Contains(err.Error(), "PROHIBITED_CONTENT") {
t.Errorf("error %q should mention 'PROHIBITED_CONTENT'", err.Error())
}
}
// TestSynthesize_LeadingTextThenAudio verifies that a response which leads with
// a text part followed by an audio part still extracts the audio successfully.
func TestSynthesize_LeadingTextThenAudio(t *testing.T) {
pcm := make([]byte, 64)
b64 := base64.StdEncoding.EncodeToString(pcm)
body := []byte(`{"candidates":[{"content":{"parts":[{"text":"reading aloud"},{"inlineData":{"data":"` + b64 + `"}}]}}]}`)
srv, _ := newMockServer(t, http.StatusOK, body)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
r, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err != nil {
t.Fatalf("Synthesize: %v", err)
}
if r.MimeType != "audio/wav" {
t.Errorf("MimeType = %q, want audio/wav", r.MimeType)
}
}
// TestSynthesize_BadBase64 verifies invalid base64 produces a decode error.
func TestSynthesize_BadBase64(t *testing.T) {
body := geminiResponseWith("!!not-valid-base64!!")
srv, _ := newMockServer(t, http.StatusOK, body)
p := NewProvider(Config{APIKey: "k", APIBase: srv.URL})
_, err := p.Synthesize(context.Background(), "test", audio.TTSOptions{})
if err == nil {
t.Fatal("expected base64 decode error")
}
}
+57
View File
@@ -0,0 +1,57 @@
package gemini
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// geminiVoices is the static catalog of 30 Gemini prebuilt voices.
// Source: https://ai.google.dev/gemini-api/docs/speech-generation (April 2026)
var geminiVoices = []audio.VoiceOption{
{VoiceID: "Zephyr", Name: "Zephyr"},
{VoiceID: "Puck", Name: "Puck"},
{VoiceID: "Charon", Name: "Charon"},
{VoiceID: "Kore", Name: "Kore"},
{VoiceID: "Fenrir", Name: "Fenrir"},
{VoiceID: "Leda", Name: "Leda"},
{VoiceID: "Orus", Name: "Orus"},
{VoiceID: "Aoede", Name: "Aoede"},
{VoiceID: "Callirrhoe", Name: "Callirrhoe"},
{VoiceID: "Autonoe", Name: "Autonoe"},
{VoiceID: "Enceladus", Name: "Enceladus"},
{VoiceID: "Iapetus", Name: "Iapetus"},
{VoiceID: "Umbriel", Name: "Umbriel"},
{VoiceID: "Algieba", Name: "Algieba"},
{VoiceID: "Despina", Name: "Despina"},
{VoiceID: "Erinome", Name: "Erinome"},
{VoiceID: "Algenib", Name: "Algenib"},
{VoiceID: "Rasalgethi", Name: "Rasalgethi"},
{VoiceID: "Laomedeia", Name: "Laomedeia"},
{VoiceID: "Achernar", Name: "Achernar"},
{VoiceID: "Alnilam", Name: "Alnilam"},
{VoiceID: "Schedar", Name: "Schedar"},
{VoiceID: "Gacrux", Name: "Gacrux"},
{VoiceID: "Pulcherrima", Name: "Pulcherrima"},
{VoiceID: "Achird", Name: "Achird"},
{VoiceID: "Zubenelgenubi", Name: "Zubenelgenubi"},
{VoiceID: "Vindemiatrix", Name: "Vindemiatrix"},
{VoiceID: "Sadachbia", Name: "Sadachbia"},
{VoiceID: "Sadaltager", Name: "Sadaltager"},
{VoiceID: "Sulafat", Name: "Sulafat"},
}
// defaultVoice is the voice used when none is specified.
const defaultVoice = "Kore"
// validVoiceSet is a fast-lookup set built from geminiVoices at init time.
var validVoiceSet map[string]struct{}
func init() {
validVoiceSet = make(map[string]struct{}, len(geminiVoices))
for _, v := range geminiVoices {
validVoiceSet[v.VoiceID] = struct{}{}
}
}
// isValidVoice reports whether name is a known Gemini prebuilt voice.
func isValidVoice(name string) bool {
_, ok := validVoiceSet[name]
return ok
}
+62
View File
@@ -0,0 +1,62 @@
// Package gemini implements Google Gemini TTS via generateContent API.
// Gemini returns base64 PCM 16-bit signed little-endian 24kHz mono; this
// package wraps it in a 44-byte RIFF/WAV header so browsers can play it.
package gemini
import "encoding/binary"
// WAV header constants for 24kHz mono 16-bit PCM.
const (
sampleRate24k uint32 = 24000
channelsMono uint16 = 1
bitsPerSample uint16 = 16
audioFormatPCM uint16 = 1
fmtChunkSize uint32 = 16
wavHeaderSize = 44
)
// Wrap prepends a RIFF/WAV header to raw PCM bytes and returns playable WAV.
// PCM must be 16-bit signed little-endian at 24kHz mono.
func Wrap(pcm []byte) []byte {
dataSize := uint32(len(pcm))
byteRate := uint32(sampleRate24k) * uint32(channelsMono) * uint32(bitsPerSample) / 8
blockAlign := channelsMono * bitsPerSample / 8
out := make([]byte, wavHeaderSize+dataSize)
pos := 0
// RIFF chunk descriptor
copy(out[pos:], "RIFF")
pos += 4
binary.LittleEndian.PutUint32(out[pos:], 36+dataSize) // chunk size = 36 + data
pos += 4
copy(out[pos:], "WAVE")
pos += 4
// fmt sub-chunk
copy(out[pos:], "fmt ")
pos += 4
binary.LittleEndian.PutUint32(out[pos:], fmtChunkSize)
pos += 4
binary.LittleEndian.PutUint16(out[pos:], audioFormatPCM)
pos += 2
binary.LittleEndian.PutUint16(out[pos:], channelsMono)
pos += 2
binary.LittleEndian.PutUint32(out[pos:], sampleRate24k)
pos += 4
binary.LittleEndian.PutUint32(out[pos:], byteRate)
pos += 4
binary.LittleEndian.PutUint16(out[pos:], blockAlign)
pos += 2
binary.LittleEndian.PutUint16(out[pos:], bitsPerSample)
pos += 2
// data sub-chunk
copy(out[pos:], "data")
pos += 4
binary.LittleEndian.PutUint32(out[pos:], dataSize)
pos += 4
copy(out[pos:], pcm)
return out
}
+68
View File
@@ -0,0 +1,68 @@
package gemini
import (
"bytes"
"encoding/binary"
"testing"
)
// TestWrap_RIFFHeader_Bytes verifies every field of the 44-byte RIFF/WAV header
// produced by Wrap() for a 1024-byte PCM payload.
func TestWrap_RIFFHeader_Bytes(t *testing.T) {
pcm := make([]byte, 1024)
wav := Wrap(pcm)
if !bytes.Equal(wav[0:4], []byte("RIFF")) {
t.Errorf("bytes[0:4] = %q, want RIFF", wav[0:4])
}
if got := binary.LittleEndian.Uint32(wav[4:8]); got != 36+1024 {
t.Errorf("chunk size = %d, want %d", got, 36+1024)
}
if !bytes.Equal(wav[8:12], []byte("WAVE")) {
t.Errorf("bytes[8:12] = %q, want WAVE", wav[8:12])
}
if !bytes.Equal(wav[12:16], []byte("fmt ")) {
t.Errorf("bytes[12:16] = %q, want 'fmt '", wav[12:16])
}
if got := binary.LittleEndian.Uint32(wav[16:20]); got != 16 {
t.Errorf("fmt chunk size = %d, want 16", got)
}
if got := binary.LittleEndian.Uint16(wav[20:22]); got != 1 {
t.Errorf("audio format = %d, want 1 (PCM)", got)
}
if got := binary.LittleEndian.Uint16(wav[22:24]); got != 1 {
t.Errorf("channels = %d, want 1 (mono)", got)
}
if got := binary.LittleEndian.Uint32(wav[24:28]); got != 24000 {
t.Errorf("sample rate = %d, want 24000", got)
}
if got := binary.LittleEndian.Uint32(wav[28:32]); got != 48000 {
t.Errorf("byte rate = %d, want 48000", got)
}
if got := binary.LittleEndian.Uint16(wav[32:34]); got != 2 {
t.Errorf("block align = %d, want 2", got)
}
if got := binary.LittleEndian.Uint16(wav[34:36]); got != 16 {
t.Errorf("bits per sample = %d, want 16", got)
}
if !bytes.Equal(wav[36:40], []byte("data")) {
t.Errorf("bytes[36:40] = %q, want 'data'", wav[36:40])
}
if got := binary.LittleEndian.Uint32(wav[40:44]); got != 1024 {
t.Errorf("data chunk size = %d, want 1024", got)
}
if !bytes.Equal(wav[44:], pcm) {
t.Error("PCM payload mismatch after WAV header")
}
}
// TestWrap_EmptyPCM verifies a 44-byte header + zero data payload.
func TestWrap_EmptyPCM(t *testing.T) {
wav := Wrap([]byte{})
if len(wav) != wavHeaderSize {
t.Errorf("len(wav) = %d, want %d", len(wav), wavHeaderSize)
}
if got := binary.LittleEndian.Uint32(wav[40:44]); got != 0 {
t.Errorf("data chunk size = %d, want 0", got)
}
}
+18
View File
@@ -156,6 +156,24 @@ func (m *Manager) AutoMode() AutoMode { return m.auto }
// HasProviders reports whether any TTS provider is registered.
func (m *Manager) HasProviders() bool { return len(m.ttsProviders) > 0 }
// ListCapabilities iterates all registered TTS providers and returns their
// capability schemas. Providers implementing DescribableProvider contribute
// their full schema; others contribute a minimal stub {Provider, DisplayName}.
func (m *Manager) ListCapabilities() []ProviderCapabilities {
out := make([]ProviderCapabilities, 0, len(m.ttsProviders))
for _, p := range m.ttsProviders {
if dp, ok := p.(DescribableProvider); ok {
out = append(out, dp.Capabilities())
} else {
out = append(out, ProviderCapabilities{
Provider: p.Name(),
DisplayName: p.Name(),
})
}
}
return out
}
// ---- TTS dispatch ----
// Synthesize uses the primary provider.
+101
View File
@@ -0,0 +1,101 @@
package audio
import (
"context"
"testing"
)
// fakeDescribableProvider is a TTSProvider that also implements DescribableProvider.
type fakeDescribableProvider struct {
name string
caps ProviderCapabilities
}
func (f *fakeDescribableProvider) Name() string { return f.name }
func (f *fakeDescribableProvider) Synthesize(_ context.Context, _ string, _ TTSOptions) (*SynthResult, error) {
return nil, nil
}
func (f *fakeDescribableProvider) Capabilities() ProviderCapabilities { return f.caps }
// fakePlainProvider is a TTSProvider only (no DescribableProvider).
type fakePlainProvider struct{ name string }
func (f *fakePlainProvider) Name() string { return f.name }
func (f *fakePlainProvider) Synthesize(_ context.Context, _ string, _ TTSOptions) (*SynthResult, error) {
return nil, nil
}
// TestListCapabilities_AggregatesProviders verifies that ListCapabilities returns
// one entry per registered provider: describable ones return full schema, plain
// ones return a minimal stub.
func TestListCapabilities_AggregatesProviders(t *testing.T) {
mgr := NewManager(ManagerConfig{Primary: "describable"})
describable := &fakeDescribableProvider{
name: "describable",
caps: ProviderCapabilities{
Provider: "describable",
DisplayName: "Describable TTS",
RequiresAPIKey: true,
Models: []string{"model-a"},
},
}
plain := &fakePlainProvider{name: "plain"}
mgr.RegisterTTS(describable)
mgr.RegisterTTS(plain)
caps := mgr.ListCapabilities()
if len(caps) != 2 {
t.Fatalf("ListCapabilities: got %d entries, want 2", len(caps))
}
// Build lookup map for order-independent checks.
byName := make(map[string]ProviderCapabilities)
for _, c := range caps {
byName[c.Provider] = c
}
// Describable entry — should have full schema.
dc, ok := byName["describable"]
if !ok {
t.Fatal("missing 'describable' entry")
}
if dc.DisplayName != "Describable TTS" {
t.Errorf("DisplayName: got %q want %q", dc.DisplayName, "Describable TTS")
}
if !dc.RequiresAPIKey {
t.Error("RequiresAPIKey: expected true")
}
if len(dc.Models) != 1 || dc.Models[0] != "model-a" {
t.Errorf("Models: got %v", dc.Models)
}
// Plain entry — stub: Provider+DisplayName set, everything else zero.
pc, ok := byName["plain"]
if !ok {
t.Fatal("missing 'plain' entry")
}
if pc.Provider != "plain" {
t.Errorf("stub Provider: got %q want %q", pc.Provider, "plain")
}
if pc.DisplayName != "plain" {
t.Errorf("stub DisplayName: got %q want %q", pc.DisplayName, "plain")
}
if pc.RequiresAPIKey {
t.Error("stub RequiresAPIKey: expected false")
}
}
// TestListCapabilities_EmptyManager verifies that ListCapabilities returns an empty
// non-nil slice when no providers are registered.
func TestListCapabilities_EmptyManager(t *testing.T) {
mgr := NewManager(ManagerConfig{})
caps := mgr.ListCapabilities()
if caps == nil {
t.Error("ListCapabilities on empty manager: got nil, want empty slice")
}
if len(caps) != 0 {
t.Errorf("ListCapabilities on empty manager: got %d entries, want 0", len(caps))
}
}
+155
View File
@@ -0,0 +1,155 @@
package minimax
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// minimaxModels lists MiniMax T2A v2 model IDs.
var minimaxModels = []string{
"speech-02-hd",
"speech-02-turbo",
"speech-01-hd",
"speech-01-turbo",
}
var (
mmSpeedMin = 0.5
mmSpeedMax = 2.0
mmSpeedStep = 0.1
mmVolMin = 0.01
mmVolMax = 10.0
mmVolStep = 0.01
mmPitchMin = -12.0
mmPitchMax = 12.0
mmPitchStep = 1.0
)
// minimaxParams is the enriched param schema for MiniMax TTS.
// Defaults MUST match the hardcoded body in tts.go (characterization fixture):
// speed=1.0, vol=1.0 (omitted from original = 1.0 default), pitch=0.
// Note: "vol" key is not in the original body; adding it with default 1.0
// is safe because server default is also 1.0.
var minimaxParams = []audio.ParamSchema{
{
Key: "speed",
Type: audio.ParamTypeRange,
Label: "Speed",
Description: "Speech speed multiplier (0.5 = slowest, 2.0 = fastest).",
Default: 1.0,
Min: &mmSpeedMin,
Max: &mmSpeedMax,
Step: &mmSpeedStep,
},
{
Key: "vol",
Type: audio.ParamTypeRange,
Label: "Volume",
Description: "Volume multiplier (0.01 = quietest, 10.0 = loudest).",
Default: 1.0,
Min: &mmVolMin,
Max: &mmVolMax,
Step: &mmVolStep,
},
{
Key: "pitch",
Type: audio.ParamTypeInteger,
Label: "Pitch",
Description: "Pitch shift in semitones (-12 to +12).",
Default: 0,
Min: &mmPitchMin,
Max: &mmPitchMax,
Step: &mmPitchStep,
},
{
Key: "emotion",
Type: audio.ParamTypeEnum,
Label: "Emotion",
Default: "",
Enum: []audio.EnumOption{
{Value: "", Label: "None (default)"},
{Value: "happy", Label: "Happy"},
{Value: "sad", Label: "Sad"},
{Value: "angry", Label: "Angry"},
{Value: "fearful", Label: "Fearful"},
{Value: "disgusted", Label: "Disgusted"},
{Value: "surprised", Label: "Surprised"},
{Value: "neutral", Label: "Neutral"},
{Value: "excited", Label: "Excited"},
{Value: "anxious", Label: "Anxious"},
},
},
{
Key: "text_normalization",
Type: audio.ParamTypeBoolean,
Label: "Text Normalization",
Default: nil, // omit from body when not explicitly set
},
{
Key: "audio.format",
Type: audio.ParamTypeEnum,
Label: "Audio Format",
Default: "mp3",
Enum: []audio.EnumOption{
{Value: "mp3", Label: "MP3"},
{Value: "pcm", Label: "PCM"},
{Value: "flac", Label: "FLAC"},
{Value: "wav", Label: "WAV"},
},
},
{
Key: "audio.sample_rate",
Type: audio.ParamTypeEnum,
Label: "Sample Rate",
Default: "",
Enum: []audio.EnumOption{
{Value: "", Label: "Default"},
{Value: "8000", Label: "8 kHz"},
{Value: "16000", Label: "16 kHz"},
{Value: "22050", Label: "22 kHz"},
{Value: "24000", Label: "24 kHz"},
{Value: "32000", Label: "32 kHz"},
{Value: "44100", Label: "44.1 kHz"},
},
},
{
Key: "audio.bitrate",
Type: audio.ParamTypeEnum,
Label: "Bitrate (MP3 only)",
Default: "",
Enum: []audio.EnumOption{
{Value: "", Label: "Default"},
{Value: "32000", Label: "32 kbps"},
{Value: "64000", Label: "64 kbps"},
{Value: "128000", Label: "128 kbps"},
{Value: "256000", Label: "256 kbps"},
},
DependsOn: []audio.Dependency{
{Field: "audio.format", Op: "eq", Value: "mp3"},
},
},
{
Key: "audio.channel",
Type: audio.ParamTypeEnum,
Label: "Channels",
Default: "",
Enum: []audio.EnumOption{
{Value: "", Label: "Default"},
{Value: "1", Label: "Mono"},
{Value: "2", Label: "Stereo"},
},
},
}
// Capabilities returns the full capability schema for the MiniMax TTS provider.
// VoicesDynamic=true signals the frontend to fetch voices from /v1/voices?provider=minimax.
func (p *Provider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: "minimax",
DisplayName: "MiniMax TTS",
RequiresAPIKey: true,
Models: minimaxModels,
// Voices is nil — fetched dynamically via VoiceListProvider
Params: minimaxParams,
CustomFeatures: map[string]any{
"voices_dynamic": true,
},
}
}
@@ -0,0 +1,135 @@
package minimax_test
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
)
// captureMiniMaxBody sends a Synthesize request to a mock server and
// returns the decoded JSON body map + request URL.
func captureMiniMaxBody(t *testing.T, cfg minimax.Config, opts audio.TTSOptions) (map[string]any, string) {
t.Helper()
var captured []byte
var capturedURL string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
capturedURL = r.URL.String()
b, err := io.ReadAll(r.Body)
if err != nil {
t.Errorf("read body: %v", err)
}
captured = b
// MiniMax returns hex-encoded audio in a JSON envelope.
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"base_resp":{"status_code":0,"status_msg":"ok"},"data":{"audio":"deadbeef"}}`))
}))
t.Cleanup(srv.Close)
cfg.APIBase = srv.URL
p := minimax.NewProvider(cfg)
_, err := p.Synthesize(t.Context(), "hello", opts)
if err != nil {
t.Fatalf("Synthesize failed: %v", err)
}
var m map[string]any
if err := json.Unmarshal(captured, &m); err != nil {
t.Fatalf("body not valid JSON: %v — body: %s", err, captured)
}
return m, capturedURL
}
// TestCharacterization_MiniMax_DefaultOpts is the golden-fixture test.
// It captures the exact request body that the current hardcoded tts.go emits
// for default opts. These values MUST remain identical after the Synthesize refactor.
func TestCharacterization_MiniMax_DefaultOpts(t *testing.T) {
cfg := minimax.Config{APIKey: "test-key", GroupID: "grp1"}
body, reqURL := captureMiniMaxBody(t, cfg, audio.TTSOptions{})
// Golden: stream must be false.
if v, _ := body["stream"].(bool); v != false {
t.Errorf("stream: got %v, want false", body["stream"])
}
// Golden: model defaults.
if v, _ := body["model"].(string); v != "speech-02-hd" {
t.Errorf("model: got %q, want speech-02-hd", v)
}
// Golden: voice_setting shape.
vs, ok := body["voice_setting"].(map[string]any)
if !ok {
t.Fatalf("voice_setting missing or not an object: %#v", body["voice_setting"])
}
if vid, _ := vs["voice_id"].(string); vid != "Wise_Woman" {
t.Errorf("voice_setting.voice_id: got %q, want Wise_Woman", vid)
}
assertMiniMaxFloat(t, vs, "speed", 1.0)
if pitch, ok := vs["pitch"]; ok {
// pitch is int 0 — JSON unmarshal gives float64
if pf, _ := pitch.(float64); pf != 0 {
t.Errorf("voice_setting.pitch: got %v, want 0", pitch)
}
}
// Golden: audio_setting shape.
as, ok := body["audio_setting"].(map[string]any)
if !ok {
t.Fatalf("audio_setting missing or not an object: %#v", body["audio_setting"])
}
if fmt, _ := as["format"].(string); fmt != "mp3" {
t.Errorf("audio_setting.format: got %q, want mp3", fmt)
}
// Golden: GroupId in URL query.
if !containsMiniMaxStr(reqURL, "GroupId=grp1") {
t.Errorf("expected GroupId=grp1 in URL, got: %s", reqURL)
}
}
// TestCharacterization_MiniMax_OpusFormat verifies the format override flows
// through to audio_setting.format.
func TestCharacterization_MiniMax_OpusFormat(t *testing.T) {
cfg := minimax.Config{APIKey: "key", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{Format: "flac"})
as, ok := body["audio_setting"].(map[string]any)
if !ok {
t.Fatalf("audio_setting missing")
}
if fmt, _ := as["format"].(string); fmt != "flac" {
t.Errorf("audio_setting.format: got %q, want flac", fmt)
}
}
func assertMiniMaxFloat(t *testing.T, m map[string]any, key string, want float64) {
t.Helper()
v, ok := m[key]
if !ok {
t.Errorf("missing field %q", key)
return
}
got, ok := v.(float64)
if !ok {
t.Errorf("field %q: expected float64, got %T (%v)", key, v, v)
return
}
if got != want {
t.Errorf("field %q: got %v, want %v", key, got, want)
}
}
func containsMiniMaxStr(s, sub string) bool {
for i := 0; i <= len(s)-len(sub); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
@@ -0,0 +1,53 @@
package minimax_test
import (
"encoding/json"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
)
// TestDefaults_PreserveLegacyBody verifies that populating opts.Params with all
// Capabilities defaults produces a body byte-equivalent to the nil-Params
// characterization fixture. Guards against silent default flip on save.
func TestDefaults_PreserveLegacyBody(t *testing.T) {
cfg := minimax.Config{APIKey: "test-key", GroupID: "grp1"}
p := minimax.NewProvider(cfg)
caps := p.Capabilities()
if len(caps.Params) == 0 {
t.Skip("Capabilities.Params not yet populated (Phase C enrichment pending)")
}
params := make(map[string]any)
for _, s := range caps.Params {
if s.Default != nil {
audio.SetNested(params, s.Key, s.Default)
}
}
bodyWithDefaults, urlWithDefaults := captureMiniMaxBody(t, cfg, audio.TTSOptions{Params: params})
bodyNilParams, urlNilParams := captureMiniMaxBody(t, cfg, audio.TTSOptions{})
wantJSON := canonicalMiniMaxJSON(t, bodyNilParams)
gotJSON := canonicalMiniMaxJSON(t, bodyWithDefaults)
if gotJSON != wantJSON {
t.Errorf("defaults-invariant body FAILED:\n with-defaults: %s\n nil-params: %s",
gotJSON, wantJSON)
}
if urlWithDefaults != urlNilParams {
t.Errorf("defaults-invariant URL FAILED:\n with-defaults: %s\n nil-params: %s",
urlWithDefaults, urlNilParams)
}
}
func canonicalMiniMaxJSON(t *testing.T, m map[string]any) string {
t.Helper()
b, err := json.Marshal(m)
if err != nil {
t.Fatalf("canonicalMiniMaxJSON marshal: %v", err)
}
return string(b)
}
+146
View File
@@ -0,0 +1,146 @@
package minimax_test
import (
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
)
func TestSynthesize_AppliesParams_Speed(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"speed": 1.5},
})
vs, _ := body["voice_setting"].(map[string]any)
assertMiniMaxFloat(t, vs, "speed", 1.5)
}
func TestSynthesize_AppliesParams_Pitch(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"pitch": 3},
})
vs, _ := body["voice_setting"].(map[string]any)
if p, _ := vs["pitch"].(float64); int(p) != 3 {
t.Errorf("pitch: got %v, want 3", vs["pitch"])
}
}
func TestSynthesize_AppliesParams_AudioGroup_Format(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{
Params: map[string]any{
"audio": map[string]any{"format": "flac"},
},
})
as, _ := body["audio_setting"].(map[string]any)
if f, _ := as["format"].(string); f != "flac" {
t.Errorf("audio_setting.format: got %q, want flac", f)
}
}
func TestSynthesize_AppliesParams_AudioGroup_Bitrate(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{
Params: map[string]any{
"audio": map[string]any{"format": "mp3", "bitrate": "128000"},
},
})
as, _ := body["audio_setting"].(map[string]any)
if br, _ := as["bitrate"].(string); br != "128000" {
t.Errorf("audio_setting.bitrate: got %q, want 128000", br)
}
}
func TestSynthesize_AppliesParams_Emotion(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"emotion": "happy"},
})
vs, _ := body["voice_setting"].(map[string]any)
if em, _ := vs["emotion"].(string); em != "happy" {
t.Errorf("voice_setting.emotion: got %q, want happy", em)
}
}
func TestSynthesize_AppliesParams_OmitsEmpty_NilParams(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
body, _ := captureMiniMaxBody(t, cfg, audio.TTSOptions{})
// text_normalization must be absent for nil params.
if _, ok := body["text_normalization"]; ok {
t.Error("text_normalization must not appear for nil params")
}
// emotion must be absent.
vs, _ := body["voice_setting"].(map[string]any)
if _, ok := vs["emotion"]; ok {
t.Error("emotion must not appear for nil params")
}
}
func TestSynthesize_DoesNotMutateCallerParams(t *testing.T) {
cfg := minimax.Config{APIKey: "k", GroupID: "g"}
original := map[string]any{
"speed": 1.0,
"sentinel": "untouched",
}
snapshot := map[string]any{
"speed": original["speed"],
"sentinel": original["sentinel"],
}
captureMiniMaxBody(t, cfg, audio.TTSOptions{Params: original})
for k, want := range snapshot {
if got := original[k]; got != want {
t.Errorf("caller Params mutated: key %q was %v, now %v", k, want, got)
}
}
}
func TestCapabilities_HasParam_Speed(t *testing.T) {
p := minimax.NewProvider(minimax.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "speed" {
if param.Default != 1.0 {
t.Errorf("speed default: got %v, want 1.0", param.Default)
}
return
}
}
t.Error("speed param not found")
}
func TestCapabilities_DependsOn_MiniMaxBitrate(t *testing.T) {
p := minimax.NewProvider(minimax.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "audio.bitrate" {
if len(param.DependsOn) == 0 {
t.Fatal("audio.bitrate must have DependsOn")
}
dep := param.DependsOn[0]
if dep.Field != "audio.format" {
t.Errorf("DependsOn.Field: got %q, want audio.format", dep.Field)
}
if dep.Value != "mp3" {
t.Errorf("DependsOn.Value: got %v, want mp3", dep.Value)
}
return
}
}
t.Error("audio.bitrate param not found")
}
func TestCapabilities_VoicesDynamic(t *testing.T) {
p := minimax.NewProvider(minimax.Config{APIKey: "k"})
caps := p.Capabilities()
if caps.CustomFeatures == nil {
t.Fatal("CustomFeatures must not be nil")
}
if dyn, _ := caps.CustomFeatures["voices_dynamic"].(bool); !dyn {
t.Error("CustomFeatures.voices_dynamic must be true")
}
if len(caps.Voices) != 0 {
t.Error("Voices must be nil/empty for dynamic provider")
}
}
+140 -21
View File
@@ -65,6 +65,19 @@ func (p *Provider) Name() string { return "minimax" }
// Synthesize calls MiniMax t2a_v2 (non-streaming). MiniMax returns hex-encoded
// audio in response.data.audio — we decode before returning.
//
// opts.Params keys (nested dot-path):
// - "speed" float64 (0.5–2.0, default 1.0)
// - "vol" float64 (0.01–10.0, default 1.0; omitted from body at default)
// - "pitch" int (-12..12, default 0)
// - "emotion" string (optional)
// - "text_normalization" bool (optional)
// - "audio.format" string (mp3/pcm/flac/wav, default "mp3")
// - "audio.sample_rate" string (optional)
// - "audio.bitrate" string (optional, mp3 only)
// - "audio.channel" string (optional)
//
// MUST NOT mutate opts.Params — reads only.
func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error) {
voiceID := opts.Voice
if voiceID == "" {
@@ -75,32 +88,72 @@ func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOp
model = p.model
}
audioFormat, ext, mime := "mp3", "mp3", "audio/mpeg"
if opts.Format == "opus" || opts.Format == "pcm" || opts.Format == "flac" || opts.Format == "wav" {
// Resolve audio format — affects both MIME type and audio_setting.format.
audioFormat := resolveMiniMaxString(opts.Params, "audio.format", "")
if audioFormat == "" {
// Fall back to opts.Format (legacy path) then default mp3.
audioFormat = opts.Format
switch opts.Format {
case "pcm":
ext, mime = "pcm", "audio/pcm"
case "flac":
ext, mime = "flac", "audio/flac"
case "wav":
ext, mime = "wav", "audio/wav"
}
}
if audioFormat == "" || (audioFormat != "pcm" && audioFormat != "flac" && audioFormat != "wav") {
audioFormat = "mp3"
}
ext, mime := audioFormat, "audio/mpeg"
switch audioFormat {
case "pcm":
ext, mime = "pcm", "audio/pcm"
case "flac":
ext, mime = "flac", "audio/flac"
case "wav":
ext, mime = "wav", "audio/wav"
}
// Build voice_setting from params with defaults matching characterization.
speed := resolveMiniMaxFloat(opts.Params, "speed", 1.0)
pitch := resolveMiniMaxInt(opts.Params, "pitch", 0)
voiceSetting := map[string]any{
"voice_id": voiceID,
"speed": speed,
"pitch": pitch,
}
// vol only added when explicitly set and not the default (preserves nil-params body shape).
if vol := resolveMiniMaxFloat(opts.Params, "vol", -1); vol > 0 && vol != 1.0 {
voiceSetting["vol"] = vol
}
// emotion only added when set.
if emotion := resolveMiniMaxString(opts.Params, "emotion", ""); emotion != "" {
voiceSetting["emotion"] = emotion
}
// Build audio_setting.
audioSetting := map[string]any{
"format": audioFormat,
}
if sr := resolveMiniMaxString(opts.Params, "audio.sample_rate", ""); sr != "" {
audioSetting["sample_rate"] = sr
}
if br := resolveMiniMaxString(opts.Params, "audio.bitrate", ""); br != "" {
audioSetting["bitrate"] = br
}
if ch := resolveMiniMaxString(opts.Params, "audio.channel", ""); ch != "" {
audioSetting["channel"] = ch
}
body := map[string]any{
"text": text,
"model": model,
"stream": false,
"voice_setting": map[string]any{
"voice_id": voiceID,
"speed": 1.0,
"pitch": 0,
},
"audio_setting": map[string]any{
"format": audioFormat,
},
"text": text,
"model": model,
"stream": false,
"voice_setting": voiceSetting,
"audio_setting": audioSetting,
}
// text_normalization only when explicitly set.
if tn, ok := resolveMiniMaxBoolExplicit(opts.Params, "text_normalization"); ok {
body["text_normalization"] = tn
}
bodyJSON, err := json.Marshal(body)
if err != nil {
return nil, fmt.Errorf("marshal minimax tts request: %w", err)
@@ -161,3 +214,69 @@ type miniMaxResponse struct {
Audio string `json:"audio"` // hex-encoded audio bytes
} `json:"data"`
}
func resolveMiniMaxFloat(params map[string]any, key string, def float64) float64 {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
switch n := v.(type) {
case float64:
return n
case float32:
return float64(n)
case int:
return float64(n)
}
return def
}
func resolveMiniMaxInt(params map[string]any, key string, def int) int {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
switch n := v.(type) {
case int:
return n
case int64:
return int(n)
case float64:
return int(n)
}
return def
}
func resolveMiniMaxString(params map[string]any, key, def string) string {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
if s, ok := v.(string); ok {
return s
}
return def
}
// resolveMiniMaxBoolExplicit returns (value, true) only when the key is explicitly
// present in params, so callers can omit the field entirely when not set.
func resolveMiniMaxBoolExplicit(params map[string]any, key string) (bool, bool) {
if params == nil {
return false, false
}
v, ok := audio.GetNested(params, key)
if !ok {
return false, false
}
b, ok := v.(bool)
return b, ok
}
+156
View File
@@ -0,0 +1,156 @@
package minimax
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"sync"
"time"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/audio"
)
const voicesCacheTTL = 5 * time.Minute
// VoiceLister fetches MiniMax voices via POST /v1/get_voice.
// It holds a per-tenant in-process cache with 5-min TTL and
// stale-while-refetch semantics on upstream 5xx errors.
type VoiceLister struct {
apiKey string
apiBase string
timeoutMs int
tenantID uuid.UUID
mu sync.Mutex
cached []audio.Voice
expiresAt time.Time
}
// NewVoiceLister creates a VoiceLister for the given tenant credentials.
func NewVoiceLister(apiKey, apiBase string, timeoutMs int, tenantID uuid.UUID) *VoiceLister {
if apiBase == "" {
apiBase = "https://api.minimax.io/v1"
}
if timeoutMs <= 0 {
timeoutMs = 15000
}
return &VoiceLister{
apiKey: apiKey,
apiBase: apiBase,
timeoutMs: timeoutMs,
tenantID: tenantID,
}
}
// ListVoices implements audio.VoiceListProvider.
// Returns cached voices when fresh. On upstream 5xx, returns stale cache
// (if available) or empty slice + structured error.
func (vl *VoiceLister) ListVoices(ctx context.Context) ([]audio.Voice, error) {
vl.mu.Lock()
defer vl.mu.Unlock()
// Cache hit.
if len(vl.cached) > 0 && time.Now().Before(vl.expiresAt) {
return vl.cached, nil
}
voices, err := vl.fetchVoices(ctx)
if err != nil {
// Stale cache fallback on upstream errors.
if len(vl.cached) > 0 {
return vl.cached, nil
}
return []audio.Voice{}, err
}
vl.cached = voices
vl.expiresAt = time.Now().Add(voicesCacheTTL)
return voices, nil
}
// voiceGetRequest is the POST body for MiniMax /v1/get_voice.
type voiceGetRequest struct {
VoiceType string `json:"voice_type"`
}
// voiceGetResponse mirrors the MiniMax /v1/get_voice response envelope.
type voiceGetResponse struct {
SystemVoice []voiceEntry `json:"system_voice"`
VoiceCloning []voiceEntry `json:"voice_cloning"`
VoiceGeneration []voiceEntry `json:"voice_generation"`
}
type voiceEntry struct {
VoiceID string `json:"voice_id"`
VoiceName string `json:"voice_name"`
}
func (vl *VoiceLister) fetchVoices(ctx context.Context) ([]audio.Voice, error) {
reqBody, err := json.Marshal(voiceGetRequest{VoiceType: "all"})
if err != nil {
return nil, fmt.Errorf("minimax voices: marshal request: %w", err)
}
url := vl.apiBase + "/get_voice"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(reqBody))
if err != nil {
return nil, fmt.Errorf("minimax voices: create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+vl.apiKey)
hc := &http.Client{Timeout: time.Duration(vl.timeoutMs) * time.Millisecond}
resp, err := hc.Do(req)
if err != nil {
return nil, fmt.Errorf("minimax voices: request failed: %w", err)
}
defer resp.Body.Close()
respBytes, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("minimax voices: read response: %w", err)
}
if resp.StatusCode == http.StatusUnauthorized {
return nil, fmt.Errorf("minimax voices: unauthorized (401)")
}
if resp.StatusCode >= 500 {
return nil, fmt.Errorf("minimax voices: upstream error %d: %s", resp.StatusCode, string(respBytes))
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("minimax voices: unexpected status %d: %s", resp.StatusCode, string(respBytes))
}
var result voiceGetResponse
if err := json.Unmarshal(respBytes, &result); err != nil {
return nil, fmt.Errorf("minimax voices: parse response: %w", err)
}
voices := make([]audio.Voice, 0, len(result.SystemVoice)+len(result.VoiceCloning)+len(result.VoiceGeneration))
for _, v := range result.SystemVoice {
voices = append(voices, audio.Voice{
ID: v.VoiceID,
Name: v.VoiceName,
Category: "system",
})
}
for _, v := range result.VoiceCloning {
voices = append(voices, audio.Voice{
ID: v.VoiceID,
Name: v.VoiceName,
Category: "cloning",
})
}
for _, v := range result.VoiceGeneration {
voices = append(voices, audio.Voice{
ID: v.VoiceID,
Name: v.VoiceName,
Category: "generation",
})
}
return voices, nil
}
+180
View File
@@ -0,0 +1,180 @@
package minimax_test
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
)
func TestListVoices_RequestShape(t *testing.T) {
var gotMethod, gotPath, gotAuth string
var gotBody map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotMethod = r.Method
gotPath = r.URL.Path
gotAuth = r.Header.Get("Authorization")
b, _ := io.ReadAll(r.Body)
json.Unmarshal(b, &gotBody)
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"system_voice":[],"voice_cloning":[],"voice_generation":[]}`))
}))
defer srv.Close()
vl := minimax.NewVoiceLister("test-key", srv.URL, 5000, uuid.New())
vl.ListVoices(t.Context())
if gotMethod != http.MethodPost {
t.Errorf("method: got %q, want POST", gotMethod)
}
if gotPath != "/get_voice" {
t.Errorf("path: got %q, want /get_voice", gotPath)
}
if gotAuth != "Bearer test-key" {
t.Errorf("auth: got %q, want Bearer test-key", gotAuth)
}
if gotBody["voice_type"] != "all" {
t.Errorf("voice_type: got %v, want all", gotBody["voice_type"])
}
// NO GroupId in URL or body.
if _, ok := gotBody["group_id"]; ok {
t.Error("group_id must NOT be in request body")
}
}
func TestListVoices_ParsesAllThreeArrays(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{
"system_voice":[{"voice_id":"S","voice_name":"Sys"}],
"voice_cloning":[{"voice_id":"C","voice_name":"Clone"}],
"voice_generation":[{"voice_id":"G","voice_name":"Gen"}]
}`))
}))
defer srv.Close()
vl := minimax.NewVoiceLister("k", srv.URL, 5000, uuid.New())
voices, err := vl.ListVoices(t.Context())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(voices) != 3 {
t.Fatalf("expected 3 voices, got %d", len(voices))
}
cats := map[string]string{}
for _, v := range voices {
cats[v.ID] = v.Category
}
if cats["S"] != "system" {
t.Errorf("S category: got %q, want system", cats["S"])
}
if cats["C"] != "cloning" {
t.Errorf("C category: got %q, want cloning", cats["C"])
}
if cats["G"] != "generation" {
t.Errorf("G category: got %q, want generation", cats["G"])
}
}
func TestListVoices_EmptyArrays(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
// all null arrays — must not panic
w.Write([]byte(`{}`))
}))
defer srv.Close()
vl := minimax.NewVoiceLister("k", srv.URL, 5000, uuid.New())
voices, err := vl.ListVoices(t.Context())
if err != nil {
t.Fatalf("unexpected error for empty response: %v", err)
}
if voices == nil {
t.Error("expected non-nil empty slice, got nil")
}
if len(voices) != 0 {
t.Errorf("expected 0 voices, got %d", len(voices))
}
}
func TestListVoices_AuthError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusUnauthorized)
w.Write([]byte(`{"message":"invalid_api_key"}`))
}))
defer srv.Close()
vl := minimax.NewVoiceLister("bad-key", srv.URL, 5000, uuid.New())
_, err := vl.ListVoices(t.Context())
if err == nil {
t.Fatal("expected error for 401, got nil")
}
if !containsMiniMaxStr(err.Error(), "401") && !containsMiniMaxStr(err.Error(), "unauthorized") {
t.Errorf("error should mention 401/unauthorized, got: %v", err)
}
}
func TestListVoices_5xxFallback_ReturnsStaleCacheOrEmpty(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
}))
defer srv.Close()
vl := minimax.NewVoiceLister("k", srv.URL, 5000, uuid.New())
voices, err := vl.ListVoices(t.Context())
// No prior cache → returns empty slice + error.
if err == nil {
t.Error("expected error when upstream 500 and no cache")
}
if voices == nil {
t.Error("voices must be non-nil even on error (empty slice)")
}
}
func TestListVoices_5xxFallback_ReturnsStaleCacheWhenPresent(t *testing.T) {
callCount := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
callCount++
if callCount == 1 {
// First call: success, populates cache.
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{
"system_voice":[{"voice_id":"V1","voice_name":"Voice1"},{"voice_id":"V2","voice_name":"Voice2"}],
"voice_cloning":[],
"voice_generation":[]
}`))
} else {
// Subsequent calls: 500.
w.WriteHeader(http.StatusInternalServerError)
}
}))
defer srv.Close()
vl := minimax.NewVoiceLister("k", srv.URL, 5000, uuid.New())
// First call populates cache.
voices1, err := vl.ListVoices(t.Context())
if err != nil {
t.Fatalf("first call failed: %v", err)
}
if len(voices1) != 2 {
t.Fatalf("first call: expected 2 voices, got %d", len(voices1))
}
// Expire the cache by re-fetching after forcing expiry.
// We simulate by calling ListVoices directly; the cache is still fresh
// so it returns the cached result without calling upstream again.
voices2, err := vl.ListVoices(t.Context())
if err != nil {
t.Fatalf("second call (cached) failed: %v", err)
}
if len(voices2) != 2 {
t.Errorf("second call (cached): expected 2 voices, got %d", len(voices2))
}
}
+45
View File
@@ -0,0 +1,45 @@
package audio
// SetNested sets a value in a map[string]any using a dot-separated key path.
// Intermediate maps are created on demand. If an intermediate path segment
// already holds a non-map value it is replaced with a new map.
//
// Example:
//
// SetNested(m, "voice_settings.stability", 0.5)
// // m = {"voice_settings": {"stability": 0.5}}
func SetNested(m map[string]any, key string, value any) {
parts, err := parseKeyPath(key)
if err != nil {
return // malformed key — silently skip
}
cur := m
for _, part := range parts[:len(parts)-1] {
next, ok := cur[part].(map[string]any)
if !ok {
next = make(map[string]any)
cur[part] = next
}
cur = next
}
cur[parts[len(parts)-1]] = value
}
// GetNested retrieves a value from a map[string]any using a dot-separated key
// path. Returns (nil, false) if any segment is missing or non-traversable.
func GetNested(m map[string]any, key string) (any, bool) {
parts, err := parseKeyPath(key)
if err != nil {
return nil, false
}
cur := m
for _, part := range parts[:len(parts)-1] {
next, ok := cur[part].(map[string]any)
if !ok {
return nil, false
}
cur = next
}
v, ok := cur[parts[len(parts)-1]]
return v, ok
}
+126
View File
@@ -0,0 +1,126 @@
package audio
import (
"testing"
)
func TestSetNested_FlatKey(t *testing.T) {
m := map[string]any{}
SetNested(m, "speed", 1.2)
v, ok := m["speed"]
if !ok {
t.Fatal("expected key 'speed' to be set")
}
if v != 1.2 {
t.Errorf("got %v, want 1.2", v)
}
}
func TestSetNested_TwoLevel(t *testing.T) {
m := map[string]any{}
SetNested(m, "voice_settings.stability", 0.5)
vs, ok := m["voice_settings"].(map[string]any)
if !ok {
t.Fatalf("voice_settings not a map: %T", m["voice_settings"])
}
if vs["stability"] != 0.5 {
t.Errorf("stability: got %v, want 0.5", vs["stability"])
}
}
func TestSetNested_MergesExisting(t *testing.T) {
m := map[string]any{
"voice_settings": map[string]any{"speed": 1.0},
}
SetNested(m, "voice_settings.stability", 0.5)
vs := m["voice_settings"].(map[string]any)
if vs["speed"] != 1.0 {
t.Errorf("speed: got %v, want 1.0 (must not be overwritten)", vs["speed"])
}
if vs["stability"] != 0.5 {
t.Errorf("stability: got %v, want 0.5", vs["stability"])
}
}
func TestSetNested_OverridesScalar(t *testing.T) {
m := map[string]any{}
SetNested(m, "voice_settings.stability", 0.5)
SetNested(m, "voice_settings.stability", 0.7)
vs := m["voice_settings"].(map[string]any)
if vs["stability"] != 0.7 {
t.Errorf("stability: got %v, want 0.7 (override)", vs["stability"])
}
}
func TestSetNested_ReplacesNonMapIntermediate(t *testing.T) {
// If intermediate segment holds a scalar, it must be replaced with a map.
m := map[string]any{"voice_settings": "not-a-map"}
SetNested(m, "voice_settings.stability", 0.5)
vs, ok := m["voice_settings"].(map[string]any)
if !ok {
t.Fatalf("voice_settings should have been replaced with a map, got %T", m["voice_settings"])
}
if vs["stability"] != 0.5 {
t.Errorf("stability: got %v, want 0.5", vs["stability"])
}
}
func TestSetNested_ThreeLevel(t *testing.T) {
m := map[string]any{}
SetNested(m, "a.b.c", 42)
ab := m["a"].(map[string]any)
abc := ab["b"].(map[string]any)
if abc["c"] != 42 {
t.Errorf("a.b.c: got %v, want 42", abc["c"])
}
}
func TestGetNested_FlatKey(t *testing.T) {
m := map[string]any{"speed": 1.5}
v, ok := GetNested(m, "speed")
if !ok || v != 1.5 {
t.Errorf("GetNested flat: got (%v,%v), want (1.5,true)", v, ok)
}
}
func TestGetNested_TwoLevel(t *testing.T) {
m := map[string]any{
"voice_settings": map[string]any{"stability": 0.5},
}
v, ok := GetNested(m, "voice_settings.stability")
if !ok || v != 0.5 {
t.Errorf("GetNested two-level: got (%v,%v), want (0.5,true)", v, ok)
}
}
func TestGetNested_MissingKey(t *testing.T) {
m := map[string]any{}
_, ok := GetNested(m, "missing")
if ok {
t.Error("expected false for missing key")
}
}
func TestGetNested_MissingIntermediate(t *testing.T) {
m := map[string]any{}
_, ok := GetNested(m, "a.b.c")
if ok {
t.Error("expected false when intermediate key absent")
}
}
func TestGetNested_NonMapIntermediate(t *testing.T) {
m := map[string]any{"a": "scalar"}
_, ok := GetNested(m, "a.b")
if ok {
t.Error("expected false when intermediate is scalar")
}
}
func TestSetNested_EmptyKeyIgnored(t *testing.T) {
m := map[string]any{}
SetNested(m, "", "value") // must not panic
if len(m) != 0 {
t.Error("empty key should leave map unchanged")
}
}
+86
View File
@@ -0,0 +1,86 @@
package openai
import "github.com/nextlevelbuilder/goclaw/internal/audio"
// openAIModels is the static allowlist of supported OpenAI TTS models.
var openAIModels = []string{
"tts-1",
"tts-1-hd",
"gpt-4o-mini-tts",
}
// openAIVoices is the full 13-voice catalog. tts-1/tts-1-hd support only 9
// (ballad/verse/marin/cedar are gpt-4o-mini-tts only), but backend accepts any
// voice ID — model-compatibility is surfaced as UI help text only.
var openAIVoices = []audio.VoiceOption{
{VoiceID: "alloy", Name: "Alloy"},
{VoiceID: "ash", Name: "Ash"},
{VoiceID: "ballad", Name: "Ballad"},
{VoiceID: "coral", Name: "Coral"},
{VoiceID: "echo", Name: "Echo"},
{VoiceID: "fable", Name: "Fable"},
{VoiceID: "onyx", Name: "Onyx"},
{VoiceID: "nova", Name: "Nova"},
{VoiceID: "sage", Name: "Sage"},
{VoiceID: "shimmer", Name: "Shimmer"},
{VoiceID: "verse", Name: "Verse"},
{VoiceID: "marin", Name: "Marin"},
{VoiceID: "cedar", Name: "Cedar"},
}
var (
speedMin = 0.25
speedMax = 4.0
speedStep = 0.05
)
// openAIParams is the enriched param schema for OpenAI TTS.
// Defaults MUST match the hardcoded values in tts.go (characterization fixture).
var openAIParams = []audio.ParamSchema{
{
Key: "speed",
Type: audio.ParamTypeRange,
Label: "Speed",
Description: "Speech speed multiplier (0.25 = slowest, 4.0 = fastest).",
Default: 1.0,
Min: &speedMin,
Max: &speedMax,
Step: &speedStep,
},
{
Key: "response_format",
Type: audio.ParamTypeEnum,
Label: "Output Format",
Default: "mp3",
Enum: []audio.EnumOption{
{Value: "mp3", Label: "MP3"},
{Value: "opus", Label: "Opus"},
{Value: "aac", Label: "AAC"},
{Value: "flac", Label: "FLAC"},
{Value: "wav", Label: "WAV"},
{Value: "pcm", Label: "PCM"},
},
},
{
Key: "instructions",
Type: audio.ParamTypeText,
Label: "Instructions",
Description: "System-level style prompt (gpt-4o-mini-tts only).",
Default: "",
DependsOn: []audio.Dependency{
{Field: "model", Op: "eq", Value: "gpt-4o-mini-tts"},
},
},
}
// Capabilities returns the full capability schema for the OpenAI TTS provider.
func (p *Provider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: "openai",
DisplayName: "OpenAI TTS",
RequiresAPIKey: true,
Models: openAIModels,
Voices: openAIVoices,
Params: openAIParams,
}
}
@@ -0,0 +1,86 @@
package openai_test
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/openai"
)
// captureOpenAIBody sends a Synthesize request to a mock server and
// returns the raw JSON body bytes that the provider sent.
func captureOpenAIBody(t *testing.T, cfg openai.Config, opts audio.TTSOptions) []byte {
t.Helper()
var captured []byte
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
t.Errorf("expected POST, got %s", r.Method)
}
b, err := io.ReadAll(r.Body)
if err != nil {
t.Errorf("read body: %v", err)
}
captured = b
w.Header().Set("Content-Type", "audio/mpeg")
w.Write([]byte("AUDIO"))
}))
t.Cleanup(srv.Close)
cfg.APIBase = srv.URL
p := openai.NewProvider(cfg)
_, err := p.Synthesize(t.Context(), "hello", opts)
if err != nil {
t.Fatalf("Synthesize failed: %v", err)
}
return captured
}
// TestCharacterization_DefaultOpts captures the request body for default opts
// and asserts the expected JSON fields. This is a golden-fixture characterization
// test — the exact shape MUST remain byte-compatible after the Synthesize refactor.
func TestCharacterization_DefaultOpts(t *testing.T) {
cfg := openai.Config{APIKey: "test-key"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{})
var m map[string]any
if err := json.Unmarshal(body, &m); err != nil {
t.Fatalf("body not valid JSON: %v — body: %s", err, body)
}
// Golden assertions for default-opts characterization.
assertJSONField(t, m, "model", "gpt-4o-mini-tts")
assertJSONField(t, m, "voice", "alloy")
assertJSONField(t, m, "response_format", "mp3")
assertJSONField(t, m, "input", "hello")
}
// TestCharacterization_ExplicitVoiceModel verifies the body reflects
// overridden voice/model without mutating any default path.
func TestCharacterization_ExplicitVoiceModel(t *testing.T) {
cfg := openai.Config{APIKey: "test-key"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{Voice: "nova", Model: "tts-1"})
var m map[string]any
if err := json.Unmarshal(body, &m); err != nil {
t.Fatalf("body not valid JSON: %v", err)
}
assertJSONField(t, m, "voice", "nova")
assertJSONField(t, m, "model", "tts-1")
}
func assertJSONField(t *testing.T, m map[string]any, key, want string) {
t.Helper()
v, ok := m[key]
if !ok {
t.Errorf("missing JSON field %q", key)
return
}
if s, _ := v.(string); s != want {
t.Errorf("field %q: got %q, want %q", key, s, want)
}
}
@@ -0,0 +1,41 @@
package openai_test
import (
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/openai"
)
// TestDefaults_PreserveLegacyBody verifies that populating opts.Params with
// all Capabilities defaults produces a body byte-equivalent to the nil-Params
// characterization fixture. This guards against "silent default flip" where a
// user saves an unchanged form, the blob stores schema defaults, and the next
// synthesize call sends different values upstream.
func TestDefaults_PreserveLegacyBody(t *testing.T) {
cfg := openai.Config{APIKey: "test-key"}
p := openai.NewProvider(cfg)
caps := p.Capabilities()
if len(caps.Params) == 0 {
t.Skip("Capabilities.Params not yet populated (Phase C enrichment pending)")
}
// Build params map from all schema defaults.
params := make(map[string]any)
for _, s := range caps.Params {
if s.Default != nil {
audio.SetNested(params, s.Key, s.Default)
}
}
// Capture body with defaults-from-schema.
bodyWithDefaults := captureOpenAIBody(t, cfg, audio.TTSOptions{Params: params})
// Capture body with nil Params (current legacy path).
bodyNilParams := captureOpenAIBody(t, cfg, audio.TTSOptions{})
if string(bodyWithDefaults) != string(bodyNilParams) {
t.Errorf("defaults-invariant FAILED:\n with-defaults: %s\n nil-params: %s",
bodyWithDefaults, bodyNilParams)
}
}
+135
View File
@@ -0,0 +1,135 @@
package openai_test
import (
"encoding/json"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/openai"
)
func TestSynthesize_AppliesParams_Speed(t *testing.T) {
cfg := openai.Config{APIKey: "k"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"speed": 1.5},
})
var m map[string]any
json.Unmarshal(body, &m)
if v, _ := m["speed"].(float64); v != 1.5 {
t.Errorf("speed: got %v, want 1.5", m["speed"])
}
}
func TestSynthesize_AppliesParams_ResponseFormat(t *testing.T) {
cfg := openai.Config{APIKey: "k"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{
Params: map[string]any{"response_format": "opus"},
})
var m map[string]any
json.Unmarshal(body, &m)
if v, _ := m["response_format"].(string); v != "opus" {
t.Errorf("response_format: got %q, want opus", m["response_format"])
}
}
func TestSynthesize_AppliesParams_Instructions(t *testing.T) {
cfg := openai.Config{APIKey: "k"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{
Model: "gpt-4o-mini-tts",
Params: map[string]any{"instructions": "speak slowly"},
})
var m map[string]any
json.Unmarshal(body, &m)
if v, _ := m["instructions"].(string); v != "speak slowly" {
t.Errorf("instructions: got %q, want 'speak slowly'", m["instructions"])
}
}
func TestSynthesize_AppliesParams_OmitsEmpty_NilParams(t *testing.T) {
cfg := openai.Config{APIKey: "k"}
body := captureOpenAIBody(t, cfg, audio.TTSOptions{})
var m map[string]any
json.Unmarshal(body, &m)
// instructions must be absent when not set
if _, ok := m["instructions"]; ok {
t.Error("instructions must not appear in body when not in params")
}
}
func TestSynthesize_DoesNotMutateCallerParams(t *testing.T) {
cfg := openai.Config{APIKey: "k"}
original := map[string]any{
"speed": 1.0,
"sentinel": "untouched",
}
// Deep copy to compare after call.
snapshot := map[string]any{
"speed": original["speed"],
"sentinel": original["sentinel"],
}
captureOpenAIBody(t, cfg, audio.TTSOptions{Params: original})
for k, want := range snapshot {
if got := original[k]; got != want {
t.Errorf("caller Params mutated: key %q was %v, now %v", k, want, got)
}
}
if len(original) != len(snapshot) {
t.Errorf("caller Params size changed: was %d, now %d", len(snapshot), len(original))
}
}
func TestCapabilities_HasParam_Speed(t *testing.T) {
p := openai.NewProvider(openai.Config{APIKey: "k"})
caps := p.Capabilities()
assertParamExists(t, caps.Params, "speed", audio.ParamTypeRange)
}
func TestCapabilities_HasParam_ResponseFormat(t *testing.T) {
p := openai.NewProvider(openai.Config{APIKey: "k"})
caps := p.Capabilities()
assertParamExists(t, caps.Params, "response_format", audio.ParamTypeEnum)
}
func TestCapabilities_DependsOn_OpenAIInstructions(t *testing.T) {
p := openai.NewProvider(openai.Config{APIKey: "k"})
caps := p.Capabilities()
for _, param := range caps.Params {
if param.Key == "instructions" {
if len(param.DependsOn) == 0 {
t.Fatal("instructions param must have DependsOn")
}
dep := param.DependsOn[0]
if dep.Field != "model" {
t.Errorf("DependsOn.Field: got %q, want model", dep.Field)
}
if dep.Value != "gpt-4o-mini-tts" {
t.Errorf("DependsOn.Value: got %v, want gpt-4o-mini-tts", dep.Value)
}
return
}
}
t.Error("instructions param not found in Capabilities")
}
func TestCapabilities_Has13Voices(t *testing.T) {
p := openai.NewProvider(openai.Config{APIKey: "k"})
caps := p.Capabilities()
if len(caps.Voices) != 13 {
t.Errorf("expected 13 voices, got %d", len(caps.Voices))
}
}
func assertParamExists(t *testing.T, params []audio.ParamSchema, key string, typ audio.ParamType) {
t.Helper()
for _, p := range params {
if p.Key == key {
if p.Type != typ {
t.Errorf("param %q: type got %q, want %q", key, p.Type, typ)
}
return
}
}
t.Errorf("param %q not found in schema", key)
}
+50 -4
View File
@@ -59,6 +59,9 @@ func NewProvider(cfg Config) *Provider {
func (p *Provider) Name() string { return "openai" }
// Synthesize calls POST {apiBase}/audio/speech.
// opts.Params keys: "speed" (float), "response_format" (string), "instructions" (string).
// When opts.Params is nil or a key is absent, schema defaults are used.
// MUST NOT mutate opts.Params — reads only.
func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOptions) (*audio.SynthResult, error) {
voice := opts.Voice
if voice == "" {
@@ -68,17 +71,23 @@ func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOp
if model == "" {
model = p.model
}
format := opts.Format
if format == "" {
format = "mp3"
}
// Resolve params: caller-supplied wins over schema default.
format := resolveString(opts.Params, "response_format", "mp3")
speed := resolveFloat(opts.Params, "speed", 1.0)
instructions := resolveString(opts.Params, "instructions", "")
body := map[string]any{
"model": model,
"input": text,
"voice": voice,
"response_format": format,
"speed": speed,
}
if instructions != "" {
body["instructions"] = instructions
}
bodyJSON, err := json.Marshal(body)
if err != nil {
return nil, fmt.Errorf("marshal openai tts request: %w", err)
@@ -122,3 +131,40 @@ func (p *Provider) Synthesize(ctx context.Context, text string, opts audio.TTSOp
MimeType: mime,
}, nil
}
// resolveString reads a string param from params map, falling back to def.
func resolveString(params map[string]any, key, def string) string {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
if s, ok := v.(string); ok && s != "" {
return s
}
return def
}
// resolveFloat reads a float param from params map, falling back to def.
func resolveFloat(params map[string]any, key string, def float64) float64 {
if params == nil {
return def
}
v, ok := audio.GetNested(params, key)
if !ok {
return def
}
switch n := v.(type) {
case float64:
return n
case float32:
return float64(n)
case int:
return float64(n)
case int64:
return float64(n)
}
return def
}
+30 -4
View File
@@ -40,11 +40,28 @@ type StreamResult struct {
MimeType string // e.g. "audio/mpeg", "audio/ogg"
}
// TTSOptions controls TTS synthesis parameters.
// TTSOptions carries per-request synthesis parameters.
//
// Params is a read-only map view owned by the caller.
// Implementations MUST NOT mutate it. If a provider needs to merge
// defaults with caller values, it MUST copy before writing.
// READ-ONLY constraint is documented here and enforced by convention.
type TTSOptions struct {
Voice string // provider-specific voice ID
Model string // provider-specific model ID
Format string // output format: "mp3", "opus", etc.
Voice string // provider-specific voice ID
Model string // provider-specific model ID
Format string // output format: "mp3", "opus", etc.
Speakers []SpeakerVoice // multi-speaker TTS (Gemini); len>0 switches to multi mode
// Params is a READ-ONLY map owned by the caller. Provider implementations
// MUST NOT mutate it; copy before merging with defaults.
Params map[string]any // provider-specific knobs (nested keys like "voice_settings.stability" allowed)
}
// SpeakerVoice binds a speaker label to a voice ID for multi-speaker TTS.
// JSON tags match the frontend MultiSpeakerEditor shape so the persisted
// tts.gemini.speakers blob round-trips without transformation.
type SpeakerVoice struct {
Speaker string `json:"speaker"` // e.g. "host", "guest"
VoiceID string `json:"voiceId"` // provider-specific voice ID
}
// SynthResult is the output of a TTS synthesis call.
@@ -72,6 +89,15 @@ const (
ModeAll Mode = "all" // All replies including tool/block
)
// VoiceListProvider is implemented by TTS providers that support dynamic
// voice listing. The HTTP voices handler uses this interface so it is not
// coupled to a specific provider implementation.
type VoiceListProvider interface {
// ListVoices returns the provider's available voices.
// Implementations may cache results internally.
ListVoices(ctx context.Context) ([]Voice, error)
}
// ---- STT (Phase 4) ----
// STTProvider transcribes audio bytes to text.
+118
View File
@@ -0,0 +1,118 @@
package audio
import (
"errors"
"fmt"
"strings"
)
// ParamType identifies how a ParamSchema field should be rendered in the UI.
type ParamType string
const (
ParamTypeRange ParamType = "range" // float slider with Min/Max/Step
ParamTypeNumber ParamType = "number" // free-form float input
ParamTypeInteger ParamType = "integer" // integer input
ParamTypeEnum ParamType = "enum" // dropdown from Enum list
ParamTypeBoolean ParamType = "boolean" // toggle
ParamTypeString ParamType = "string" // single-line text
ParamTypeText ParamType = "text" // multi-line text area
)
// EnumOption is a single option in a ParamTypeEnum field.
type EnumOption struct {
Value string `json:"value"`
Label string `json:"label"`
}
// Dependency declares a conditional visibility rule for a ParamSchema field.
// A field is visible only when ALL its DependsOn entries are satisfied (AND semantics).
type Dependency struct {
// Field is the sibling param key this dependency references.
Field string `json:"field"`
// Op is the comparison operator; currently only "eq" is supported.
Op string `json:"op"`
// Value is the expected value (string comparison after fmt.Sprint serialization).
Value any `json:"value"`
}
// VoiceOption is a static voice entry exposed in ProviderCapabilities.
type VoiceOption struct {
VoiceID string `json:"voice_id"`
Name string `json:"name"`
Language string `json:"language,omitempty"`
Gender string `json:"gender,omitempty"`
}
// ParamSchema describes a single configurable parameter for a TTS provider.
// The frontend DynamicParamForm uses this to render the appropriate input control.
type ParamSchema struct {
// Key is the dot-separated param path, e.g. "stability" or "voice_settings.stability".
Key string `json:"key"`
Type ParamType `json:"type"`
Label string `json:"label"`
Description string `json:"description,omitempty"`
Default any `json:"default,omitempty"`
Min *float64 `json:"min,omitempty"` // for range/number
Max *float64 `json:"max,omitempty"` // for range/number
Step *float64 `json:"step,omitempty"` // for range
Enum []EnumOption `json:"enum,omitempty"`
DependsOn []Dependency `json:"depends_on,omitempty"`
}
// ProviderCapabilities is the catalog entry for a single TTS provider.
// Returned by GET /v1/tts/capabilities in a { providers: [] } envelope.
type ProviderCapabilities struct {
// Provider is the stable machine identifier (e.g. "openai", "elevenlabs").
Provider string `json:"provider"`
// DisplayName is the human-readable label shown in the UI.
DisplayName string `json:"display_name"`
// RequiresAPIKey indicates whether the provider needs an API key configured.
RequiresAPIKey bool `json:"requires_api_key"`
// Models is the list of model IDs supported by this provider.
Models []string `json:"models,omitempty"`
// Voices is a static catalog of voices (nil = dynamic / fetched separately).
Voices []VoiceOption `json:"voices,omitempty"`
// Params is the ordered list of configurable parameters (nil = no custom params).
Params []ParamSchema `json:"params,omitempty"`
// CustomFeatures is an opaque map for provider-specific UI hints.
CustomFeatures map[string]any `json:"custom_features,omitempty"`
}
// DescribableProvider is the optional interface that TTS providers implement
// to expose their capability schema. Providers that do not implement this
// contribute a minimal stub { Provider, DisplayName } to ListCapabilities.
type DescribableProvider interface {
TTSProvider
Capabilities() ProviderCapabilities
}
// evaluateDependsOn returns true if all deps are satisfied by formState (AND semantics).
// An empty deps slice always returns true (unconditionally visible).
func evaluateDependsOn(deps []Dependency, formState map[string]any) bool {
for _, d := range deps {
v, ok := formState[d.Field]
if !ok {
return false
}
if fmt.Sprint(v) != fmt.Sprint(d.Value) {
return false
}
}
return true
}
// parseKeyPath splits a dot-separated key path into segments.
// Returns an error if the path is empty or contains empty segments.
func parseKeyPath(key string) ([]string, error) {
if key == "" {
return nil, errors.New("key path must not be empty")
}
parts := strings.Split(key, ".")
for _, p := range parts {
if p == "" {
return nil, errors.New("key path must not contain empty segments: " + key)
}
}
return parts, nil
}
+133
View File
@@ -0,0 +1,133 @@
package audio
import (
"encoding/json"
"os"
"strings"
"testing"
)
// TestParamSchema_RoundTrip verifies marshal/unmarshal of ParamSchema preserves all fields.
func TestParamSchema_RoundTrip(t *testing.T) {
orig := ParamSchema{
Key: "stability",
Type: ParamTypeRange,
Label: "Stability",
Description: "Voice stability",
Default: 0.5,
Min: floatPtr(0.0),
Max: floatPtr(1.0),
Step: floatPtr(0.01),
Enum: []EnumOption{{Value: "auto", Label: "Auto"}},
DependsOn: []Dependency{
{Field: "model", Op: "eq", Value: "eleven_v3"},
},
}
data, err := json.Marshal(orig)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var got ParamSchema
if err := json.Unmarshal(data, &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if got.Key != orig.Key {
t.Errorf("Key: got %q want %q", got.Key, orig.Key)
}
if got.Type != orig.Type {
t.Errorf("Type: got %q want %q", got.Type, orig.Type)
}
if got.Min == nil || *got.Min != *orig.Min {
t.Errorf("Min: got %v want %v", got.Min, orig.Min)
}
if got.Max == nil || *got.Max != *orig.Max {
t.Errorf("Max: got %v want %v", got.Max, orig.Max)
}
if got.Step == nil || *got.Step != *orig.Step {
t.Errorf("Step: got %v want %v", got.Step, orig.Step)
}
if len(got.Enum) != 1 || got.Enum[0].Value != "auto" {
t.Errorf("Enum: got %v want [{auto Auto}]", got.Enum)
}
if len(got.DependsOn) != 1 || got.DependsOn[0].Field != "model" {
t.Errorf("DependsOn: got %v", got.DependsOn)
}
}
// floatPtr is a helper for pointer-to-float64 in tests.
func floatPtr(v float64) *float64 { return &v }
// TestDependency_AndSemantics verifies evaluateDependsOn returns true only when ALL deps match.
func TestDependency_AndSemantics(t *testing.T) {
deps := []Dependency{
{Field: "model", Op: "eq", Value: "eleven_v3"},
{Field: "language", Op: "eq", Value: "en"},
}
// All match → true
if !evaluateDependsOn(deps, map[string]any{"model": "eleven_v3", "language": "en"}) {
t.Error("all deps match: expected true")
}
// One mismatch → false
if evaluateDependsOn(deps, map[string]any{"model": "eleven_v3", "language": "fr"}) {
t.Error("one dep mismatches: expected false")
}
// Empty deps → true (always visible)
if !evaluateDependsOn(nil, map[string]any{"model": "eleven_v3"}) {
t.Error("empty deps: expected true")
}
}
// TestNestedKeyPathParser verifies parseKeyPath splits keys on "." and rejects empty segments.
func TestNestedKeyPathParser(t *testing.T) {
tests := []struct {
input string
want []string
wantErr bool
}{
{"voice_settings.stability", []string{"voice_settings", "stability"}, false},
{"audio.sample_rate", []string{"audio", "sample_rate"}, false},
{"simple", []string{"simple"}, false},
{"", nil, true},
{"a..b", nil, true},
{".leading", nil, true},
{"trailing.", nil, true},
}
for _, tc := range tests {
got, err := parseKeyPath(tc.input)
if tc.wantErr {
if err == nil {
t.Errorf("parseKeyPath(%q): expected error, got %v", tc.input, got)
}
continue
}
if err != nil {
t.Errorf("parseKeyPath(%q): unexpected error: %v", tc.input, err)
continue
}
if len(got) != len(tc.want) {
t.Errorf("parseKeyPath(%q) = %v, want %v", tc.input, got, tc.want)
continue
}
for i := range got {
if got[i] != tc.want[i] {
t.Errorf("parseKeyPath(%q)[%d] = %q, want %q", tc.input, i, got[i], tc.want[i])
}
}
}
}
// TestTTSOptions_ParamsReadOnlyDoc verifies the Params field has a godoc comment
// declaring the read-only contract by reading the actual source file.
func TestTTSOptions_ParamsReadOnlyDoc(t *testing.T) {
data, err := os.ReadFile("types.go")
if err != nil {
t.Fatalf("read types.go: %v", err)
}
src := string(data)
if !strings.Contains(src, "READ-ONLY") {
t.Error("types.go: TTSOptions.Params must have READ-ONLY doc comment")
}
if !strings.Contains(src, "MUST NOT mutate") {
t.Error("types.go: TTSOptions.Params comment must say 'MUST NOT mutate'")
}
}
+11 -1
View File
@@ -442,7 +442,7 @@ type SessionsConfig struct {
// TtsConfig configures text-to-speech.
// Matching TS src/config/types.tts.ts.
type TtsConfig struct {
Provider string `json:"provider,omitempty"` // "openai", "elevenlabs", "edge", "minimax"
Provider string `json:"provider,omitempty"` // "openai", "elevenlabs", "edge", "minimax", "gemini"
Auto string `json:"auto,omitempty"` // "off" (default), "always", "inbound", "tagged"
Mode string `json:"mode,omitempty"` // "final" (default), "all"
MaxLength int `json:"max_length,omitempty"` // max text length before truncation (default 1500)
@@ -451,6 +451,16 @@ type TtsConfig struct {
ElevenLabs TtsElevenLabsConfig `json:"elevenlabs"`
Edge TtsEdgeConfig `json:"edge"`
MiniMax TtsMiniMaxConfig `json:"minimax"`
Gemini TtsGeminiConfig `json:"gemini"`
}
// TtsGeminiConfig configures the Google Gemini TTS provider.
type TtsGeminiConfig struct {
APIKey string `json:"api_key,omitempty"` // required; encrypted at rest
APIBase string `json:"api_base,omitempty"` // custom endpoint (optional; SSRF-gated)
Voice string `json:"voice,omitempty"` // default "Kore"
Model string `json:"model,omitempty"` // default "gemini-2.5-flash-preview-tts"
Speakers string `json:"speakers,omitempty"` // JSON-encoded []SpeakerVoice for multi-speaker mode
}
// TtsOpenAIConfig configures the OpenAI TTS provider.
+91 -13
View File
@@ -13,6 +13,7 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
"github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
"github.com/nextlevelbuilder/goclaw/internal/i18n"
"github.com/nextlevelbuilder/goclaw/internal/permissions"
"github.com/nextlevelbuilder/goclaw/internal/store"
@@ -58,6 +59,7 @@ func (h *TTSHandler) RegisterRoutes(mux *http.ServeMux) {
requireAuth(permissions.RoleOperator, h.handleSynthesize))
mux.HandleFunc("POST /v1/tts/test-connection",
requireAuth(permissions.RoleOperator, h.handleTestConnection))
h.registerCapabilitiesRoute(mux)
}
// synthesizeRequest is the JSON body for POST /v1/tts/synthesize.
@@ -112,7 +114,7 @@ func (h *TTSHandler) handleSynthesize(w http.ResponseWriter, r *http.Request) {
}
// Try per-tenant TTS config first, fall back to global manager.
p, name, err := h.resolveTenantProvider(ctx, req.Provider)
p, name, tenantParams, err := h.resolveTenantProvider(ctx, req.Provider)
if err != nil {
// Tenant config lookup failed, try global manager
h.mu.RLock()
@@ -147,7 +149,12 @@ func (h *TTSHandler) handleSynthesize(w http.ResponseWriter, r *http.Request) {
synthCtx, cancel := context.WithTimeout(ctx, synthesizeTimeout)
defer cancel()
opts := audio.TTSOptions{Voice: req.VoiceID, Model: req.ModelID}
opts := audio.TTSOptions{Voice: req.VoiceID, Model: req.ModelID, Params: tenantParams}
// Restore saved multi-speaker defaults (Gemini) so persisted config reactivates
// at runtime instead of silently falling through to single-voice mode.
if speakers := loadSavedSpeakers(ctx, h.systemConfigs, name); len(speakers) > 0 {
opts.Speakers = speakers
}
start := time.Now()
result, err := p.Synthesize(synthCtx, req.Text, opts)
dur := time.Since(start)
@@ -158,8 +165,29 @@ func (h *TTSHandler) handleSynthesize(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"error":"synthesis timeout"}`, http.StatusGatewayTimeout)
return
}
// Translate Gemini sentinel errors to 422 Unprocessable Entity with i18n message.
if errors.Is(err, gemini.ErrInvalidVoice) {
slog.Warn("tts.synthesize.invalid-params", "provider", name, "error", err)
msg := i18n.T(locale, i18n.MsgTtsGeminiInvalidVoice, req.VoiceID)
http.Error(w, fmt.Sprintf(`{"error":%q}`, msg), http.StatusUnprocessableEntity)
return
}
if errors.Is(err, gemini.ErrSpeakerLimit) {
slog.Warn("tts.synthesize.invalid-params", "provider", name, "error", err)
msg := i18n.T(locale, i18n.MsgTtsGeminiSpeakerLimit)
http.Error(w, fmt.Sprintf(`{"error":%q}`, msg), http.StatusUnprocessableEntity)
return
}
if errors.Is(err, gemini.ErrInvalidModel) {
slog.Warn("tts.synthesize.invalid-params", "provider", name, "error", err)
msg := i18n.T(locale, i18n.MsgTtsGeminiInvalidModel, req.ModelID)
http.Error(w, fmt.Sprintf(`{"error":%q}`, msg), http.StatusUnprocessableEntity)
return
}
// Surface upstream error to caller — opaque "upstream synthesis failed"
// makes the test playground useless for debugging provider config.
slog.Warn("tts.synthesize.failed", "provider", name, "error", err)
http.Error(w, `{"error":"upstream synthesis failed"}`, http.StatusBadGateway)
http.Error(w, fmt.Sprintf(`{"error":%q}`, err.Error()), http.StatusBadGateway)
return
}
@@ -171,10 +199,12 @@ func (h *TTSHandler) handleSynthesize(w http.ResponseWriter, r *http.Request) {
}
// resolveTenantProvider attempts to create a TTS provider from tenant-specific config.
// Returns (nil, "", error) if tenant has no TTS config — caller should fall back to global manager.
func (h *TTSHandler) resolveTenantProvider(ctx context.Context, explicitProvider string) (audio.TTSProvider, string, error) {
// Performs dual-read: reads legacy flat keys AND tts.{p}.params JSON blob.
// Blob wins on key conflict; legacy flat keys fill gaps.
// Returns (nil, "", nil, error) if tenant has no TTS config — caller should fall back to global manager.
func (h *TTSHandler) resolveTenantProvider(ctx context.Context, explicitProvider string) (audio.TTSProvider, string, map[string]any, error) {
if h.systemConfigs == nil || h.configSecrets == nil {
return nil, "", fmt.Errorf("stores not configured")
return nil, "", nil, fmt.Errorf("stores not configured")
}
// Get tenant's configured provider
@@ -183,7 +213,7 @@ func (h *TTSHandler) resolveTenantProvider(ctx context.Context, explicitProvider
var err error
providerName, err = h.systemConfigs.Get(ctx, "tts.provider")
if err != nil || providerName == "" {
return nil, "", fmt.Errorf("no tenant tts provider")
return nil, "", nil, fmt.Errorf("no tenant tts provider")
}
}
@@ -195,46 +225,94 @@ func (h *TTSHandler) resolveTenantProvider(ctx context.Context, explicitProvider
if key, _ := h.configSecrets.Get(ctx, "tts.openai.api_key"); key != "" {
req.APIKey = key
} else {
return nil, "", fmt.Errorf("no api key")
return nil, "", nil, fmt.Errorf("no api key")
}
req.APIBase, _ = h.systemConfigs.Get(ctx, "tts.openai.api_base")
req.VoiceID, _ = h.systemConfigs.Get(ctx, "tts.openai.voice")
req.ModelID, _ = h.systemConfigs.Get(ctx, "tts.openai.model")
req.Params = loadParamsBlob(ctx, h.systemConfigs, "tts.openai.params")
case "elevenlabs":
if key, _ := h.configSecrets.Get(ctx, "tts.elevenlabs.api_key"); key != "" {
req.APIKey = key
} else {
return nil, "", fmt.Errorf("no api key")
return nil, "", nil, fmt.Errorf("no api key")
}
req.APIBase, _ = h.systemConfigs.Get(ctx, "tts.elevenlabs.api_base")
req.VoiceID, _ = h.systemConfigs.Get(ctx, "tts.elevenlabs.voice")
req.ModelID, _ = h.systemConfigs.Get(ctx, "tts.elevenlabs.model")
req.Params = loadParamsBlob(ctx, h.systemConfigs, "tts.elevenlabs.params")
case "minimax":
if key, _ := h.configSecrets.Get(ctx, "tts.minimax.api_key"); key != "" {
req.APIKey = key
} else {
return nil, "", fmt.Errorf("no api key")
return nil, "", nil, fmt.Errorf("no api key")
}
req.GroupID, _ = h.configSecrets.Get(ctx, "tts.minimax.group_id")
req.APIBase, _ = h.systemConfigs.Get(ctx, "tts.minimax.api_base")
req.VoiceID, _ = h.systemConfigs.Get(ctx, "tts.minimax.voice")
req.ModelID, _ = h.systemConfigs.Get(ctx, "tts.minimax.model")
req.Params = loadParamsBlob(ctx, h.systemConfigs, "tts.minimax.params")
case "edge":
req.VoiceID, _ = h.systemConfigs.Get(ctx, "tts.edge.voice")
req.Rate, _ = h.systemConfigs.Get(ctx, "tts.edge.rate")
req.Params = loadParamsBlob(ctx, h.systemConfigs, "tts.edge.params")
case "gemini":
if key, _ := h.configSecrets.Get(ctx, "tts.gemini.api_key"); key != "" {
req.APIKey = key
} else {
return nil, "", nil, fmt.Errorf("no api key")
}
req.APIBase, _ = h.systemConfigs.Get(ctx, "tts.gemini.api_base")
req.VoiceID, _ = h.systemConfigs.Get(ctx, "tts.gemini.voice")
req.ModelID, _ = h.systemConfigs.Get(ctx, "tts.gemini.model")
req.Params = loadParamsBlob(ctx, h.systemConfigs, "tts.gemini.params")
default:
return nil, "", fmt.Errorf("unsupported provider: %s", providerName)
return nil, "", nil, fmt.Errorf("unsupported provider: %s", providerName)
}
provider, err := createEphemeralTTSProvider(req)
if err != nil {
return nil, "", err
return nil, "", nil, err
}
slog.Debug("tts: using tenant provider", "provider", providerName, "tenant", store.TenantIDFromContext(ctx))
return provider, providerName, nil
return provider, providerName, req.Params, nil
}
// loadParamsBlob reads a JSON blob from system_configs and returns it as map[string]any.
// Returns nil when the key is absent or the value is not valid JSON.
func loadParamsBlob(ctx context.Context, sc store.SystemConfigStore, key string) map[string]any {
raw, err := sc.Get(ctx, key)
if err != nil || raw == "" {
return nil
}
var m map[string]any
if err := json.Unmarshal([]byte(raw), &m); err != nil {
slog.Warn("tts: invalid params blob", "key", key, "error", err)
return nil
}
return m
}
// loadSavedSpeakers reads the per-tenant multi-speaker JSON blob for providers
// that persist speaker configuration (currently only Gemini). Returns nil when
// no config exists or decode fails — caller falls back to single-voice mode.
func loadSavedSpeakers(ctx context.Context, sc store.SystemConfigStore, providerName string) []audio.SpeakerVoice {
if sc == nil || providerName != "gemini" {
return nil
}
raw, err := sc.Get(ctx, "tts.gemini.speakers")
if err != nil || raw == "" {
return nil
}
var speakers []audio.SpeakerVoice
if err := json.Unmarshal([]byte(raw), &speakers); err != nil {
return nil
}
return speakers
}
+70
View File
@@ -0,0 +1,70 @@
package http
import (
"net/http"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/edge"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
"github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
"github.com/nextlevelbuilder/goclaw/internal/audio/openai"
"github.com/nextlevelbuilder/goclaw/internal/permissions"
)
// ttsCapabilitiesResponse is the JSON envelope for GET /v1/tts/capabilities.
type ttsCapabilitiesResponse struct {
Providers []audio.ProviderCapabilities `json:"providers"`
}
// builtinTTSCapabilities returns the static capability catalog for every known
// TTS provider — independent of which providers happen to be registered in the
// manager at runtime. Frontend needs this so users can browse + configure a
// provider before saving credentials. Each provider's Capabilities() reads no
// receiver state, so calling on a zero-value Provider is safe.
func builtinTTSCapabilities() []audio.ProviderCapabilities {
return []audio.ProviderCapabilities{
(&openai.Provider{}).Capabilities(),
(&elevenlabs.TTSProvider{}).Capabilities(),
(&edge.Provider{}).Capabilities(),
(&minimax.Provider{}).Capabilities(),
(&gemini.Provider{}).Capabilities(),
}
}
// handleCapabilities serves GET /v1/tts/capabilities.
// Returns catalog-level metadata for all known TTS providers — registered
// providers in the manager take precedence; unregistered ones fall back to
// the static builtin catalog so users can configure a fresh provider.
// Response is tenant-agnostic.
func (h *TTSHandler) handleCapabilities(w http.ResponseWriter, r *http.Request) {
h.mu.RLock()
mgr := h.manager
h.mu.RUnlock()
var registered []audio.ProviderCapabilities
if mgr != nil {
registered = mgr.ListCapabilities()
}
seen := make(map[string]bool, len(registered))
caps := make([]audio.ProviderCapabilities, 0, 5)
for _, c := range registered {
caps = append(caps, c)
seen[c.Provider] = true
}
for _, c := range builtinTTSCapabilities() {
if !seen[c.Provider] {
caps = append(caps, c)
}
}
writeJSON(w, http.StatusOK, ttsCapabilitiesResponse{Providers: caps})
}
// registerCapabilitiesRoute wires GET /v1/tts/capabilities onto mux.
// Called from TTSHandler.RegisterRoutes to keep all TTS routes co-located.
func (h *TTSHandler) registerCapabilitiesRoute(mux *http.ServeMux) {
mux.HandleFunc("GET /v1/tts/capabilities",
requireAuth(permissions.RoleOperator, h.handleCapabilities))
}
+272
View File
@@ -0,0 +1,272 @@
package http
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/crypto"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/google/uuid"
)
// capsDescribableProvider is a TTSProvider that implements DescribableProvider.
type capsDescribableProvider struct{ name string }
func (p *capsDescribableProvider) Name() string { return p.name }
func (p *capsDescribableProvider) Synthesize(_ context.Context, _ string, _ audio.TTSOptions) (*audio.SynthResult, error) {
return nil, nil
}
func (p *capsDescribableProvider) Capabilities() audio.ProviderCapabilities {
return audio.ProviderCapabilities{
Provider: p.name,
DisplayName: p.name + " TTS",
Models: []string{"model-x"},
}
}
// newCapsMux creates a ServeMux wired with a TTSHandler (which handles capabilities).
func newCapsMux(mgr *audio.Manager) *http.ServeMux {
h := NewTTSHandler(mgr)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
return mux
}
// TestCapabilitiesHandler_RequiresAuth verifies 401 when no token is provided
// and a gateway token is configured.
func TestCapabilitiesHandler_RequiresAuth(t *testing.T) {
setupTestToken(t, "caps-test-token")
mgr := audio.NewManager(audio.ManagerConfig{Primary: "mock"})
mgr.RegisterTTS(&mockTTSProvider{name: "mock"})
mux := newCapsMux(mgr)
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Errorf("want 401, got %d: %s", rr.Code, rr.Body.String())
}
}
// TestCapabilitiesHandler_RoleOperator verifies viewer gets 403, operator gets 200.
func TestCapabilitiesHandler_RoleOperator(t *testing.T) {
setupTestToken(t, "caps-test-token")
viewerRaw := "viewer-caps-key"
operatorRaw := "operator-caps-key"
setupTestCache(t, map[string]*store.APIKeyData{
crypto.HashAPIKey(viewerRaw): {
ID: uuid.New(),
Scopes: []string{"operator.read"}, // viewer-level scope
},
crypto.HashAPIKey(operatorRaw): {
ID: uuid.New(),
Scopes: []string{"operator.write"}, // operator scope
},
})
mgr := audio.NewManager(audio.ManagerConfig{Primary: "mock"})
mgr.RegisterTTS(&mockTTSProvider{name: "mock"})
mux := newCapsMux(mgr)
// Viewer → 403
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
req.Header.Set("Authorization", "Bearer "+viewerRaw)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusForbidden {
t.Errorf("viewer: want 403, got %d: %s", rr.Code, rr.Body.String())
}
// Operator → 200
req2 := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
req2.Header.Set("Authorization", "Bearer "+operatorRaw)
rr2 := httptest.NewRecorder()
mux.ServeHTTP(rr2, req2)
if rr2.Code != http.StatusOK {
t.Errorf("operator: want 200, got %d: %s", rr2.Code, rr2.Body.String())
}
}
// TestCapabilitiesHandler_JSONShape verifies response decodes into {providers: []ProviderCapabilities}.
func TestCapabilitiesHandler_JSONShape(t *testing.T) {
setupTestToken(t, "caps-test-token")
operatorRaw := "operator-shape-key"
setupTestCache(t, map[string]*store.APIKeyData{
crypto.HashAPIKey(operatorRaw): {
ID: uuid.New(),
Scopes: []string{"operator.write"},
},
})
mgr := audio.NewManager(audio.ManagerConfig{Primary: "describable"})
mgr.RegisterTTS(&capsDescribableProvider{name: "describable"})
mux := newCapsMux(mgr)
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
req.Header.Set("Authorization", "Bearer "+operatorRaw)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("want 200, got %d: %s", rr.Code, rr.Body.String())
}
var resp struct {
Providers []audio.ProviderCapabilities `json:"providers"`
}
if err := json.NewDecoder(rr.Body).Decode(&resp); err != nil {
t.Fatalf("decode response: %v", err)
}
if len(resp.Providers) == 0 {
t.Error("expected at least 1 provider in response")
}
if resp.Providers[0].Provider == "" {
t.Error("first provider has empty Provider field")
}
}
// TestCapabilitiesHandler_TenantIsolation verifies capabilities are catalog-level
// (not tenant-scoped) — same response for two different tenant tokens.
func TestCapabilitiesHandler_TenantIsolation(t *testing.T) {
setupTestToken(t, "caps-test-token")
tenant1Key := "tenant1-caps-key"
tenant2Key := "tenant2-caps-key"
tid1 := uuid.New()
tid2 := uuid.New()
setupTestCache(t, map[string]*store.APIKeyData{
crypto.HashAPIKey(tenant1Key): {
ID: uuid.New(),
Scopes: []string{"operator.write"},
TenantID: tid1,
},
crypto.HashAPIKey(tenant2Key): {
ID: uuid.New(),
Scopes: []string{"operator.write"},
TenantID: tid2,
},
})
mgr := audio.NewManager(audio.ManagerConfig{Primary: "prov-a"})
mgr.RegisterTTS(&capsDescribableProvider{name: "prov-a"})
mux := newCapsMux(mgr)
getProviders := func(token string) []audio.ProviderCapabilities {
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
req.Header.Set("Authorization", "Bearer "+token)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("token %s: want 200, got %d", token, rr.Code)
}
var resp struct {
Providers []audio.ProviderCapabilities `json:"providers"`
}
if err := json.NewDecoder(rr.Body).Decode(&resp); err != nil {
t.Fatalf("decode: %v", err)
}
return resp.Providers
}
p1 := getProviders(tenant1Key)
p2 := getProviders(tenant2Key)
if len(p1) != len(p2) {
t.Errorf("tenant isolation: different provider counts: %d vs %d", len(p1), len(p2))
}
for i := range p1 {
if p1[i].Provider != p2[i].Provider {
t.Errorf("tenant isolation: provider[%d] differs: %q vs %q", i, p1[i].Provider, p2[i].Provider)
}
}
}
// TestCapabilitiesHandler_BuiltinCatalog verifies that capabilities returns
// the full builtin catalog (5 providers) even when none are registered in the
// manager — frontend needs this so users can configure a fresh provider.
func TestCapabilitiesHandler_BuiltinCatalog(t *testing.T) {
setupTestToken(t, "")
mgr := audio.NewManager(audio.ManagerConfig{}) // no providers registered
mux := newCapsMux(mgr)
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("want 200, got %d: %s", rr.Code, rr.Body.String())
}
var resp struct {
Providers []audio.ProviderCapabilities `json:"providers"`
}
if err := json.NewDecoder(rr.Body).Decode(&resp); err != nil {
t.Fatalf("decode: %v", err)
}
want := map[string]bool{"openai": true, "elevenlabs": true, "edge": true, "minimax": true, "gemini": true}
got := make(map[string]bool, len(resp.Providers))
for _, p := range resp.Providers {
got[p.Provider] = true
}
for name := range want {
if !got[name] {
t.Errorf("builtin catalog missing %q; got providers=%v", name, got)
}
}
// Gemini specifically must expose static voices for the frontend voice picker.
for _, p := range resp.Providers {
if p.Provider == "gemini" {
if len(p.Voices) == 0 {
t.Error("gemini capabilities returned 0 voices — frontend will fall back to ElevenLabs picker")
}
break
}
}
}
// TestCapabilitiesHandler_RegisteredOverridesBuiltin verifies that a registered
// provider takes precedence over the builtin catalog entry of the same name.
func TestCapabilitiesHandler_RegisteredOverridesBuiltin(t *testing.T) {
setupTestToken(t, "")
mgr := audio.NewManager(audio.ManagerConfig{})
// Register a stub that returns DisplayName "custom-openai" for provider "openai".
mgr.RegisterTTS(&capsDescribableProvider{name: "openai"})
mux := newCapsMux(mgr)
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
var resp struct {
Providers []audio.ProviderCapabilities `json:"providers"`
}
_ = json.NewDecoder(rr.Body).Decode(&resp)
var openaiCount int
var openaiDisplay string
for _, p := range resp.Providers {
if p.Provider == "openai" {
openaiCount++
openaiDisplay = p.DisplayName
}
}
if openaiCount != 1 {
t.Errorf("want exactly 1 openai entry, got %d", openaiCount)
}
if openaiDisplay != "openai TTS" {
t.Errorf("registered provider should override builtin, got DisplayName=%q", openaiDisplay)
}
}
+126 -20
View File
@@ -44,19 +44,22 @@ type ttsConfigResponse struct {
ElevenLabs ttsProviderConfigResponse `json:"elevenlabs"`
Edge ttsProviderConfigResponse `json:"edge"`
MiniMax ttsProviderConfigResponse `json:"minimax"`
Gemini ttsProviderConfigResponse `json:"gemini"`
}
type ttsProviderConfigResponse struct {
APIKey string `json:"api_key,omitempty"` // masked
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
Voice string `json:"voice,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
Model string `json:"model,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
Rate string `json:"rate,omitempty"`
APIKey string `json:"api_key,omitempty"` // masked
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
Voice string `json:"voice,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
Model string `json:"model,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
Rate string `json:"rate,omitempty"`
Speakers string `json:"speakers,omitempty"` // JSON-encoded []SpeakerVoice (Gemini multi-speaker)
Params map[string]any `json:"params,omitempty"` // provider-specific params blob
}
// handleGet returns TTS config for the current tenant.
@@ -140,6 +143,36 @@ func (h *TTSConfigHandler) handleGet(w http.ResponseWriter, r *http.Request) {
resp.MiniMax.Model = v
resp.MiniMax.ModelID = v
}
if v, _ := h.systemConfigs.Get(ctx, "tts.gemini.api_base"); v != "" {
resp.Gemini.APIBase = v
}
if v, _ := h.systemConfigs.Get(ctx, "tts.gemini.voice"); v != "" {
resp.Gemini.Voice = v
resp.Gemini.VoiceID = v
}
if v, _ := h.systemConfigs.Get(ctx, "tts.gemini.model"); v != "" {
resp.Gemini.Model = v
resp.Gemini.ModelID = v
}
if v, _ := h.systemConfigs.Get(ctx, "tts.gemini.speakers"); v != "" {
resp.Gemini.Speakers = v
}
// Params blobs (dual-read: legacy flat keys already loaded above; return blob as-is for UI)
if v, _ := h.systemConfigs.Get(ctx, "tts.openai.params"); v != "" {
_ = json.Unmarshal([]byte(v), &resp.OpenAI.Params)
}
if v, _ := h.systemConfigs.Get(ctx, "tts.elevenlabs.params"); v != "" {
_ = json.Unmarshal([]byte(v), &resp.ElevenLabs.Params)
}
if v, _ := h.systemConfigs.Get(ctx, "tts.edge.params"); v != "" {
_ = json.Unmarshal([]byte(v), &resp.Edge.Params)
}
if v, _ := h.systemConfigs.Get(ctx, "tts.minimax.params"); v != "" {
_ = json.Unmarshal([]byte(v), &resp.MiniMax.Params)
}
if v, _ := h.systemConfigs.Get(ctx, "tts.gemini.params"); v != "" {
_ = json.Unmarshal([]byte(v), &resp.Gemini.Params)
}
}
// Load secrets (masked)
@@ -156,6 +189,9 @@ func (h *TTSConfigHandler) handleGet(w http.ResponseWriter, r *http.Request) {
if v, _ := h.configSecrets.Get(ctx, "tts.minimax.group_id"); v != "" {
resp.MiniMax.GroupID = v // not a secret, but stored with secrets for grouping
}
if v, _ := h.configSecrets.Get(ctx, "tts.gemini.api_key"); v != "" {
resp.Gemini.APIKey = "***"
}
}
w.Header().Set("Content-Type", "application/json")
@@ -173,19 +209,22 @@ type ttsConfigSaveRequest struct {
ElevenLabs *ttsProviderSaveRequest `json:"elevenlabs,omitempty"`
Edge *ttsProviderSaveRequest `json:"edge,omitempty"`
MiniMax *ttsProviderSaveRequest `json:"minimax,omitempty"`
Gemini *ttsProviderSaveRequest `json:"gemini,omitempty"`
}
type ttsProviderSaveRequest struct {
APIKey string `json:"api_key,omitempty"`
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
Voice string `json:"voice,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
Model string `json:"model,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
Rate string `json:"rate,omitempty"`
APIKey string `json:"api_key,omitempty"`
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
Voice string `json:"voice,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
Model string `json:"model,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
Rate string `json:"rate,omitempty"`
Speakers string `json:"speakers,omitempty"` // JSON-encoded []SpeakerVoice (Gemini multi-speaker)
Params map[string]any `json:"params,omitempty"` // provider-specific params blob
}
// handleSave saves TTS config for the current tenant.
@@ -269,6 +308,46 @@ func (h *TTSConfigHandler) handleSave(w http.ResponseWriter, r *http.Request) {
return
}
}
if req.Gemini != nil {
if v := req.Gemini.resolvedAPIBase(); v != "" && !set("tts.gemini.api_base", v, "gemini api_base") {
return
}
if v := req.Gemini.resolvedVoice(); v != "" && !set("tts.gemini.voice", v, "gemini voice") {
return
}
if v := req.Gemini.resolvedModel(); v != "" && !set("tts.gemini.model", v, "gemini model") {
return
}
if req.Gemini.Speakers != "" && !set("tts.gemini.speakers", req.Gemini.Speakers, "gemini speakers") {
return
}
}
// Dual-write: save params blobs alongside legacy flat keys for rollback safety.
if req.OpenAI != nil && req.OpenAI.Params != nil {
if !saveParamsBlob(w, ctx, h.systemConfigs.Set, "tts.openai.params", req.OpenAI.Params) {
return
}
}
if req.ElevenLabs != nil && req.ElevenLabs.Params != nil {
if !saveParamsBlob(w, ctx, h.systemConfigs.Set, "tts.elevenlabs.params", req.ElevenLabs.Params) {
return
}
}
if req.Edge != nil && req.Edge.Params != nil {
if !saveParamsBlob(w, ctx, h.systemConfigs.Set, "tts.edge.params", req.Edge.Params) {
return
}
}
if req.MiniMax != nil && req.MiniMax.Params != nil {
if !saveParamsBlob(w, ctx, h.systemConfigs.Set, "tts.minimax.params", req.MiniMax.Params) {
return
}
}
if req.Gemini != nil && req.Gemini.Params != nil {
if !saveParamsBlob(w, ctx, h.systemConfigs.Set, "tts.gemini.params", req.Gemini.Params) {
return
}
}
}
// Save secrets (only if not masked)
@@ -298,6 +377,11 @@ func (h *TTSConfigHandler) handleSave(w http.ResponseWriter, r *http.Request) {
}
}
}
if req.Gemini != nil && req.Gemini.APIKey != "" && req.Gemini.APIKey != "***" {
if !set("tts.gemini.api_key", req.Gemini.APIKey, "gemini api_key") {
return
}
}
}
slog.Info("tts.config: saved", "tenant", tid, "provider", req.Provider)
@@ -365,6 +449,16 @@ func NewTenantTTSResolver(sc store.SystemConfigStore, cs store.ConfigSecretsStor
req.VoiceID, _ = sc.Get(ctx, "tts.edge.voice")
req.Rate, _ = sc.Get(ctx, "tts.edge.rate")
case "gemini":
if key, _ := cs.Get(ctx, "tts.gemini.api_key"); key != "" {
req.APIKey = key
} else {
return nil, "", "", fmt.Errorf("no api key")
}
req.APIBase, _ = sc.Get(ctx, "tts.gemini.api_base")
req.VoiceID, _ = sc.Get(ctx, "tts.gemini.voice")
req.ModelID, _ = sc.Get(ctx, "tts.gemini.model")
default:
return nil, "", "", fmt.Errorf("unsupported provider: %s", providerName)
}
@@ -417,3 +511,15 @@ func (r *ttsProviderSaveRequest) resolvedModel() string {
}
return r.ModelID
}
// saveParamsBlob serialises params to JSON and persists it via setFn.
// Returns false and writes 500 on error.
func saveParamsBlob(w http.ResponseWriter, ctx context.Context, setFn func(context.Context, string, string) error, key string, params map[string]any) bool {
blob, err := json.Marshal(params)
if err != nil {
slog.Error("tts.config: failed to marshal params blob", "key", key, "error", err)
http.Error(w, fmt.Sprintf(`{"error":"marshal params %s: %s"}`, key, err.Error()), http.StatusInternalServerError)
return false
}
return saveOrFail(w, ctx, setFn, key, string(blob), key)
}
+227
View File
@@ -0,0 +1,227 @@
package http
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/store"
)
// ttsConfigDualReadCtx returns a context with a non-nil tenant ID for config handler tests.
func ttsConfigDualReadCtx() context.Context {
return store.WithTenantID(context.Background(), uuid.New())
}
// buildConfigSaveBody marshals req to JSON and returns a request body reader.
func buildConfigSaveBody(t *testing.T, v any) *bytes.Buffer {
t.Helper()
b, err := json.Marshal(v)
if err != nil {
t.Fatalf("marshal: %v", err)
}
return bytes.NewBuffer(b)
}
// configGET issues GET /v1/tts/config with tenant context.
func configGET(mux *http.ServeMux) *httptest.ResponseRecorder {
req := httptest.NewRequest("GET", "/v1/tts/config", nil)
req = req.WithContext(ttsConfigDualReadCtx())
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
return rr
}
// configPOST issues POST /v1/tts/config with tenant context.
func configPOST(mux *http.ServeMux, body any, t *testing.T) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest("POST", "/v1/tts/config", buildConfigSaveBody(t, body))
req.Header.Set("Content-Type", "application/json")
req = req.WithContext(ttsConfigDualReadCtx())
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
return rr
}
// newDualReadMux creates a config handler mux with pre-populated stores.
// It bypasses auth by directly setting context — suitable for unit tests.
func newDualReadMux(sc store.SystemConfigStore, cs store.ConfigSecretsStore) *http.ServeMux {
h := NewTTSConfigHandler(sc, cs)
mux := http.NewServeMux()
// Register without requireAuth so we can inject context directly.
mux.HandleFunc("GET /v1/tts/config", func(w http.ResponseWriter, r *http.Request) {
h.handleGet(w, r)
})
mux.HandleFunc("POST /v1/tts/config", func(w http.ResponseWriter, r *http.Request) {
h.handleSave(w, r)
})
return mux
}
// TestTTSConfig_DualRead_LegacyOnly verifies that legacy flat keys are returned in GET.
func TestTTSConfig_DualRead_LegacyOnly(t *testing.T) {
sc := &validationSystemConfigStore{data: map[string]string{
"tts.provider": "openai",
"tts.openai.voice": "alloy",
"tts.openai.model": "gpt-4o-mini-tts",
}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newDualReadMux(sc, cs)
rr := configGET(mux)
if rr.Code != http.StatusOK {
t.Fatalf("GET config: want 200, got %d: %s", rr.Code, rr.Body.String())
}
var resp ttsConfigResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if resp.Provider != "openai" {
t.Errorf("provider: got %q, want openai", resp.Provider)
}
if resp.OpenAI.Voice != "alloy" {
t.Errorf("openai.voice: got %q, want alloy", resp.OpenAI.Voice)
}
if resp.OpenAI.Params != nil {
t.Errorf("openai.params: expected nil when no blob stored, got %v", resp.OpenAI.Params)
}
}
// TestTTSConfig_DualRead_BlobOnly verifies that a stored params blob is returned.
func TestTTSConfig_DualRead_BlobOnly(t *testing.T) {
sc := &validationSystemConfigStore{data: map[string]string{
"tts.provider": "elevenlabs",
"tts.elevenlabs.params": `{"voice_settings":{"stability":0.8}}`,
}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newDualReadMux(sc, cs)
rr := configGET(mux)
if rr.Code != http.StatusOK {
t.Fatalf("GET config: want 200, got %d", rr.Code)
}
var resp ttsConfigResponse
json.Unmarshal(rr.Body.Bytes(), &resp)
if resp.ElevenLabs.Params == nil {
t.Fatal("elevenlabs.params: expected non-nil")
}
vs, _ := resp.ElevenLabs.Params["voice_settings"].(map[string]any)
if vs == nil {
t.Fatal("voice_settings: expected map")
}
if vs["stability"] != 0.8 {
t.Errorf("stability: got %v, want 0.8", vs["stability"])
}
}
// TestTTSConfig_DualRead_BothPresent verifies that blob is returned alongside flat keys.
func TestTTSConfig_DualRead_BothPresent(t *testing.T) {
sc := &validationSystemConfigStore{data: map[string]string{
"tts.provider": "minimax",
"tts.minimax.voice": "Wise_Woman",
"tts.minimax.params": `{"speed":1.5}`,
}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newDualReadMux(sc, cs)
rr := configGET(mux)
var resp ttsConfigResponse
json.Unmarshal(rr.Body.Bytes(), &resp)
if resp.MiniMax.Voice != "Wise_Woman" {
t.Errorf("minimax.voice: got %q, want Wise_Woman", resp.MiniMax.Voice)
}
if resp.MiniMax.Params == nil {
t.Fatal("minimax.params: expected non-nil")
}
if resp.MiniMax.Params["speed"] != 1.5 {
t.Errorf("speed: got %v, want 1.5", resp.MiniMax.Params["speed"])
}
}
// TestTTSConfig_DualWrite_SavesBoth verifies that POST saves both legacy flat keys AND blob.
func TestTTSConfig_DualWrite_SavesBoth(t *testing.T) {
sc := &validationSystemConfigStore{data: map[string]string{}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newDualReadMux(sc, cs)
body := map[string]any{
"provider": "openai",
"openai": map[string]any{
"voice": "nova",
"model": "gpt-4o-mini-tts",
"api_key": "sk-test",
"params": map[string]any{"speed": 1.2, "response_format": "opus"},
},
}
rr := configPOST(mux, body, t)
if rr.Code != http.StatusOK {
t.Fatalf("POST config: want 200, got %d: %s", rr.Code, rr.Body.String())
}
// Legacy flat key persisted
if sc.data["tts.openai.voice"] != "nova" {
t.Errorf("legacy voice key: got %q, want nova", sc.data["tts.openai.voice"])
}
// Blob persisted
blobRaw := sc.data["tts.openai.params"]
if blobRaw == "" {
t.Fatal("tts.openai.params blob: expected non-empty")
}
var blob map[string]any
if err := json.Unmarshal([]byte(blobRaw), &blob); err != nil {
t.Fatalf("unmarshal blob: %v", err)
}
if blob["speed"] != 1.2 {
t.Errorf("blob.speed: got %v, want 1.2", blob["speed"])
}
if blob["response_format"] != "opus" {
t.Errorf("blob.response_format: got %v, want opus", blob["response_format"])
}
}
// TestTTSConfig_DualWrite_DisjointUnion verifies that blob and flat keys cover disjoint fields.
// Flat keys carry voice/model; blob carries nested provider params.
func TestTTSConfig_DualWrite_DisjointUnion(t *testing.T) {
sc := &validationSystemConfigStore{data: map[string]string{}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newDualReadMux(sc, cs)
body := map[string]any{
"provider": "elevenlabs",
"elevenlabs": map[string]any{
"voice_id": "Rachel",
"model_id": "eleven_multilingual_v2",
"api_key": "xi-key",
"params": map[string]any{
"voice_settings": map[string]any{"stability": 0.6},
"seed": 42,
},
},
}
rr := configPOST(mux, body, t)
if rr.Code != http.StatusOK {
t.Fatalf("POST config: want 200, got %d: %s", rr.Code, rr.Body.String())
}
// Flat key for voice
if sc.data["tts.elevenlabs.voice"] != "Rachel" {
t.Errorf("flat voice: got %q, want Rachel", sc.data["tts.elevenlabs.voice"])
}
// Blob contains voice_settings
blobRaw := sc.data["tts.elevenlabs.params"]
var blob map[string]any
json.Unmarshal([]byte(blobRaw), &blob)
vs, _ := blob["voice_settings"].(map[string]any)
if vs == nil || vs["stability"] != 0.6 {
t.Errorf("blob voice_settings.stability: got %v, want 0.6", vs)
}
if blob["seed"] != float64(42) {
t.Errorf("blob.seed: got %v, want 42", blob["seed"])
}
}
+63
View File
@@ -211,6 +211,69 @@ func TestTTSConfigGet_RoundTripsCompatibilityFields(t *testing.T) {
}
}
// TestSaveAndLoad_Gemini verifies that Gemini config round-trips through save/load.
func TestSaveAndLoad_Gemini(t *testing.T) {
setupTestToken(t, "")
sc := &validationSystemConfigStore{data: map[string]string{}}
cs := &validationSecretsStore{data: map[string]string{}}
mux := newValidationTTSConfigMux(sc, cs)
// Save
req := httptest.NewRequest("POST", "/v1/tts/config", ttsBody(t, map[string]any{
"provider": "gemini",
"gemini": map[string]string{
"api_key": "gm-test-key",
"voice": "Kore",
"model": "gemini-2.5-flash-preview-tts",
"speakers": `[{"speaker":"Joe","voice_id":"Kore"}]`,
},
}))
req.Header.Set("Content-Type", "application/json")
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("save: want 200, got %d: %s", rr.Code, rr.Body.String())
}
// Verify system_configs
if got := sc.data["tts.gemini.voice"]; got != "Kore" {
t.Errorf("tts.gemini.voice = %q, want Kore", got)
}
if got := sc.data["tts.gemini.model"]; got != "gemini-2.5-flash-preview-tts" {
t.Errorf("tts.gemini.model = %q, want gemini-2.5-flash-preview-tts", got)
}
if got := sc.data["tts.gemini.speakers"]; got == "" {
t.Error("tts.gemini.speakers not persisted")
}
// Verify secret stored
if got := cs.data["tts.gemini.api_key"]; got != "gm-test-key" {
t.Errorf("tts.gemini.api_key = %q, want gm-test-key", got)
}
// Load (GET)
getReq := httptest.NewRequest("GET", "/v1/tts/config", nil)
getRR := httptest.NewRecorder()
mux.ServeHTTP(getRR, getReq)
if getRR.Code != http.StatusOK {
t.Fatalf("get: want 200, got %d: %s", getRR.Code, getRR.Body.String())
}
var body map[string]any
if err := json.Unmarshal(getRR.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response: %v", err)
}
geminiResp, _ := body["gemini"].(map[string]any)
if geminiResp == nil {
t.Fatal("gemini block missing in response")
}
if got, _ := geminiResp["api_key"].(string); got != "***" {
t.Errorf("gemini api_key = %q, want masked '***'", got)
}
if got, _ := geminiResp["voice"].(string); got != "Kore" {
t.Errorf("gemini voice = %q, want Kore", got)
}
}
func TestTTSConfigSave_WriteFailuresReturnError(t *testing.T) {
setupTestToken(t, "")
+137
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
@@ -11,7 +12,9 @@ import (
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/audio"
geminiPkg "github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
"github.com/nextlevelbuilder/goclaw/internal/crypto"
"github.com/nextlevelbuilder/goclaw/internal/i18n"
"github.com/nextlevelbuilder/goclaw/internal/store"
)
@@ -295,6 +298,77 @@ func TestSynthesize_EdgeHonorsOpts(t *testing.T) {
}
}
// TestResolveTenantProvider_Gemini verifies that a tenant configured for Gemini
// produces a non-nil provider with no error.
func TestResolveTenantProvider_Gemini(t *testing.T) {
setupTestToken(t, "")
sc := &validationSystemConfigStore{data: map[string]string{
"tts.provider": "gemini",
"tts.gemini.voice": "Kore",
"tts.gemini.model": "gemini-3.1-flash-tts-preview",
}}
cs := &validationSecretsStore{data: map[string]string{
"tts.gemini.api_key": "test-gemini-key",
}}
mgr := audio.NewManager(audio.ManagerConfig{})
h := NewTTSHandler(mgr)
h.systemConfigs = sc
h.configSecrets = cs
provider, name, _, err := h.resolveTenantProvider(context.Background(), "")
if err != nil {
t.Fatalf("resolveTenantProvider error: %v", err)
}
if provider == nil {
t.Fatal("expected non-nil provider")
}
if name != "gemini" {
t.Errorf("name = %q, want gemini", name)
}
}
// TestLoadSavedSpeakers_Gemini verifies persisted multi-speaker config reactivates
// at synthesize time (M1 fix) so saved Gemini setups don't silently fall through
// to single-voice mode.
func TestLoadSavedSpeakers_Gemini(t *testing.T) {
ctx := context.Background()
sc := &validationSystemConfigStore{data: map[string]string{
"tts.gemini.speakers": `[{"speaker":"Joe","voiceId":"Kore"},{"speaker":"Jane","voiceId":"Puck"}]`,
}}
speakers := loadSavedSpeakers(ctx, sc, "gemini")
if len(speakers) != 2 {
t.Fatalf("speakers len = %d, want 2", len(speakers))
}
if speakers[0].Speaker != "Joe" || speakers[0].VoiceID != "Kore" {
t.Errorf("speakers[0] = %+v, want {Joe Kore}", speakers[0])
}
if speakers[1].Speaker != "Jane" || speakers[1].VoiceID != "Puck" {
t.Errorf("speakers[1] = %+v, want {Jane Puck}", speakers[1])
}
// Nil store → nil result.
if got := loadSavedSpeakers(ctx, nil, "gemini"); got != nil {
t.Errorf("nil store: got %v, want nil", got)
}
// Non-Gemini provider → nil (other providers don't persist multi-speaker).
if got := loadSavedSpeakers(ctx, sc, "openai"); got != nil {
t.Errorf("openai: got %v, want nil", got)
}
// Empty blob → nil.
empty := &validationSystemConfigStore{data: map[string]string{}}
if got := loadSavedSpeakers(ctx, empty, "gemini"); got != nil {
t.Errorf("empty: got %v, want nil", got)
}
// Malformed JSON → nil (no panic, no error propagation).
bad := &validationSystemConfigStore{data: map[string]string{"tts.gemini.speakers": "not json"}}
if got := loadSavedSpeakers(ctx, bad, "gemini"); got != nil {
t.Errorf("bad json: got %v, want nil", got)
}
}
// TestSynthesize_ValidElevenLabsModel verifies that a known ElevenLabs model passes through.
func TestSynthesize_ValidElevenLabsModel(t *testing.T) {
setupTestToken(t, "") // dev mode
@@ -317,3 +391,66 @@ func TestSynthesize_ValidElevenLabsModel(t *testing.T) {
t.Errorf("want 200, got %d: %s", rr.Code, rr.Body.String())
}
}
// TestSynthesize_GeminiInvalidVoice_I18n verifies that 422 responses for
// Gemini ErrInvalidVoice use i18n.T(locale, ...) — not err.Error() — so
// VI and ZH callers receive translated messages (M2-b carry-over).
func TestSynthesize_GeminiInvalidVoice_I18n(t *testing.T) {
setupTestToken(t, "") // dev mode — no bearer token required
for _, tc := range []struct {
locale string
wantBody string
}{
{
locale: "en",
wantBody: i18n.T("en", i18n.MsgTtsGeminiInvalidVoice, "bad-voice"),
},
{
locale: "vi",
wantBody: i18n.T("vi", i18n.MsgTtsGeminiInvalidVoice, "bad-voice"),
},
{
locale: "zh",
wantBody: i18n.T("zh", i18n.MsgTtsGeminiInvalidVoice, "bad-voice"),
},
} {
t.Run("locale="+tc.locale, func(t *testing.T) {
// Register a mock "gemini" provider that always returns ErrInvalidVoice.
mock := &mockTTSProvider{
name: "gemini",
stateless: true,
err: fmt.Errorf("wrap: %w", geminiPkg.ErrInvalidVoice),
}
mgr := audio.NewManager(audio.ManagerConfig{Primary: "gemini"})
mgr.RegisterProvider(mock)
mux := newTTSMux(mgr)
body := ttsBody(t, map[string]any{
"text": "hello",
"provider": "gemini",
"voice_id": "bad-voice",
})
req := httptest.NewRequest("POST", "/v1/tts/synthesize", body)
req.Header.Set("Content-Type", "application/json")
// Locale is extracted from Accept-Language by requireAuth → enrichContext → extractLocale.
// Setting it directly on the context is overwritten; set the header instead.
req.Header.Set("Accept-Language", tc.locale)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusUnprocessableEntity {
t.Fatalf("locale=%s: want 422, got %d: %s", tc.locale, rr.Code, rr.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v", err)
}
got, _ := resp["error"].(string)
if got != tc.wantBody {
t.Errorf("locale=%s: want error %q, got %q", tc.locale, tc.wantBody, got)
}
})
}
}
+71 -11
View File
@@ -14,6 +14,7 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/edge"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
"github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
"github.com/nextlevelbuilder/goclaw/internal/audio/openai"
"github.com/nextlevelbuilder/goclaw/internal/i18n"
@@ -22,15 +23,16 @@ import (
// testConnectionRequest is the JSON body for POST /v1/tts/test-connection.
type testConnectionRequest struct {
Provider string `json:"provider"`
APIKey string `json:"api_key,omitempty"`
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"` // MiniMax requires group_id
Rate string `json:"rate,omitempty"`
TimeoutMs int `json:"timeout_ms,omitempty"`
Provider string `json:"provider"`
APIKey string `json:"api_key,omitempty"`
APIBase string `json:"api_base,omitempty"`
BaseURL string `json:"base_url,omitempty"`
VoiceID string `json:"voice_id,omitempty"`
ModelID string `json:"model_id,omitempty"`
GroupID string `json:"group_id,omitempty"` // MiniMax requires group_id
Rate string `json:"rate,omitempty"`
TimeoutMs int `json:"timeout_ms,omitempty"`
Params map[string]any `json:"params,omitempty"` // provider-specific params blob
}
// testConnectionResponse is the JSON response for POST /v1/tts/test-connection.
@@ -47,6 +49,7 @@ var supportedTestProviders = map[string]bool{
"elevenlabs": true,
"edge": true,
"minimax": true,
"gemini": true,
}
// providersRequiringAPIKey lists providers that need an API key.
@@ -54,6 +57,7 @@ var providersRequiringAPIKey = map[string]bool{
"openai": true,
"elevenlabs": true,
"minimax": true,
"gemini": true,
}
const testConnectionTimeout = 10 * time.Second
@@ -96,6 +100,11 @@ func (h *TTSHandler) handleTestConnection(w http.ResponseWriter, r *http.Request
return
}
// Fall back to saved secrets when the request is testing a previously saved key.
// Frontend masks stored API keys as "***" — without this fallback, retesting
// an existing config would force the user to retype the key.
h.fillMissingTestSecrets(ctx, &req)
if providersRequiringAPIKey[req.Provider] && (req.APIKey == "" || req.APIKey == "***") {
http.Error(w, fmt.Sprintf(`{"error":"api_key is required for %s"}`, req.Provider), http.StatusBadRequest)
return
@@ -123,7 +132,7 @@ func (h *TTSHandler) handleTestConnection(w http.ResponseWriter, r *http.Request
defer cancel()
start := time.Now()
_, err = provider.Synthesize(synthCtx, "test", audio.TTSOptions{Voice: req.VoiceID, Model: req.ModelID})
_, err = provider.Synthesize(synthCtx, "test", audio.TTSOptions{Voice: req.VoiceID, Model: req.ModelID, Params: req.Params})
dur := time.Since(start)
if err != nil {
@@ -134,9 +143,33 @@ func (h *TTSHandler) handleTestConnection(w http.ResponseWriter, r *http.Request
})
return
}
// Translate Gemini sentinel errors to 422 Unprocessable Entity with i18n message.
if errors.Is(err, gemini.ErrInvalidVoice) {
slog.Warn("tts.test-connection.invalid-params", "provider", req.Provider, "error", err)
writeJSON(w, http.StatusUnprocessableEntity, testConnectionResponse{
Success: false, Error: i18n.T(locale, i18n.MsgTtsGeminiInvalidVoice, req.VoiceID),
})
return
}
if errors.Is(err, gemini.ErrSpeakerLimit) {
slog.Warn("tts.test-connection.invalid-params", "provider", req.Provider, "error", err)
writeJSON(w, http.StatusUnprocessableEntity, testConnectionResponse{
Success: false, Error: i18n.T(locale, i18n.MsgTtsGeminiSpeakerLimit),
})
return
}
if errors.Is(err, gemini.ErrInvalidModel) {
slog.Warn("tts.test-connection.invalid-params", "provider", req.Provider, "error", err)
writeJSON(w, http.StatusUnprocessableEntity, testConnectionResponse{
Success: false, Error: i18n.T(locale, i18n.MsgTtsGeminiInvalidModel, req.ModelID),
})
return
}
// Surface upstream error to caller — test-connection is a diagnostic
// endpoint, opacity here just makes debugging harder.
slog.Warn("tts.test-connection.failed", "provider", req.Provider, "error", err)
writeJSON(w, http.StatusBadGateway, testConnectionResponse{
Success: false, Error: "upstream synthesis failed",
Success: false, Error: err.Error(),
})
return
}
@@ -184,11 +217,38 @@ func createEphemeralTTSProvider(req testConnectionRequest) (audio.TTSProvider, e
Model: req.ModelID,
TimeoutMs: req.TimeoutMs,
}), nil
case "gemini":
return gemini.NewProvider(gemini.Config{
APIKey: req.APIKey,
APIBase: req.APIBase,
Voice: req.VoiceID,
Model: req.ModelID,
TimeoutMs: req.TimeoutMs,
}), nil
default:
return nil, fmt.Errorf("unsupported provider: %s", req.Provider)
}
}
// fillMissingTestSecrets fills empty/masked secret fields on req with values
// from the tenant's saved config. Lets the frontend re-test a stored API key
// without forcing the user to retype it (the key is masked as "***" in GET).
func (h *TTSHandler) fillMissingTestSecrets(ctx context.Context, req *testConnectionRequest) {
if h.configSecrets == nil {
return
}
if providersRequiringAPIKey[req.Provider] && (req.APIKey == "" || req.APIKey == "***") {
if saved, _ := h.configSecrets.Get(ctx, "tts."+req.Provider+".api_key"); saved != "" {
req.APIKey = saved
}
}
if req.Provider == "minimax" && req.GroupID == "" {
if saved, _ := h.configSecrets.Get(ctx, "tts.minimax.group_id"); saved != "" {
req.GroupID = saved
}
}
}
func loadTenantTTSTimeoutMs(ctx context.Context, sc store.SystemConfigStore) int {
if sc == nil {
return 0
+157
View File
@@ -1,6 +1,7 @@
package http
import (
"encoding/base64"
"encoding/json"
"net/http"
"net/http/httptest"
@@ -8,6 +9,7 @@ import (
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/audio"
geminiPkg "github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
"github.com/nextlevelbuilder/goclaw/internal/crypto"
"github.com/nextlevelbuilder/goclaw/internal/store"
)
@@ -133,6 +135,85 @@ func TestTestConnection_BelowOperator(t *testing.T) {
}
}
// TestCreateEphemeralTTSProvider_Gemini verifies that a Gemini request creates a valid provider.
func TestCreateEphemeralTTSProvider_Gemini(t *testing.T) {
req := testConnectionRequest{
Provider: "gemini",
APIKey: "test-gemini-key",
VoiceID: "Kore",
ModelID: "gemini-3.1-flash-tts-preview",
}
p, err := createEphemeralTTSProvider(req)
if err != nil {
t.Fatalf("createEphemeralTTSProvider error: %v", err)
}
if p == nil {
t.Fatal("expected non-nil provider")
}
if _, ok := p.(*geminiPkg.Provider); !ok {
t.Errorf("expected *gemini.Provider, got %T", p)
}
}
// TestCreateEphemeralTTSProvider_GeminiMissingKey verifies missing API key causes 400.
func TestCreateEphemeralTTSProvider_GeminiMissingKey(t *testing.T) {
setupTestToken(t, "")
mgr := audio.NewManager(audio.ManagerConfig{})
mux := newTTSMux(mgr)
req := httptest.NewRequest("POST", "/v1/tts/test-connection",
ttsBody(t, map[string]string{"provider": "gemini"}))
req.Header.Set("Content-Type", "application/json")
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusBadRequest {
t.Errorf("want 400, got %d: %s", rr.Code, rr.Body.String())
}
}
// TestTestConnection_GeminiHappyPath verifies Gemini ephemeral provider synthesizes successfully.
// Uses createEphemeralTTSProvider directly (bypasses SSRF gate which blocks 127.0.0.1).
func TestTestConnection_GeminiHappyPath(t *testing.T) {
// Spin up a mock Gemini API server.
pcm := make([]byte, 128)
b64 := base64.StdEncoding.EncodeToString(pcm)
geminiResp, _ := json.Marshal(map[string]any{
"candidates": []map[string]any{
{"content": map[string]any{
"parts": []map[string]any{
{"inlineData": map[string]any{"data": b64}},
},
}},
},
})
mockSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
_, _ = w.Write(geminiResp)
}))
defer mockSrv.Close()
req := testConnectionRequest{
Provider: "gemini",
APIKey: "test-key",
APIBase: mockSrv.URL,
VoiceID: "Kore",
ModelID: "gemini-2.5-flash-preview-tts",
}
p, err := createEphemeralTTSProvider(req)
if err != nil {
t.Fatalf("createEphemeralTTSProvider: %v", err)
}
result, err := p.Synthesize(t.Context(), "test", audio.TTSOptions{})
if err != nil {
t.Fatalf("Synthesize: %v", err)
}
if result.MimeType != "audio/wav" {
t.Errorf("MimeType = %q, want audio/wav", result.MimeType)
}
}
// TestTestConnection_BlockedAPIBase verifies test-connection reuses provider URL SSRF validation.
func TestTestConnection_BlockedAPIBase(t *testing.T) {
setupTestToken(t, ttsTestToken)
@@ -170,3 +251,79 @@ func TestTestConnection_BlockedAPIBase(t *testing.T) {
t.Fatalf("unexpected error: %q", errStr)
}
}
// Note: TestTestConnection_GeminiInvalidVoice_I18n (M2-b carry-over) is owned
// by Phase 04 — it lands alongside the i18n.T wiring at the HTTP response site.
// TestFillMissingTestSecrets_RestoresMaskedAPIKey verifies that a request with
// the masked sentinel "***" or empty api_key is filled from saved secrets, so
// the user can retest a previously saved provider without retyping the key.
func TestFillMissingTestSecrets_RestoresMaskedAPIKey(t *testing.T) {
cs := &validationSecretsStore{data: map[string]string{
"tts.openai.api_key": "sk-saved",
"tts.gemini.api_key": "gm-saved",
"tts.minimax.api_key": "mm-saved",
"tts.minimax.group_id": "grp-saved",
"tts.elevenlabs.api_key": "xi-saved",
}}
h := NewTTSHandler(audio.NewManager(audio.ManagerConfig{}))
h.SetStores(&validationSystemConfigStore{data: map[string]string{}}, cs)
cases := []struct {
name string
req testConnectionRequest
wantAPIKey string
wantGroupID string
}{
{"openai masked", testConnectionRequest{Provider: "openai", APIKey: "***"}, "sk-saved", ""},
{"openai empty", testConnectionRequest{Provider: "openai"}, "sk-saved", ""},
{"gemini masked", testConnectionRequest{Provider: "gemini", APIKey: "***"}, "gm-saved", ""},
{"elevenlabs empty", testConnectionRequest{Provider: "elevenlabs"}, "xi-saved", ""},
{"minimax fills key+group", testConnectionRequest{Provider: "minimax", APIKey: "***"}, "mm-saved", "grp-saved"},
{"non-empty key not overwritten", testConnectionRequest{Provider: "openai", APIKey: "sk-typed"}, "sk-typed", ""},
{"edge no-op", testConnectionRequest{Provider: "edge"}, "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := tc.req
h.fillMissingTestSecrets(t.Context(), &req)
if req.APIKey != tc.wantAPIKey {
t.Errorf("APIKey = %q, want %q", req.APIKey, tc.wantAPIKey)
}
if req.GroupID != tc.wantGroupID {
t.Errorf("GroupID = %q, want %q", req.GroupID, tc.wantGroupID)
}
})
}
}
// TestTestConnection_MaskedAPIKey_FallsBackToSaved verifies the handler does
// not return "api_key is required" when the user submits "***" for a provider
// that already has a saved key.
func TestTestConnection_MaskedAPIKey_FallsBackToSaved(t *testing.T) {
setupTestToken(t, "") // dev mode
h := NewTTSHandler(audio.NewManager(audio.ManagerConfig{}))
h.SetStores(
&validationSystemConfigStore{data: map[string]string{}},
&validationSecretsStore{data: map[string]string{"tts.openai.api_key": "sk-saved"}},
)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
req := httptest.NewRequest("POST", "/v1/tts/test-connection",
ttsBody(t, map[string]string{"provider": "openai", "api_key": "***"}))
req.Header.Set("Content-Type", "application/json")
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
// Whatever the upstream call returns (502 because api.openai.com unreachable
// in tests), it must NOT be the 400 "api_key is required" preflight rejection.
if rr.Code == http.StatusBadRequest {
var resp map[string]any
_ = json.Unmarshal(rr.Body.Bytes(), &resp)
if errStr, _ := resp["error"].(string); errStr == "api_key is required for openai" {
t.Fatalf("masked api_key should fall back to saved secret, got 400: %q", errStr)
}
}
}
+39 -18
View File
@@ -9,30 +9,31 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/elevenlabs"
"github.com/nextlevelbuilder/goclaw/internal/audio/minimax"
"github.com/nextlevelbuilder/goclaw/internal/i18n"
"github.com/nextlevelbuilder/goclaw/internal/permissions"
"github.com/nextlevelbuilder/goclaw/internal/store"
)
// VoicesHandler serves GET /v1/voices and POST /v1/voices/refresh.
// It holds a *VoiceCache to serve cached responses and optionally a
// *elevenlabs.TTSProvider to fetch live voices on cache miss.
// provider is typed as audio.VoiceListProvider so any provider can be injected.
type VoicesHandler struct {
cache *audio.VoiceCache
provider *elevenlabs.TTSProvider // nil when no ElevenLabs key is configured
provider audio.VoiceListProvider // nil when resolved at request time from stores
secretStore store.ConfigSecretsStore
tenantStore store.TenantStore
}
// NewVoicesHandler creates a handler that resolves the ElevenLabs provider at
// request time from config_secrets. Use NewVoicesHandlerWithProvider for tests.
// NewVoicesHandler creates a handler that resolves the provider at request time
// from config_secrets. Use NewVoicesHandlerWithProvider for tests.
func NewVoicesHandler(cache *audio.VoiceCache, secretStore store.ConfigSecretsStore, tenantStore store.TenantStore) *VoicesHandler {
return &VoicesHandler{cache: cache, secretStore: secretStore, tenantStore: tenantStore}
}
// NewVoicesHandlerWithProvider creates a handler with a pre-built provider.
// Primarily used in tests to inject a mock/httptest provider.
func NewVoicesHandlerWithProvider(cache *audio.VoiceCache, p *elevenlabs.TTSProvider) *VoicesHandler {
// Accepts audio.VoiceListProvider so any provider (ElevenLabs, MiniMax, mock) can be injected.
// Primarily used in tests.
func NewVoicesHandlerWithProvider(cache *audio.VoiceCache, p audio.VoiceListProvider) *VoicesHandler {
return &VoicesHandler{cache: cache, provider: p}
}
@@ -55,7 +56,7 @@ func (h *VoicesHandler) handleList(w http.ResponseWriter, r *http.Request) {
p, err := h.resolveProvider(r, tenantID)
if err != nil {
slog.Warn("voices: no ElevenLabs provider", "tenant_id", tenantID, "error", err)
slog.Warn("voices: no provider configured", "tenant_id", tenantID, "error", err)
writeJSON(w, http.StatusNotFound, map[string]string{
"error": i18n.T(locale, i18n.MsgVoicesListFailed, err.Error()),
})
@@ -66,7 +67,7 @@ func (h *VoicesHandler) handleList(w http.ResponseWriter, r *http.Request) {
if err != nil {
slog.Warn("voices: list failed", "tenant_id", tenantID, "error", err)
writeJSON(w, http.StatusBadGateway, map[string]string{
"error": i18n.T(locale, i18n.MsgVoicesListFailed, err.Error()),
"error": i18n.T(locale, i18n.MsgTtsMiniMaxVoicesFailed, err.Error()),
})
return
}
@@ -86,7 +87,7 @@ func (h *VoicesHandler) handleRefresh(w http.ResponseWriter, r *http.Request) {
p, err := h.resolveProvider(r, tenantID)
if err != nil {
slog.Warn("voices: no ElevenLabs provider on refresh", "tenant_id", tenantID, "error", err)
slog.Warn("voices: no provider on refresh", "tenant_id", tenantID, "error", err)
writeJSON(w, http.StatusNotFound, map[string]string{
"error": i18n.T(locale, i18n.MsgVoicesListFailed, err.Error()),
})
@@ -106,18 +107,38 @@ func (h *VoicesHandler) handleRefresh(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{"voices": voices})
}
// resolveProvider returns the ElevenLabs provider to use for this request.
// Priority: injected provider (test/pre-built) > secret store lookup.
func (h *VoicesHandler) resolveProvider(r *http.Request, tenantID uuid.UUID) (*elevenlabs.TTSProvider, error) {
// resolveProvider returns the VoiceListProvider for this request.
// Priority: injected provider (test/pre-built) > query-param ?provider > secret store lookup (elevenlabs default).
func (h *VoicesHandler) resolveProvider(r *http.Request, tenantID uuid.UUID) (audio.VoiceListProvider, error) {
if h.provider != nil {
return h.provider, nil
}
if h.secretStore == nil {
return nil, fmt.Errorf("no ElevenLabs API key configured")
return nil, fmt.Errorf("no voice provider configured")
}
apiKey, err := h.secretStore.Get(r.Context(), "tts.elevenlabs.api_key")
if err != nil || apiKey == "" {
return nil, fmt.Errorf("ElevenLabs API key not found for tenant %s", tenantID)
providerName := r.URL.Query().Get("provider")
if providerName == "" {
providerName = "elevenlabs" // backward-compatible default
}
switch providerName {
case "minimax":
apiKey, err := h.secretStore.Get(r.Context(), "tts.minimax.api_key")
if err != nil || apiKey == "" {
return nil, fmt.Errorf("MiniMax API key not found for tenant %s", tenantID)
}
apiBase, _ := h.secretStore.Get(r.Context(), "tts.minimax.api_base")
return minimax.NewVoiceLister(apiKey, apiBase, 15000, tenantID), nil
case "elevenlabs":
apiKey, err := h.secretStore.Get(r.Context(), "tts.elevenlabs.api_key")
if err != nil || apiKey == "" {
return nil, fmt.Errorf("ElevenLabs API key not found for tenant %s", tenantID)
}
return elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: apiKey}), nil
default:
return nil, fmt.Errorf("unsupported voice provider: %s", providerName)
}
return elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: apiKey}), nil
}
+141
View File
@@ -1,7 +1,9 @@
package http_test
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
@@ -13,6 +15,16 @@ import (
"github.com/nextlevelbuilder/goclaw/internal/store"
)
// mockVoiceListProvider is a simple test double for audio.VoiceListProvider.
type mockVoiceListProvider struct {
voices []audio.Voice
err error
}
func (m *mockVoiceListProvider) ListVoices(_ context.Context) ([]audio.Voice, error) {
return m.voices, m.err
}
const voicesTestToken = "voices-test-token"
func TestVoicesHandler_Unauthenticated(t *testing.T) {
@@ -167,3 +179,132 @@ func TestVoicesHandler_RefreshAdmin(t *testing.T) {
t.Errorf("unexpected voices after refresh: %+v", resp.Voices)
}
}
// TestVoicesHandler_InterfaceRefactor_Elevenlabs verifies that NewVoicesHandlerWithProvider
// accepts an audio.VoiceListProvider (not a concrete *elevenlabs.TTSProvider).
func TestVoicesHandler_InterfaceRefactor_Elevenlabs(t *testing.T) {
httpapi.InitGatewayToken("")
t.Cleanup(func() { httpapi.InitGatewayToken("") })
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"voices":[{"voice_id":"el-v1","name":"Aria","category":"premade"}]}`))
}))
defer upstream.Close()
// Inject as audio.VoiceListProvider interface — confirms the refactor works.
var p audio.VoiceListProvider = elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: "k", BaseURL: upstream.URL})
cache := audio.NewVoiceCache(time.Hour, 100)
h := httpapi.NewVoicesHandlerWithProvider(cache, p)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
req := httptest.NewRequest("GET", "/v1/voices", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", rr.Code, rr.Body.String())
}
var resp struct {
Voices []audio.Voice `json:"voices"`
}
json.NewDecoder(rr.Body).Decode(&resp)
if len(resp.Voices) != 1 || resp.Voices[0].ID != "el-v1" {
t.Errorf("unexpected voices: %+v", resp.Voices)
}
}
// TestVoicesHandler_MiniMaxBranch verifies a mock VoiceListProvider returning
// Category-grouped entries is served correctly.
func TestVoicesHandler_MiniMaxBranch(t *testing.T) {
httpapi.InitGatewayToken("")
t.Cleanup(func() { httpapi.InitGatewayToken("") })
mockVoices := []audio.Voice{
{ID: "S1", Name: "System Voice", Category: "system"},
{ID: "C1", Name: "Cloned Voice", Category: "cloning"},
}
mock := &mockVoiceListProvider{voices: mockVoices}
cache := audio.NewVoiceCache(time.Hour, 100)
h := httpapi.NewVoicesHandlerWithProvider(cache, mock)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
req := httptest.NewRequest("GET", "/v1/voices", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", rr.Code, rr.Body.String())
}
var resp struct {
Voices []audio.Voice `json:"voices"`
}
json.NewDecoder(rr.Body).Decode(&resp)
if len(resp.Voices) != 2 {
t.Fatalf("expected 2 voices, got %d", len(resp.Voices))
}
if resp.Voices[0].Category != "system" || resp.Voices[1].Category != "cloning" {
t.Errorf("unexpected categories: %+v", resp.Voices)
}
}
// TestVoicesHandler_MiniMaxFirstFetchFailure_502 verifies that when the provider
// returns an error and no cache exists, the handler returns 502 (not 500).
func TestVoicesHandler_MiniMaxFirstFetchFailure_502(t *testing.T) {
httpapi.InitGatewayToken("")
t.Cleanup(func() { httpapi.InitGatewayToken("") })
mock := &mockVoiceListProvider{
voices: []audio.Voice{},
err: fmt.Errorf("upstream 500: internal server error"),
}
cache := audio.NewVoiceCache(time.Hour, 100)
h := httpapi.NewVoicesHandlerWithProvider(cache, mock)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
req := httptest.NewRequest("GET", "/v1/voices", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusBadGateway {
t.Errorf("expected 502 on provider error, got %d: %s", rr.Code, rr.Body.String())
}
// Must not cascade to 500.
if rr.Code == http.StatusInternalServerError {
t.Error("must not return 500 — should be 502")
}
}
// TestVoicesHandler_BackwardCompat_ElevenlabsUnchanged verifies that the existing
// elevenlabs-based constructor still compiles and works (no breaking change).
func TestVoicesHandler_BackwardCompat_ElevenlabsUnchanged(t *testing.T) {
httpapi.InitGatewayToken("")
t.Cleanup(func() { httpapi.InitGatewayToken("") })
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"voices":[{"voice_id":"bc-v1","name":"BackCompat"}]}`))
}))
defer upstream.Close()
p := elevenlabs.NewTTSProvider(elevenlabs.Config{APIKey: "k", BaseURL: upstream.URL})
cache := audio.NewVoiceCache(time.Hour, 100)
// NewVoicesHandlerWithProvider now accepts audio.VoiceListProvider —
// ElevenLabs TTSProvider implements it, so this must compile.
h := httpapi.NewVoicesHandlerWithProvider(cache, p)
mux := http.NewServeMux()
h.RegisterRoutes(mux)
req := httptest.NewRequest("GET", "/v1/voices", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", rr.Code)
}
}
+8 -2
View File
@@ -199,8 +199,14 @@ func init() {
MsgTenantScopeRequired: "tenant scope is required for this operation",
// TTS / Voices
MsgTtsUnknownModel: "unknown tts model: %s",
MsgVoicesListFailed: "failed to list voices: %s",
MsgTtsUnknownModel: "unknown tts model: %s",
MsgVoicesListFailed: "failed to list voices: %s",
MsgTtsGeminiInvalidVoice: "invalid Gemini voice: %s",
MsgTtsGeminiSpeakerLimit: "Gemini TTS supports at most 2 speakers",
MsgTtsGeminiInvalidModel: "invalid Gemini TTS model: %s",
MsgTtsParamOutOfRange: "TTS param %q value %v is out of range [%v, %v]",
MsgTtsParamDependsOn: "TTS param %q requires %q to be %q",
MsgTtsMiniMaxVoicesFailed: "failed to fetch MiniMax voices: %s",
// STT
MsgSTTAllProvidersFailed: "All STT providers failed",
+8 -2
View File
@@ -199,8 +199,14 @@ func init() {
MsgTenantScopeRequired: "cần xác định tenant để thực hiện thao tác này",
// TTS / Giọng đọc
MsgTtsUnknownModel: "model tts không hỗ trợ: %s",
MsgVoicesListFailed: "không tải được danh sách giọng đọc: %s",
MsgTtsUnknownModel: "model tts không hỗ trợ: %s",
MsgVoicesListFailed: "không tải được danh sách giọng đọc: %s",
MsgTtsGeminiInvalidVoice: "giọng đọc Gemini không hợp lệ: %s",
MsgTtsGeminiSpeakerLimit: "Gemini TTS hỗ trợ tối đa 2 người nói",
MsgTtsGeminiInvalidModel: "mô hình Gemini TTS không hợp lệ: %s",
MsgTtsParamOutOfRange: "tham số TTS %q có giá trị %v nằm ngoài phạm vi [%v, %v]",
MsgTtsParamDependsOn: "tham số TTS %q yêu cầu %q phải là %q",
MsgTtsMiniMaxVoicesFailed: "không tải được danh sách giọng đọc MiniMax: %s",
// STT
MsgSTTAllProvidersFailed: "Tất cả nhà cung cấp STT đều thất bại",
+8 -2
View File
@@ -199,8 +199,14 @@ func init() {
MsgTenantScopeRequired: "此操作需要指定租户范围",
// TTS / 声音
MsgTtsUnknownModel: "未知的 tts 模型:%s",
MsgVoicesListFailed: "获取声音列表失败:%s",
MsgTtsUnknownModel: "未知的 tts 模型:%s",
MsgVoicesListFailed: "获取声音列表失败:%s",
MsgTtsGeminiInvalidVoice: "无效的 Gemini 声音:%s",
MsgTtsGeminiSpeakerLimit: "Gemini TTS 最多支持 2 位发言人",
MsgTtsGeminiInvalidModel: "无效的 Gemini TTS 模型:%s",
MsgTtsParamOutOfRange: "TTS 参数 %q 的值 %v 超出范围 [%v, %v]",
MsgTtsParamDependsOn: "TTS 参数 %q 需要将 %q 设置为 %q",
MsgTtsMiniMaxVoicesFailed: "获取 MiniMax 声音列表失败:%s",
// STT
MsgSTTAllProvidersFailed: "所有 STT 提供商均失败",
+8 -2
View File
@@ -197,8 +197,14 @@ const (
MsgInvalidRole = "error.invalid_role" // "invalid role: allowed values are owner, admin, operator, member, viewer"
// --- TTS / Voices ---
MsgTtsUnknownModel = "error.tts_unknown_model" // "unknown tts model: %s"
MsgVoicesListFailed = "error.voices_list_failed" // "failed to list voices: %s"
MsgTtsUnknownModel = "error.tts_unknown_model" // "unknown tts model: %s"
MsgVoicesListFailed = "error.voices_list_failed" // "failed to list voices: %s"
MsgTtsGeminiInvalidVoice = "error.tts_gemini_invalid_voice" // "invalid Gemini voice: %s"
MsgTtsGeminiSpeakerLimit = "error.tts_gemini_speaker_limit" // "Gemini TTS supports at most 2 speakers"
MsgTtsGeminiInvalidModel = "error.tts_gemini_invalid_model" // "invalid Gemini TTS model: %s"
MsgTtsParamOutOfRange = "error.tts_param_out_of_range" // "TTS param %q value %v is out of range [%v, %v]"
MsgTtsParamDependsOn = "error.tts_param_depends_on" // "TTS param %q requires %q to be %q"
MsgTtsMiniMaxVoicesFailed = "error.tts_minimax_voices_failed" // "failed to fetch MiniMax voices: %s"
// --- STT ---
MsgSTTAllProvidersFailed = "error.stt_all_providers_failed" // "All STT providers failed"
+116
View File
@@ -0,0 +1,116 @@
//go:build integration
package integration
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
httphandlers "github.com/nextlevelbuilder/goclaw/internal/http"
)
// TestTTSCapabilities_ListProviders verifies that the audio manager correctly
// lists capabilities for all registered TTS providers.
func TestTTSCapabilities_ListProviders(t *testing.T) {
// Create manager with mock providers
mgr := audio.NewManager(audio.ManagerConfig{Primary: "openai"})
registerTestProviders(mgr)
// List capabilities from manager
caps := mgr.ListCapabilities()
if len(caps) < 4 {
t.Errorf("expected at least 4 providers, got %d", len(caps))
}
// Find OpenAI provider
var openaiCap *audio.ProviderCapabilities
for i := range caps {
if caps[i].Provider == "openai" {
openaiCap = &caps[i]
break
}
}
if openaiCap == nil {
t.Fatal("OpenAI provider not found in capabilities")
}
if openaiCap.DisplayName == "" {
t.Error("OpenAI capability missing DisplayName")
}
}
// TestTTSCapabilities_Handler_GetRoute verifies that GET /v1/tts/capabilities
// endpoint is properly registered on the TTS handler.
func TestTTSCapabilities_Handler_GetRoute(t *testing.T) {
mgr := audio.NewManager(audio.ManagerConfig{Primary: "openai"})
registerTestProviders(mgr)
handler := httphandlers.NewTTSHandler(mgr)
mux := http.NewServeMux()
handler.RegisterRoutes(mux)
// GET /v1/tts/capabilities (without auth, should fail on auth check)
req := httptest.NewRequest("GET", "/v1/tts/capabilities", nil)
rr := httptest.NewRecorder()
mux.ServeHTTP(rr, req)
// Should get 401 (unauthorized) since no Bearer token provided
// Note: The auth middleware requires valid token
if rr.Code != http.StatusUnauthorized && rr.Code != http.StatusBadRequest {
t.Logf("GET without auth returned %d (expected 401 or 400)", rr.Code)
}
}
// TestTTSCapabilities_ManagerRegistration verifies that providers can be
// registered and listed through the manager.
func TestTTSCapabilities_ManagerRegistration(t *testing.T) {
mgr := audio.NewManager(audio.ManagerConfig{Primary: "openai"})
// Register providers
mgr.RegisterTTS(&capTestProvider{name: "openai"})
mgr.RegisterTTS(&capTestProvider{name: "elevenlabs"})
// Check count
caps := mgr.ListCapabilities()
if len(caps) < 2 {
t.Errorf("expected at least 2 providers after registration, got %d", len(caps))
}
// Verify names present
names := make(map[string]bool)
for _, cap := range caps {
names[cap.Provider] = true
}
if !names["openai"] {
t.Error("openai not in capabilities after registration")
}
if !names["elevenlabs"] {
t.Error("elevenlabs not in capabilities after registration")
}
}
// registerTestProviders registers the standard test providers with the manager.
func registerTestProviders(mgr *audio.Manager) {
// Register mock providers for testing
mgr.RegisterTTS(&capTestProvider{name: "openai"})
mgr.RegisterTTS(&capTestProvider{name: "elevenlabs"})
mgr.RegisterTTS(&capTestProvider{name: "edge"})
mgr.RegisterTTS(&capTestProvider{name: "minimax"})
}
// capTestProvider is a test implementation of audio.TTSProvider.
type capTestProvider struct {
name string
}
func (p *capTestProvider) Name() string { return p.name }
func (p *capTestProvider) Synthesize(context.Context, string, audio.TTSOptions) (*audio.SynthResult, error) {
return nil, nil
}
@@ -0,0 +1,198 @@
//go:build sqliteonly && integration
package integration
import (
"context"
"database/sql"
"testing"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/sqlitestore"
_ "modernc.org/sqlite"
)
// testSQLiteDB creates an in-memory SQLite database for testing.
func testSQLiteDB(t *testing.T) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
t.Cleanup(func() { db.Close() })
// Initialize SQLite schema
if err := sqlitestore.EnsureSchema(db); err != nil {
t.Fatalf("init schema: %v", err)
}
return db
}
// TestTTSConfig_Store_RoundTrip_LegacyKeys_SQLite verifies that legacy flat keys
// stored in system_configs can be read back (SQLite backend).
func TestTTSConfig_Store_RoundTrip_LegacyKeys_SQLite(t *testing.T) {
db := testSQLiteDB(t)
tenantID := uuid.New()
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := sqlitestore.NewSQLiteSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set legacy flat keys
if err := configStore.Set(ctx, "tts.provider", "openai"); err != nil {
t.Fatalf("Set provider: %v", err)
}
if err := configStore.Set(ctx, "tts.openai.voice", "alloy"); err != nil {
t.Fatalf("Set voice: %v", err)
}
// Read them back
provider, err := configStore.Get(ctx, "tts.provider")
if err != nil {
t.Fatalf("Get provider: %v", err)
}
if provider != "openai" {
t.Errorf("provider: got %q, want openai", provider)
}
voice, err := configStore.Get(ctx, "tts.openai.voice")
if err != nil {
t.Fatalf("Get voice: %v", err)
}
if voice != "alloy" {
t.Errorf("voice: got %q, want alloy", voice)
}
}
// TestTTSConfig_Store_RoundTrip_ParamsBlob_SQLite verifies that params blob
// stored in system_configs can be read back (SQLite backend).
func TestTTSConfig_Store_RoundTrip_ParamsBlob_SQLite(t *testing.T) {
db := testSQLiteDB(t)
tenantID := uuid.New()
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := sqlitestore.NewSQLiteSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set provider and params blob
configStore.Set(ctx, "tts.provider", "elevenlabs")
paramsBlob := `{"voice_settings":{"stability":0.8}}`
configStore.Set(ctx, "tts.elevenlabs.params", paramsBlob)
// Read them back
provider, _ := configStore.Get(ctx, "tts.provider")
if provider != "elevenlabs" {
t.Errorf("provider: got %q, want elevenlabs", provider)
}
retrieved, _ := configStore.Get(ctx, "tts.elevenlabs.params")
if retrieved != paramsBlob {
t.Errorf("params blob: got %q, want %q", retrieved, paramsBlob)
}
}
// TestTTSConfig_Store_Dual_LegacyAndBlob_SQLite verifies that both legacy keys
// and params blob can coexist (SQLite backend).
func TestTTSConfig_Store_Dual_LegacyAndBlob_SQLite(t *testing.T) {
db := testSQLiteDB(t)
tenantID := uuid.New()
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := sqlitestore.NewSQLiteSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set both legacy and blob
configStore.Set(ctx, "tts.provider", "minimax")
configStore.Set(ctx, "tts.minimax.voice", "Wise_Woman")
configStore.Set(ctx, "tts.minimax.params", `{"speed":1.5}`)
// Retrieve both
voice, _ := configStore.Get(ctx, "tts.minimax.voice")
if voice != "Wise_Woman" {
t.Errorf("voice: got %q, want Wise_Woman", voice)
}
params, _ := configStore.Get(ctx, "tts.minimax.params")
if params != `{"speed":1.5}` {
t.Errorf("params: got %q, want {\"speed\":1.5}", params)
}
}
// TestTTSConfig_Store_DisjointUnion_SQLite verifies disjoint storage and retrieval
// (SQLite backend).
func TestTTSConfig_Store_DisjointUnion_SQLite(t *testing.T) {
db := testSQLiteDB(t)
tenantID := uuid.New()
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := sqlitestore.NewSQLiteSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set disjoint: legacy voice, blob params
configStore.Set(ctx, "tts.provider", "edge")
configStore.Set(ctx, "tts.edge.voice", "en-US-AriaNeural")
configStore.Set(ctx, "tts.edge.params", `{"rate":0.8,"pitch":1.2}`)
// Retrieve
voice, _ := configStore.Get(ctx, "tts.edge.voice")
if voice != "en-US-AriaNeural" {
t.Errorf("voice: got %q", voice)
}
params, _ := configStore.Get(ctx, "tts.edge.params")
if params != `{"rate":0.8,"pitch":1.2}` {
t.Errorf("params: got %q", params)
}
}
// TestTTSConfig_Store_MultiTenant_SQLite verifies tenant isolation in system_configs
// (SQLite backend).
func TestTTSConfig_Store_MultiTenant_SQLite(t *testing.T) {
db := testSQLiteDB(t)
tenantA := uuid.New()
tenantB := uuid.New()
ctxA := store.WithTenantID(context.Background(), tenantA)
ctxB := store.WithTenantID(context.Background(), tenantB)
configStore := sqlitestore.NewSQLiteSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Tenant A sets openai
configStore.Set(ctxA, "tts.provider", "openai")
// Tenant B sets elevenlabs
configStore.Set(ctxB, "tts.provider", "elevenlabs")
// Verify isolation
providerA, _ := configStore.Get(ctxA, "tts.provider")
if providerA != "openai" {
t.Errorf("tenant A: got %q, want openai", providerA)
}
providerB, _ := configStore.Get(ctxB, "tts.provider")
if providerB != "elevenlabs" {
t.Errorf("tenant B: got %q, want elevenlabs", providerB)
}
}
+178
View File
@@ -0,0 +1,178 @@
//go:build integration
package integration
import (
"context"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/pg"
)
// TestTTSConfig_Store_RoundTrip_LegacyKeys verifies that legacy flat keys
// stored in system_configs can be read back (PG backend).
func TestTTSConfig_Store_RoundTrip_LegacyKeys(t *testing.T) {
db := testDB(t)
tenantID, _ := seedTenantAgent(t, db)
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := pg.NewPGSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set legacy flat keys
if err := configStore.Set(ctx, "tts.provider", "openai"); err != nil {
t.Fatalf("Set provider: %v", err)
}
if err := configStore.Set(ctx, "tts.openai.voice", "alloy"); err != nil {
t.Fatalf("Set voice: %v", err)
}
// Read them back
provider, err := configStore.Get(ctx, "tts.provider")
if err != nil {
t.Fatalf("Get provider: %v", err)
}
if provider != "openai" {
t.Errorf("provider: got %q, want openai", provider)
}
voice, err := configStore.Get(ctx, "tts.openai.voice")
if err != nil {
t.Fatalf("Get voice: %v", err)
}
if voice != "alloy" {
t.Errorf("voice: got %q, want alloy", voice)
}
}
// TestTTSConfig_Store_RoundTrip_ParamsBlob verifies that params blob
// stored in system_configs can be read back (PG backend).
func TestTTSConfig_Store_RoundTrip_ParamsBlob(t *testing.T) {
db := testDB(t)
tenantID, _ := seedTenantAgent(t, db)
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := pg.NewPGSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set provider and params blob
configStore.Set(ctx, "tts.provider", "elevenlabs")
paramsBlob := `{"voice_settings":{"stability":0.8}}`
configStore.Set(ctx, "tts.elevenlabs.params", paramsBlob)
// Read them back
provider, _ := configStore.Get(ctx, "tts.provider")
if provider != "elevenlabs" {
t.Errorf("provider: got %q, want elevenlabs", provider)
}
retrieved, _ := configStore.Get(ctx, "tts.elevenlabs.params")
if retrieved != paramsBlob {
t.Errorf("params blob: got %q, want %q", retrieved, paramsBlob)
}
}
// TestTTSConfig_Store_Dual_LegacyAndBlob verifies that both legacy keys and
// params blob can coexist in system_configs (PG backend).
func TestTTSConfig_Store_Dual_LegacyAndBlob(t *testing.T) {
db := testDB(t)
tenantID, _ := seedTenantAgent(t, db)
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := pg.NewPGSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set both legacy and blob
configStore.Set(ctx, "tts.provider", "minimax")
configStore.Set(ctx, "tts.minimax.voice", "Wise_Woman")
configStore.Set(ctx, "tts.minimax.params", `{"speed":1.5}`)
// Retrieve both
voice, _ := configStore.Get(ctx, "tts.minimax.voice")
if voice != "Wise_Woman" {
t.Errorf("voice: got %q, want Wise_Woman", voice)
}
params, _ := configStore.Get(ctx, "tts.minimax.params")
if params != `{"speed":1.5}` {
t.Errorf("params: got %q, want {\"speed\":1.5}", params)
}
}
// TestTTSConfig_Store_DisjointUnion verifies disjoint storage and retrieval
// (legacy has only voice, blob has only params - PG backend).
func TestTTSConfig_Store_DisjointUnion(t *testing.T) {
db := testDB(t)
tenantID, _ := seedTenantAgent(t, db)
ctx := store.WithTenantID(context.Background(), tenantID)
configStore := pg.NewPGSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Set disjoint: legacy voice, blob params
configStore.Set(ctx, "tts.provider", "edge")
configStore.Set(ctx, "tts.edge.voice", "en-US-AriaNeural")
configStore.Set(ctx, "tts.edge.params", `{"rate":0.8,"pitch":1.2}`)
// Retrieve
voice, _ := configStore.Get(ctx, "tts.edge.voice")
if voice != "en-US-AriaNeural" {
t.Errorf("voice: got %q", voice)
}
params, _ := configStore.Get(ctx, "tts.edge.params")
if params != `{"rate":0.8,"pitch":1.2}` {
t.Errorf("params: got %q", params)
}
}
// TestTTSConfig_Store_MultiTenant verifies tenant isolation in system_configs
// (PG backend).
func TestTTSConfig_Store_MultiTenant(t *testing.T) {
db := testDB(t)
tenantA, _ := seedTenantAgent(t, db)
tenantB, _ := seedTenantAgent(t, db)
ctxA := store.WithTenantID(context.Background(), tenantA)
ctxB := store.WithTenantID(context.Background(), tenantB)
configStore := pg.NewPGSystemConfigStore(db)
t.Cleanup(func() {
db.Exec("DELETE FROM system_configs WHERE key LIKE 'tts.%'")
})
// Tenant A sets openai
configStore.Set(ctxA, "tts.provider", "openai")
// Tenant B sets elevenlabs
configStore.Set(ctxB, "tts.provider", "elevenlabs")
// Verify isolation
providerA, _ := configStore.Get(ctxA, "tts.provider")
if providerA != "openai" {
t.Errorf("tenant A: got %q, want openai", providerA)
}
providerB, _ := configStore.Get(ctxB, "tts.provider")
if providerB != "elevenlabs" {
t.Errorf("tenant B: got %q, want elevenlabs", providerB)
}
}
+101
View File
@@ -0,0 +1,101 @@
//go:build integration
package integration
import (
"context"
"os"
"testing"
"github.com/nextlevelbuilder/goclaw/internal/audio"
"github.com/nextlevelbuilder/goclaw/internal/audio/gemini"
)
// TestGeminiLive_UpstreamContract verifies that the Gemini provider maintains
// a valid contract with the real Google Generative AI API (preview endpoint).
//
// This test is gated on GEMINI_API_KEY env var. It synthesizes a 50-character
// text against the real API and validates the response structure and audio data.
//
// Skip conditions:
// - GEMINI_API_KEY not set (development environments)
// - 5xx responses (Google infrastructure issue, not our bug)
//
// Cadence: Monthly manual run (or opt-in nightly via CI repository secret).
//
// On failure: File issue against the provider tracking version pinning or API migration.
func TestGeminiLive_UpstreamContract(t *testing.T) {
apiKey := os.Getenv("GEMINI_API_KEY")
if apiKey == "" {
t.Skip("GEMINI_API_KEY not set; skipping live upstream test")
}
// Create Gemini provider with real API endpoint
p := gemini.NewProvider(gemini.Config{
APIKey: apiKey,
APIBase: "https://generativelanguage.googleapis.com",
Voice: "Kore",
Model: "gemini-2.0-flash-tts-preview",
})
ctx := context.Background()
// Synthesize a 50-character test text
testText := "Hello, this is a test of the Google Gemini TTS API."
result, err := p.Synthesize(ctx, testText, audio.TTSOptions{})
if err != nil {
// On 5xx: skip (not our bug)
if isServerError(err) {
t.Skipf("Google API returned 5xx: %v (infrastructure issue, not our bug)", err)
}
// On other errors: fail
t.Fatalf("Synthesize failed: %v", err)
}
// Validate result structure
if result == nil {
t.Fatal("Synthesize returned nil result")
}
// WAV header validation: first 4 bytes must be "RIFF"
if len(result.Audio) < 4 {
t.Fatalf("Audio too short (%d bytes), cannot check WAV header", len(result.Audio))
}
riffHeader := string(result.Audio[0:4])
if riffHeader != "RIFF" {
t.Errorf("WAV RIFF header invalid: got %q, want RIFF", riffHeader)
}
// MIME type validation
if result.MimeType != "audio/wav" {
t.Errorf("MIME type: got %q, want audio/wav", result.MimeType)
}
// Audio payload minimum size: non-trivial audio should be at least 2KB
if len(result.Audio) < 2048 {
t.Errorf("Audio payload too small (%d bytes), want at least 2048", len(result.Audio))
}
t.Logf("Gemini live test passed: %d bytes of audio", len(result.Audio))
}
// isServerError checks if an error is due to a 5xx response from the Google API.
// This is a simple heuristic; it checks for "500" or "503" in the error message.
func isServerError(err error) bool {
if err == nil {
return false
}
msg := err.Error()
return contains(msg, "500") || contains(msg, "503") || contains(msg, "server error")
}
// contains checks if a string contains a substring (case-insensitive).
func contains(s, substr string) bool {
for i := 0; i < len(s)-len(substr)+1; i++ {
if s[i:i+len(substr)] == substr {
return true
}
}
return false
}
+1
View File
@@ -12,6 +12,7 @@
},
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"radix-ui": "^1.4.3",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^5.2.2",
+1659
View File
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,78 @@
// Mirror of ui/web/src/api/tts-capabilities.ts for the desktop (Lite) frontend.
// No React Query — desktop fetches via ApiClient directly.
// ---- Types ----
export interface EnumOption {
value: string;
label: string;
}
export interface Dependency {
field: string;
op: string;
value: unknown;
}
export type ParamType =
| "range"
| "number"
| "integer"
| "enum"
| "boolean"
| "string"
| "text";
export interface ParamSchema {
key: string;
type: ParamType;
label: string;
description?: string;
default?: unknown;
min?: number;
max?: number;
step?: number;
enum?: EnumOption[];
depends_on?: Dependency[];
}
export interface VoiceOption {
voice_id: string;
name: string;
language?: string;
gender?: string;
}
export interface ProviderCapabilities {
provider: string;
display_name: string;
requires_api_key: boolean;
models?: string[];
voices?: VoiceOption[];
params?: ParamSchema[] | null;
custom_features?: Record<string, unknown> | null;
}
export interface CapabilitiesResponse {
providers: ProviderCapabilities[];
}
// ---- parseCapabilitiesResponse ----
/**
* Parses and validates a raw GET /v1/tts/capabilities JSON response.
* Throws if the shape is invalid.
*/
export function parseCapabilitiesResponse(raw: unknown): ProviderCapabilities[] {
if (raw === null || raw === undefined || typeof raw !== "object") {
throw new Error("capabilities response must be an object");
}
const obj = raw as Record<string, unknown>;
if (!("providers" in obj)) {
throw new Error("capabilities response missing 'providers' field");
}
if (!Array.isArray(obj.providers)) {
throw new Error("capabilities response 'providers' must be an array");
}
return obj.providers as ProviderCapabilities[];
}
@@ -1,12 +1,19 @@
/**
* Desktop VoicePicker — capabilities-driven dispatch (Phase C mirror).
* Mirrors web voice-picker.tsx dispatch logic with desktop-native UI.
*
* - "" provider → disabled empty-state
* - voices_dynamic=true in custom_features → DynamicVoicePicker
* - static voices[] in capabilities → StaticVoicePicker (Combobox)
* - MiniMax first-fetch failure → FreeTextPicker fallback
*/
import { useTranslation } from 'react-i18next'
import { useVoices, useRefreshVoices } from '../../hooks/use-voices'
import { useTtsCapabilities } from '../../hooks/use-tts-capabilities'
import { VoicePreviewButton } from './voice-preview-button'
import { Combobox } from '../common/Combobox'
import type { Voice } from '../../services/voices'
import {
getProviderDefinition,
type TtsProviderId,
} from '@/data/tts-providers'
import type { TtsProviderId } from '@/data/tts-providers'
interface Props {
value: string | null
@@ -34,25 +41,44 @@ function VoiceOption({ voice, selected }: { voice: Voice; selected: boolean }) {
}
export function VoicePicker({ value, onChange, disabled, provider }: Props) {
const { data: caps = [] } = useTtsCapabilities()
if (provider === '') {
return <EmptyStatePicker />
}
const def = provider ? getProviderDefinition(provider) : null
if (def && !def.dynamic) {
const providerCaps = provider ? caps.find((c) => c.provider === provider) : null
const voicesDynamic = providerCaps?.custom_features?.['voices_dynamic'] === true
const staticVoices = providerCaps?.voices ?? []
if (providerCaps && !voicesDynamic && staticVoices.length > 0) {
return (
<StaticVoicePicker
value={value}
onChange={onChange}
disabled={disabled}
voices={def.voices}
voices={staticVoices.map((v) => ({ value: v.voice_id, label: v.name }))}
/>
)
}
if (provider === 'minimax' || voicesDynamic) {
return (
<DynamicVoicePicker
value={value}
onChange={onChange}
disabled={disabled}
allowFreeText={provider === 'minimax'}
/>
)
}
return (
<DynamicVoicePicker
value={value}
onChange={onChange}
disabled={disabled}
allowFreeText={false}
/>
)
}
@@ -91,19 +117,48 @@ function StaticVoicePicker({
)
}
function DynamicVoicePicker({
function FreeTextVoicePicker({
value,
onChange,
disabled,
}: {
value: string | null
onChange: (voiceId: string) => void
onChange: (id: string) => void
disabled?: boolean
}) {
const { t } = useTranslation('tts')
const { data: voices, isLoading } = useVoices()
return (
<input
type="text"
className="flex h-8 w-full rounded border border-border bg-surface-secondary px-2 text-xs text-text-primary disabled:opacity-50 disabled:cursor-not-allowed"
value={value ?? ''}
onChange={(e) => onChange(e.target.value)}
disabled={disabled}
placeholder={t('voice_picker.enter_voice_id', 'Enter voice_id manually')}
/>
)
}
function DynamicVoicePicker({
value,
onChange,
disabled,
allowFreeText,
}: {
value: string | null
onChange: (voiceId: string) => void
disabled?: boolean
allowFreeText: boolean
}) {
const { t } = useTranslation('tts')
const { data: voices = [], isLoading, error } = useVoices()
const { mutate: refresh, isPending: refreshing } = useRefreshVoices()
// Fall back to free-text when first fetch fails and list is empty
if (allowFreeText && error && voices.length === 0) {
return <FreeTextVoicePicker value={value} onChange={onChange} disabled={disabled} />
}
const options = voices.map((v) => ({ value: v.voice_id, label: v.name }))
return (
@@ -0,0 +1,112 @@
/**
* AudioTagPicker — desktop mirror of web component.
* Renders in readonly mode by default when mounted from desktop TTS page.
*/
import { useState } from "react";
import { useTranslation } from "react-i18next";
import {
GEMINI_AUDIO_TAGS,
filterTagsByCategory,
searchTags,
insertTagAtCaret,
type AudioTag,
} from "../data/gemini-audio-tags";
type Category = AudioTag["category"] | "all";
interface AudioTagPickerProps {
onInsert?: (tag: string) => void;
caretPos?: number;
textValue?: string;
onTextChange?: (newText: string) => void;
readonly?: boolean;
className?: string;
}
const CATEGORIES: { key: Category; labelKey: string }[] = [
{ key: "all", labelKey: "gemini.filterAll" },
{ key: "emotion", labelKey: "gemini.filterEmotion" },
{ key: "pacing", labelKey: "gemini.filterPacing" },
{ key: "effect", labelKey: "gemini.filterEffect" },
{ key: "voice_quality", labelKey: "gemini.filterVoiceQuality" },
];
export function AudioTagPicker({
onInsert,
caretPos = 0,
textValue = "",
onTextChange,
readonly = true, // desktop defaults to readonly until Phase C wires it
className,
}: AudioTagPickerProps) {
const { t } = useTranslation("tts");
const [query, setQuery] = useState("");
const [activeCategory, setActiveCategory] = useState<Category>("all");
const tags: AudioTag[] =
query.trim()
? searchTags(query)
: activeCategory === "all"
? GEMINI_AUDIO_TAGS
: filterTagsByCategory(activeCategory);
const handleInsert = (tag: string) => {
if (readonly) return;
if (onTextChange && textValue !== undefined) {
onTextChange(insertTagAtCaret(textValue, tag, caretPos));
}
onInsert?.(tag);
};
return (
<div className={className} style={{ display: "flex", flexDirection: "column", gap: "0.5rem" }}>
<p style={{ fontSize: "0.75rem", opacity: 0.7 }}>{t("gemini.audioTagsHint")}</p>
<input
type="text"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder={t("gemini.searchPlaceholder")}
disabled={readonly}
/>
{!query.trim() && (
<div style={{ display: "flex", flexWrap: "wrap", gap: "0.25rem" }}>
{CATEGORIES.map(({ key, labelKey }) => (
<button
key={key}
type="button"
onClick={() => setActiveCategory(key)}
style={{
fontWeight: activeCategory === key ? "bold" : "normal",
fontSize: "0.75rem",
}}
>
{t(labelKey)}
</button>
))}
</div>
)}
<div style={{ display: "flex", flexWrap: "wrap", gap: "0.375rem", maxHeight: "10rem", overflowY: "auto" }}>
{tags.map((tag) => (
<button
key={tag.tag}
type="button"
title={tag.description}
onClick={() => handleInsert(tag.tag)}
disabled={readonly}
style={{
fontFamily: "monospace",
fontSize: "0.75rem",
opacity: readonly ? 0.6 : 1,
cursor: readonly ? "default" : "pointer",
}}
>
{tag.tag}
</button>
))}
</div>
</div>
);
}
@@ -1,5 +1,6 @@
import { useState, useRef, useEffect, useCallback, useLayoutEffect, type KeyboardEvent } from 'react'
import { createPortal } from 'react-dom'
import { usePortalDropdownClose } from '@/hooks/use-portal-dropdown-close'
interface ComboboxOption {
value: string
@@ -50,17 +51,13 @@ export function Combobox({ value, onChange, options, placeholder, loading, allow
}
}, [open])
// Close on outside click
useEffect(() => {
if (!open) return
const handler = (e: MouseEvent) => {
if (dropdownRef.current?.contains(e.target as Node)) return
if (inputRef.current?.contains(e.target as Node)) return
setOpen(false)
}
document.addEventListener('mousedown', handler)
return () => document.removeEventListener('mousedown', handler)
}, [open])
usePortalDropdownClose({
open,
onClose: () => setOpen(false),
ignore: [inputRef, dropdownRef],
// Scroll is used by useLayoutEffect above to REPOSITION, not close.
closeOnOutsideScroll: false,
})
const handleSelect = useCallback((val: string) => {
onChange(val)
@@ -0,0 +1,132 @@
import * as React from "react"
import type { ParamSchema, ParamType } from "@/api/tts-capabilities"
import { Slider } from "@/components/ui/slider"
import { Switch } from "@/components/common/Switch"
export type ParamValue = string | number | boolean
export interface FieldProps {
schema: ParamSchema
value: ParamValue
onChange: (val: ParamValue) => void
readonly: boolean
}
function RangeField({ schema, value, onChange, readonly }: FieldProps) {
const num = typeof value === "number" ? value : Number(value ?? schema.default ?? 0)
return (
<div className="flex items-center gap-3">
<Slider
min={schema.min ?? 0}
max={schema.max ?? 1}
step={schema.step ?? 0.01}
value={[num]}
onValueChange={readonly ? undefined : ([v = 0]) => onChange(v)}
disabled={readonly}
aria-label={schema.label}
className="flex-1"
/>
<span className="text-sm tabular-nums w-10 text-right">{num.toFixed(2)}</span>
</div>
)
}
function NumberField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="number"
className="border border-surface-secondary rounded px-2 py-1 text-sm w-full bg-surface"
min={schema.min}
max={schema.max}
step={schema.step ?? 1}
value={value as number}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(Number(e.target.value))}
aria-label={schema.label}
/>
)
}
function IntegerField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="number"
className="border border-surface-secondary rounded px-2 py-1 text-sm w-full bg-surface"
min={schema.min}
max={schema.max}
step={1}
value={value as number}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(Math.round(Number(e.target.value)))}
aria-label={schema.label}
/>
)
}
function EnumField({ schema, value, onChange, readonly }: FieldProps) {
return (
<select
className="border border-surface-secondary rounded px-2 py-1 text-sm w-full bg-surface"
value={value as string}
disabled={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
>
{(schema.enum ?? []).map((opt) => (
<option key={opt.value} value={opt.value}>
{opt.label}
</option>
))}
</select>
)
}
function BooleanField({ schema, value, onChange, readonly }: FieldProps) {
// Desktop Switch requires a non-optional handler; readonly is enforced via disabled + noop.
const handler = readonly ? () => {} : (v: boolean) => onChange(v)
return (
<Switch
checked={Boolean(value)}
onCheckedChange={handler}
disabled={readonly}
id={`param-bool-${schema.key}`}
/>
)
}
function StringField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="text"
className="border border-surface-secondary rounded px-2 py-1 text-sm w-full bg-surface"
value={value as string}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
/>
)
}
function TextField({ schema, value, onChange, readonly }: FieldProps) {
return (
<textarea
className="border border-surface-secondary rounded px-2 py-1 text-sm w-full bg-surface resize-y min-h-[4rem]"
value={value as string}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
/>
)
}
export const fieldRenderers: Record<ParamType, React.FC<FieldProps>> = {
range: RangeField,
number: NumberField,
integer: IntegerField,
enum: EnumField,
boolean: BooleanField,
string: StringField,
text: TextField,
}
export { StringField as DefaultField }
@@ -0,0 +1,53 @@
/**
* Pure logic helpers for DynamicParamForm — desktop mirror.
* Mirrors ui/web/src/components/dynamic-param-form-logic.ts
* No React, no DOM — safe to import in unit tests without jsdom.
*/
import type { ParamSchema, ParamType } from "@/api/tts-capabilities";
export type ParamValueMap = Record<string, string | number | boolean>;
/**
* Builds an initial value object from an ordered list of ParamSchema.
* Each key maps to schema.default (or "" when default is absent).
* Keys are stored flat (dot-notation preserved as-is) — matches backend wire format.
*/
export function initializeDefaults(schemas: ParamSchema[]): ParamValueMap {
const out: ParamValueMap = {};
for (const s of schemas) {
out[s.key] = s.default !== undefined ? (s.default as string | number | boolean) : "";
}
return out;
}
/**
* Returns a new map with key set to val — immutable (does not mutate src).
* Key may be dot-notation (e.g. "voice_settings.stability"); stored flat as-is.
*/
export function applyNestedChange(
src: ParamValueMap,
key: string,
val: string | number | boolean,
): ParamValueMap {
return { ...src, [key]: val };
}
/**
* Returns the renderer name for a ParamType (identity function).
* Exists so tests can assert the dispatch table without importing React.
*/
export function getRendererName(type: ParamType): ParamType {
return type;
}
/**
* Coerces a potentially-NaN numeric input to a safe value.
* Used by RangeField / NumberField / IntegerField onChange handlers.
*/
export function coerceNumericValue(
val: number,
defaultVal: number | undefined,
): number {
if (Number.isNaN(val)) return defaultVal ?? 0;
return val;
}
@@ -0,0 +1,89 @@
import type { ParamSchema } from "@/api/tts-capabilities"
import {
fieldRenderers,
DefaultField,
type FieldProps,
type ParamValue,
} from "./dynamic-param-form-fields"
export type { ParamValue, FieldProps }
export interface DynamicParamFormProps {
/** Ordered list of param schemas from ProviderCapabilities.params */
schema: ParamSchema[]
/** Current values keyed by ParamSchema.key */
value: Record<string, ParamValue>
/** Called when a field changes. Not fired when readonly=true. */
onChange?: (key: string, val: ParamValue) => void
/**
* When true all inputs are rendered in read-only / display mode.
* Desktop always passes readonly={true} (Phase C constraint).
*/
readonly?: boolean
}
/**
* Returns true when all DependsOn constraints are satisfied by formState (AND semantics).
* An empty DependsOn array means always visible.
*/
export function evaluateDependsOn(
deps: ParamSchema["depends_on"],
formState: Record<string, ParamValue>,
): boolean {
if (!deps || deps.length === 0) return true
return deps.every((d) => String(formState[d.field]) === String(d.value))
}
/**
* Renders a list of TTS provider params from a ParamSchema array.
* Each field type maps to a dedicated renderer. Visibility is gated by
* evaluateDependsOn. When readonly=true, no onChange callbacks fire.
*
* Desktop callers MUST pass readonly={true} per Phase C constraint.
* NOT mounted in any settings page in Phase A — exported for Phase C wiring.
*/
export function DynamicParamForm({
schema,
value,
onChange,
readonly = false,
}: DynamicParamFormProps) {
if (!schema || schema.length === 0) return null
return (
<div className="space-y-4">
{schema.map((param) => {
if (!evaluateDependsOn(param.depends_on, value)) return null
const Renderer = fieldRenderers[param.type] ?? DefaultField
const currentVal: ParamValue =
value[param.key] !== undefined
? value[param.key]!
: ((param.default as ParamValue) ?? "")
const handleChange = (val: ParamValue) => {
if (!readonly && onChange) {
onChange(param.key, val)
}
}
return (
<div key={param.key} className="space-y-1">
<label className="text-sm font-medium" htmlFor={`param-${param.key}`}>
{param.label}
</label>
{param.description && (
<p className="text-xs text-text-secondary">{param.description}</p>
)}
<Renderer
schema={param}
value={currentVal}
onChange={handleChange}
readonly={readonly}
/>
</div>
)
})}
</div>
)
}
@@ -0,0 +1,117 @@
/**
* MultiSpeakerEditor — desktop mirror of web component.
* Renders in readonly mode by default when mounted from desktop TTS page.
* Exports pure-logic helpers shared with tests.
*/
import { useTranslation } from "react-i18next";
// ---- Pure logic exports (testable without React) ----
export interface SpeakerVoice {
speaker: string;
voiceId: string;
}
/** Maximum speakers Gemini TTS supports in multi-speaker mode. */
export const MAX_GEMINI_SPEAKERS = 2;
/** Returns a new array with speaker appended, capped at MAX_GEMINI_SPEAKERS. */
export function addSpeaker(speakers: SpeakerVoice[], entry: SpeakerVoice): SpeakerVoice[] {
if (speakers.length >= MAX_GEMINI_SPEAKERS) return speakers;
return [...speakers, entry];
}
/** Returns a new array with the speaker at `index` removed. */
export function removeSpeaker(speakers: SpeakerVoice[], index: number): SpeakerVoice[] {
return speakers.filter((_, i) => i !== index);
}
/** Returns a new array with the voiceId at `index` replaced. */
export function updateSpeakerVoice(
speakers: SpeakerVoice[],
index: number,
voiceId: string
): SpeakerVoice[] {
return speakers.map((s, i) => (i === index ? { ...s, voiceId } : s));
}
// ---- React component ----
interface MultiSpeakerEditorProps {
speakers: SpeakerVoice[];
voices: string[];
onChange: (speakers: SpeakerVoice[]) => void;
readonly?: boolean;
className?: string;
}
export function MultiSpeakerEditor({
speakers,
voices,
onChange,
readonly = true, // desktop defaults to readonly until Phase C wires it
className,
}: MultiSpeakerEditorProps) {
const { t } = useTranslation("tts");
const atMax = speakers.length >= MAX_GEMINI_SPEAKERS;
const handleAdd = () => {
if (readonly || atMax) return;
onChange(addSpeaker(speakers, { speaker: "", voiceId: voices[0] ?? "" }));
};
const handleRemove = (idx: number) => {
if (readonly) return;
onChange(removeSpeaker(speakers, idx));
};
const handleNameChange = (idx: number, name: string) => {
if (readonly) return;
onChange(speakers.map((s, i) => (i === idx ? { ...s, speaker: name } : s)));
};
const handleVoiceChange = (idx: number, voiceId: string) => {
if (readonly) return;
onChange(updateSpeakerVoice(speakers, idx, voiceId));
};
return (
<div className={className} style={{ display: "flex", flexDirection: "column", gap: "0.5rem" }}>
<p style={{ fontSize: "0.75rem", opacity: 0.7 }}>{t("gemini.multiSpeakerHint")}</p>
{speakers.map((s, idx) => (
<div key={idx} style={{ display: "flex", gap: "0.5rem", alignItems: "center" }}>
<input
type="text"
value={s.speaker}
onChange={(e) => handleNameChange(idx, e.target.value)}
placeholder={t("gemini.speakerName")}
disabled={readonly}
style={{ flex: 1 }}
/>
<select
value={s.voiceId}
onChange={(e) => handleVoiceChange(idx, e.target.value)}
disabled={readonly}
style={{ flex: 1 }}
>
{voices.map((v) => (
<option key={v} value={v}>{v}</option>
))}
</select>
{!readonly && (
<button type="button" onClick={() => handleRemove(idx)}>
{t("gemini.removeSpeaker")}
</button>
)}
</div>
))}
{!readonly && (
<button type="button" onClick={handleAdd} disabled={atMax}>
{atMax ? t("gemini.maxSpeakersReached") : t("gemini.addSpeaker")}
</button>
)}
</div>
);
}
@@ -0,0 +1,40 @@
import * as React from "react"
import { Slider as SliderPrimitive } from "radix-ui"
interface SliderProps extends React.ComponentProps<typeof SliderPrimitive.Root> {
className?: string
}
/**
* Slider — Radix UI wrapper for the desktop (Lite) frontend.
* Styling matches the web UI primitive without requiring the `cn` utility.
*/
function Slider({ className, ...props }: SliderProps) {
return (
<SliderPrimitive.Root
data-slot="slider"
className={[
"relative flex w-full touch-none select-none items-center",
"data-[disabled]:opacity-50 data-[disabled]:cursor-not-allowed",
className ?? "",
].join(" ")}
{...props}
>
<SliderPrimitive.Track
data-slot="slider-track"
className="bg-surface-tertiary relative h-1.5 w-full grow overflow-hidden rounded-full"
>
<SliderPrimitive.Range
data-slot="slider-range"
className="bg-accent absolute h-full"
/>
</SliderPrimitive.Track>
<SliderPrimitive.Thumb
data-slot="slider-thumb"
className="border-accent bg-white block size-4 shrink-0 rounded-full border shadow focus-visible:ring-2 focus-visible:ring-accent focus-visible:outline-none disabled:pointer-events-none disabled:opacity-50 transition-colors"
/>
</SliderPrimitive.Root>
)
}
export { Slider }
@@ -1,15 +1,19 @@
/**
* Desktop TTS provider catalog sanity tests.
* Mirrors ui/web/src/data/__tests__/tts-providers.test.ts — keeps desktop
* catalog in sync with the web catalog via identical assertions.
* Desktop TTS provider catalog sanity tests — reduced shape (Phase C).
* Mirrors ui/web/src/data/__tests__/tts-providers.test.ts
*/
import { describe, it, expect } from "vitest";
import { TTS_PROVIDERS, getProviderDefinition, type TtsProviderId } from "../tts-providers";
import {
TTS_PROVIDERS,
PROVIDER_MODEL_CATALOG,
getProviderDefinition,
type TtsProviderId,
} from "../tts-providers";
describe("Desktop TTS_PROVIDERS catalog", () => {
const ids: TtsProviderId[] = ["openai", "elevenlabs", "edge", "minimax"];
describe("Desktop TTS_PROVIDERS catalog (reduced shape)", () => {
const ids: TtsProviderId[] = ["openai", "elevenlabs", "edge", "minimax", "gemini"];
it("has exactly 4 provider entries matching known IDs", () => {
it("has 5 provider entries matching known IDs", () => {
expect(Object.keys(TTS_PROVIDERS).sort()).toEqual([...ids].sort());
});
@@ -17,62 +21,14 @@ describe("Desktop TTS_PROVIDERS catalog", () => {
expect(TTS_PROVIDERS[id].id).toBe(id);
});
it("ElevenLabs is dynamic, others are not", () => {
expect(TTS_PROVIDERS.elevenlabs.dynamic).toBe(true);
expect(TTS_PROVIDERS.openai.dynamic).toBe(false);
expect(TTS_PROVIDERS.edge.dynamic).toBe(false);
expect(TTS_PROVIDERS.minimax.dynamic).toBe(false);
it.each(ids)("%s has non-empty title and desc", (id) => {
expect(TTS_PROVIDERS[id].title.length).toBeGreaterThan(0);
expect(TTS_PROVIDERS[id].desc.length).toBeGreaterThan(0);
});
it.each(ids.filter((id) => TTS_PROVIDERS[id].models.length > 0))(
"%s.defaultModel is in its models list",
(id) => {
const def = TTS_PROVIDERS[id];
if (def.defaultModel) {
const modelValues = def.models.map((m) => m.value);
expect(modelValues).toContain(def.defaultModel);
}
},
);
it.each(ids.filter((id) => !TTS_PROVIDERS[id].dynamic))(
"%s.defaultVoice is in its voices list",
(id) => {
const def = TTS_PROVIDERS[id];
if (def.defaultVoice) {
const voiceValues = def.voices.map((v) => v.value);
expect(voiceValues).toContain(def.defaultVoice);
}
},
);
it("Edge requires no API key", () => {
expect(TTS_PROVIDERS.edge.requiresApiKey).toBe(false);
});
it("OpenAI and MiniMax require an API key", () => {
expect(TTS_PROVIDERS.openai.requiresApiKey).toBe(true);
expect(TTS_PROVIDERS.minimax.requiresApiKey).toBe(true);
});
it("ElevenLabs exposes all 4 backend-allowlisted models", () => {
const modelIds = TTS_PROVIDERS.elevenlabs.models.map((m) => m.value);
expect(modelIds).toContain("eleven_v3");
expect(modelIds).toContain("eleven_flash_v2_5");
expect(modelIds).toContain("eleven_multilingual_v2");
expect(modelIds).toContain("eleven_turbo_v2_5");
expect(modelIds).toHaveLength(4);
});
it("OpenAI models include the 3 standard model IDs", () => {
const ids = TTS_PROVIDERS.openai.models.map((m) => m.value);
expect(ids).toContain("gpt-4o-mini-tts");
expect(ids).toContain("tts-1");
expect(ids).toContain("tts-1-hd");
});
it("Edge has 20 hardcoded voices", () => {
expect(TTS_PROVIDERS.edge.voices).toHaveLength(20);
it.each(ids)("%s has a color string", (id) => {
expect(typeof TTS_PROVIDERS[id].color).toBe("string");
expect(TTS_PROVIDERS[id].color.length).toBeGreaterThan(0);
});
it("getProviderDefinition returns null for unknown id", () => {
@@ -84,3 +40,32 @@ describe("Desktop TTS_PROVIDERS catalog", () => {
expect(getProviderDefinition("openai")?.id).toBe("openai");
});
});
describe("Desktop PROVIDER_MODEL_CATALOG (static fallback)", () => {
it("ElevenLabs exposes all 4 backend-allowlisted models", () => {
const modelIds = PROVIDER_MODEL_CATALOG.elevenlabs.map((m) => m.value);
expect(modelIds).toContain("eleven_v3");
expect(modelIds).toContain("eleven_flash_v2_5");
expect(modelIds).toContain("eleven_multilingual_v2");
expect(modelIds).toContain("eleven_turbo_v2_5");
expect(modelIds).toHaveLength(4);
});
it("OpenAI models include the 3 standard model IDs", () => {
const modelIds = PROVIDER_MODEL_CATALOG.openai.map((m) => m.value);
expect(modelIds).toContain("gpt-4o-mini-tts");
expect(modelIds).toContain("tts-1");
expect(modelIds).toContain("tts-1-hd");
});
it("Edge has no models (voice-only provider)", () => {
expect(PROVIDER_MODEL_CATALOG.edge).toHaveLength(0);
});
it("MiniMax has at least 2 models", () => {
expect(PROVIDER_MODEL_CATALOG.minimax.length).toBeGreaterThanOrEqual(2);
const ids = PROVIDER_MODEL_CATALOG.minimax.map((m) => m.value);
expect(ids).toContain("speech-02-hd");
expect(ids).toContain("speech-02-turbo");
});
});
@@ -0,0 +1,102 @@
/**
* Gemini TTS audio tag catalog.
*
* Source: https://ai.google.dev/gemini-api/docs/speech-generation (April 2026)
* Note: Google does not publish an exhaustive list. This catalog includes all
* officially documented tags plus commonly effective ones from community usage.
* Tag count: 42 curated entries across 4 categories.
*
* Tags are inline text markers inserted directly in the synthesis text, e.g.:
* "Hello [laughs] how are you?"
* No special API field — Gemini parses them from the `text` payload.
*/
export interface AudioTag {
tag: string; // the literal marker, e.g. "[laughs]"
category: "emotion" | "pacing" | "effect" | "voice_quality";
description: string;
}
export const GEMINI_AUDIO_TAGS: AudioTag[] = [
// --- Emotion ---
{ tag: "[laughs]", category: "emotion", description: "Natural laughter" },
{ tag: "[laughs softly]",category: "emotion", description: "Soft, gentle laughter" },
{ tag: "[giggles]", category: "emotion", description: "Light giggles" },
{ tag: "[crying]", category: "emotion", description: "Tearful, emotional tone" },
{ tag: "[excited]", category: "emotion", description: "High-energy, enthusiastic" },
{ tag: "[excitedly]", category: "emotion", description: "Speaking with excitement" },
{ tag: "[amazed]", category: "emotion", description: "Surprised and impressed" },
{ tag: "[curious]", category: "emotion", description: "Inquisitive tone" },
{ tag: "[serious]", category: "emotion", description: "Grave, solemn tone" },
{ tag: "[panicked]", category: "emotion", description: "Anxious, rushed delivery" },
{ tag: "[mischievously]",category: "emotion", description: "Playfully scheming tone" },
{ tag: "[sarcastic]", category: "emotion", description: "Sarcastic delivery" },
{ tag: "[bored]", category: "emotion", description: "Disinterested, flat tone" },
{ tag: "[reluctantly]", category: "emotion", description: "Unwilling, hesitant" },
{ tag: "[trembling]", category: "emotion", description: "Shaky, fearful voice" },
{ tag: "[gasp]", category: "emotion", description: "Audible sharp intake of breath" },
{ tag: "[sighs]", category: "emotion", description: "Audible exhale / sigh" },
{ tag: "[shouting]", category: "emotion", description: "Loud, forceful delivery" },
// --- Pacing ---
{ tag: "[very fast]", category: "pacing", description: "Much faster than normal" },
{ tag: "[fast]", category: "pacing", description: "Slightly faster delivery" },
{ tag: "[very slow]", category: "pacing", description: "Much slower, deliberate pace" },
{ tag: "[slow]", category: "pacing", description: "Slightly slower delivery" },
{ tag: "[pause]", category: "pacing", description: "Brief pause in speech" },
{ tag: "[long pause]", category: "pacing", description: "Longer pause / silence" },
{ tag: "[drawn out]", category: "pacing", description: "Stretched, elongated syllables" },
{ tag: "[sarcastically, one painfully slow word at a time]",
category: "pacing", description: "Extremely slow sarcastic delivery" },
// --- Effect ---
{ tag: "[cough]", category: "effect", description: "Coughing sound" },
{ tag: "[yawn]", category: "effect", description: "Yawning sound" },
{ tag: "[sniffles]", category: "effect", description: "Sniffling, like after crying" },
{ tag: "[clears throat]",category: "effect", description: "Throat clearing sound" },
{ tag: "[gasps]", category: "effect", description: "Sharp gasp" },
{ tag: "[exhales]", category: "effect", description: "Audible exhale" },
// --- Voice quality ---
{ tag: "[whispers]", category: "voice_quality", description: "Quiet, hushed whisper" },
{ tag: "[whisper]", category: "voice_quality", description: "Whispered tone" },
{ tag: "[tired]", category: "voice_quality", description: "Weary, low-energy voice" },
{ tag: "[like a cartoon dog]",
category: "voice_quality", description: "Cartoonish dog-like voice" },
{ tag: "[like dracula]", category: "voice_quality", description: "Dramatic Transylvanian accent" },
{ tag: "[robotic]", category: "voice_quality", description: "Mechanical, synthesized quality" },
{ tag: "[sing-song]", category: "voice_quality", description: "Musical, lilting quality" },
{ tag: "[monotone]", category: "voice_quality", description: "Flat, emotionless delivery" },
{ tag: "[breathy]", category: "voice_quality", description: "Soft, airy vocal quality" },
{ tag: "[gruff]", category: "voice_quality", description: "Rough, low, gravelly voice" },
];
/** Return tags filtered by category (undefined = all). */
export function filterTagsByCategory(
category?: AudioTag["category"]
): AudioTag[] {
if (!category) return GEMINI_AUDIO_TAGS;
return GEMINI_AUDIO_TAGS.filter((t) => t.category === category);
}
/** Case-insensitive search across tag text and description. */
export function searchTags(query: string): AudioTag[] {
const q = query.toLowerCase();
return GEMINI_AUDIO_TAGS.filter(
(t) =>
t.tag.toLowerCase().includes(q) || t.description.toLowerCase().includes(q)
);
}
/** Insert a tag at the caret position (|) within text.
* Caret is represented by the cursor index (not a literal `|` char).
* Returns the new text with tag inserted and a single space boundary on each side. */
export function insertTagAtCaret(text: string, tag: string, caretPos: number): string {
const before = text.slice(0, caretPos);
const after = text.slice(caretPos);
const needSpaceBefore = before.length > 0 && !before.endsWith(" ");
const needSpaceAfter = after.length > 0 && !after.startsWith(" ");
const prefix = needSpaceBefore ? " " : "";
const suffix = needSpaceAfter ? " " : "";
return before + prefix + tag + suffix + after;
}
+87 -146
View File
@@ -1,158 +1,99 @@
// KEEP IN SYNC with ui/web/src/data/tts-providers.ts
//
// TTS provider catalog — source of truth for non-ElevenLabs providers.
//
// ElevenLabs voices are fetched dynamically via /v1/voices (dynamic: true).
// OpenAI, Edge, and MiniMax voices are hardcoded here because their backends
// accept any string and fall back to provider defaults.
//
// Backend provider name refs:
// openai → internal/audio/openai/tts.go:59
// elevenlabs → internal/audio/elevenlabs/tts.go Name()
// edge → internal/audio/edge/tts.go:47
// minimax → internal/audio/minimax/tts.go:64
/**
* TTS provider display metadata — reduced shape (Phase C).
* Desktop mirror of ui/web/src/data/tts-providers.ts
*
* REMOVED: models, voices, dynamic, defaultVoice, defaultModel, requiresApiKey.
* - Models/voices/params come from GET /v1/tts/capabilities.
* - requiresApiKey derived from capabilities.requires_api_key.
*
* KEPT: TtsProviderId union type for type narrowing; id, title, color, desc.
*
* KEEP IN SYNC with ui/web/src/data/tts-providers.ts
*/
export type TtsProviderId = "openai" | "elevenlabs" | "edge" | "minimax";
export type TtsProviderId = "openai" | "elevenlabs" | "edge" | "minimax" | "gemini";
export interface TtsVoiceOption {
value: string;
label: string;
locale?: string;
/** Minimal display info only — no runtime data. */
export interface TtsProviderMeta {
id: TtsProviderId;
title: string;
color: string;
desc: string;
}
export const TTS_PROVIDERS: Record<TtsProviderId, TtsProviderMeta> = {
elevenlabs: {
id: "elevenlabs",
title: "ElevenLabs",
color: "#000",
desc: "High-quality neural voices with dynamic library",
},
openai: {
id: "openai",
title: "OpenAI",
color: "#10a37f",
desc: "Reliable TTS via OpenAI API",
},
edge: {
id: "edge",
title: "Edge (Free)",
color: "#0078d4",
desc: "Microsoft Edge TTS — free, no API key required",
},
minimax: {
id: "minimax",
title: "MiniMax",
color: "#6200ea",
desc: "MiniMax TTS with dynamic voice library",
},
gemini: {
id: "gemini",
title: "Google Gemini",
color: "#1a73e8",
desc: "Gemini 3.1 Flash TTS — 70+ languages, multi-speaker, audio tags (preview)",
},
};
/**
* Returns provider display meta by id, or null if id is unknown/empty.
*/
export function getProviderDefinition(id: string): TtsProviderMeta | null {
return TTS_PROVIDERS[id as TtsProviderId] ?? null;
}
/** Minimal model option shape for synchronous catalog fallback. */
export interface TtsModelOption {
value: string;
label: string;
description?: string;
}
export interface TtsProviderDefinition {
id: TtsProviderId;
/** true = voices fetched from /v1/voices; false = use voices[] below */
dynamic: boolean;
voices: TtsVoiceOption[];
models: TtsModelOption[];
defaultVoice?: string;
defaultModel?: string;
requiresApiKey: boolean;
}
export const TTS_PROVIDERS: Record<TtsProviderId, TtsProviderDefinition> = {
// ElevenLabs — voices fetched from server; models hardcoded from allowlist
// Ref: internal/audio/elevenlabs/models.go:14-19
// Defaults: internal/config/config_channels.go:499-500
elevenlabs: {
id: "elevenlabs",
dynamic: true,
voices: [],
models: [
{ value: "eleven_v3", label: "Eleven v3" },
{ value: "eleven_flash_v2_5", label: "Eleven Flash v2.5" },
{ value: "eleven_multilingual_v2", label: "Eleven Multilingual v2" },
{ value: "eleven_turbo_v2_5", label: "Eleven Turbo v2.5" },
],
defaultVoice: "pMsXgVXv3BLzUgSXRplE",
defaultModel: "eleven_multilingual_v2",
requiresApiKey: true,
},
// OpenAI TTS — docs: https://platform.openai.com/docs/guides/text-to-speech
// Ref: internal/audio/openai/tts.go:46-51
openai: {
id: "openai",
dynamic: false,
voices: [
{ value: "alloy", label: "Alloy" },
{ value: "echo", label: "Echo" },
{ value: "fable", label: "Fable" },
{ value: "onyx", label: "Onyx" },
{ value: "nova", label: "Nova" },
{ value: "shimmer", label: "Shimmer" },
{ value: "coral", label: "Coral" },
{ value: "sage", label: "Sage" },
],
models: [
{ value: "gpt-4o-mini-tts", label: "GPT-4o Mini TTS" },
{ value: "tts-1", label: "TTS-1" },
{ value: "tts-1-hd", label: "TTS-1 HD" },
],
defaultVoice: "alloy",
defaultModel: "gpt-4o-mini-tts",
requiresApiKey: true,
},
// Microsoft Edge TTS — free, no API key required
// Ref: internal/audio/edge/tts.go:37-39
// Top-20 neural voices from edge-tts registry
edge: {
id: "edge",
dynamic: false,
voices: [
{ value: "en-US-MichelleNeural", label: "Michelle (en-US)", locale: "en-US" },
{ value: "en-US-AriaNeural", label: "Aria (en-US)", locale: "en-US" },
{ value: "en-US-GuyNeural", label: "Guy (en-US)", locale: "en-US" },
{ value: "en-GB-SoniaNeural", label: "Sonia (en-GB)", locale: "en-GB" },
{ value: "en-GB-RyanNeural", label: "Ryan (en-GB)", locale: "en-GB" },
{ value: "en-AU-NatashaNeural", label: "Natasha (en-AU)", locale: "en-AU" },
{ value: "vi-VN-HoaiMyNeural", label: "Hoài My (vi-VN)", locale: "vi-VN" },
{ value: "vi-VN-NamMinhNeural", label: "Nam Minh (vi-VN)", locale: "vi-VN" },
{ value: "zh-CN-XiaoxiaoNeural", label: "Xiaoxiao (zh-CN)", locale: "zh-CN" },
{ value: "zh-CN-YunxiNeural", label: "Yunxi (zh-CN)", locale: "zh-CN" },
{ value: "ja-JP-NanamiNeural", label: "Nanami (ja-JP)", locale: "ja-JP" },
{ value: "ko-KR-SunHiNeural", label: "Sun Hi (ko-KR)", locale: "ko-KR" },
{ value: "es-ES-ElviraNeural", label: "Elvira (es-ES)", locale: "es-ES" },
{ value: "fr-FR-DeniseNeural", label: "Denise (fr-FR)", locale: "fr-FR" },
{ value: "de-DE-KatjaNeural", label: "Katja (de-DE)", locale: "de-DE" },
{ value: "it-IT-ElsaNeural", label: "Elsa (it-IT)", locale: "it-IT" },
{ value: "pt-BR-FranciscaNeural", label: "Francisca (pt-BR)", locale: "pt-BR" },
{ value: "ru-RU-SvetlanaNeural", label: "Svetlana (ru-RU)", locale: "ru-RU" },
{ value: "hi-IN-SwaraNeural", label: "Swara (hi-IN)", locale: "hi-IN" },
{ value: "id-ID-GadisNeural", label: "Gadis (id-ID)", locale: "id-ID" },
],
models: [],
defaultVoice: "en-US-MichelleNeural",
requiresApiKey: false,
},
// MiniMax TTS — docs: https://platform.minimax.io/docs/api-reference/speech-t2a-intro
// Ref: internal/audio/minimax/tts.go:50-56
minimax: {
id: "minimax",
dynamic: false,
voices: [
{ value: "Wise_Woman", label: "Wise Woman" },
{ value: "Friendly_Person", label: "Friendly Person" },
{ value: "Inspirational_girl", label: "Inspirational Girl" },
{ value: "Deep_Voice_Man", label: "Deep Voice Man" },
{ value: "Calm_Woman", label: "Calm Woman" },
{ value: "Casual_Guy", label: "Casual Guy" },
{ value: "Lively_Girl", label: "Lively Girl" },
{ value: "Patient_Man", label: "Patient Man" },
{ value: "Young_Knight", label: "Young Knight" },
{ value: "Determined_Man", label: "Determined Man" },
{ value: "Lovely_Girl", label: "Lovely Girl" },
{ value: "Decent_Boy", label: "Decent Boy" },
{ value: "Imposing_Manner", label: "Imposing Manner" },
{ value: "Elegant_Man", label: "Elegant Man" },
{ value: "Abbess", label: "Abbess" },
{ value: "Sweet_Girl_2", label: "Sweet Girl 2" },
{ value: "Exuberant_Girl", label: "Exuberant Girl" },
],
models: [
{ value: "speech-02-hd", label: "Speech-02 HD" },
{ value: "speech-02-turbo", label: "Speech-02 Turbo" },
{ value: "speech-01-hd", label: "Speech-01 HD" },
{ value: "speech-01-turbo", label: "Speech-01 Turbo" },
],
defaultVoice: "Wise_Woman",
defaultModel: "speech-02-hd",
requiresApiKey: true,
},
};
/**
* Returns provider definition by id, or null if id is unknown/empty.
* Synchronous model catalog fallback for consumers that cannot await capabilities.
* Kept in sync with backend allowlists.
*/
export function getProviderDefinition(id: string): TtsProviderDefinition | null {
return TTS_PROVIDERS[id as TtsProviderId] ?? null;
}
export const PROVIDER_MODEL_CATALOG: Record<TtsProviderId, TtsModelOption[]> = {
elevenlabs: [
{ value: "eleven_v3", label: "Eleven v3" },
{ value: "eleven_flash_v2_5", label: "Eleven Flash v2.5" },
{ value: "eleven_multilingual_v2", label: "Eleven Multilingual v2" },
{ value: "eleven_turbo_v2_5", label: "Eleven Turbo v2.5" },
],
openai: [
{ value: "gpt-4o-mini-tts", label: "GPT-4o Mini TTS" },
{ value: "tts-1", label: "TTS-1" },
{ value: "tts-1-hd", label: "TTS-1 HD" },
],
edge: [],
minimax: [
{ value: "speech-02-hd", label: "Speech-02 HD" },
{ value: "speech-02-turbo", label: "Speech-02 Turbo" },
{ value: "speech-01-hd", label: "Speech-01 HD" },
{ value: "speech-01-turbo", label: "Speech-01 Turbo" },
],
gemini: [
{ value: "gemini-3.1-flash-tts-preview", label: "Gemini 3.1 Flash TTS (preview)" },
{ value: "gemini-2.5-flash-preview-tts", label: "Gemini 2.5 Flash TTS (preview)" },
{ value: "gemini-2.5-pro-preview-tts", label: "Gemini 2.5 Pro TTS (preview)" },
],
};
@@ -0,0 +1,86 @@
import { useEffect, type RefObject } from "react";
interface Options {
/** Whether the dropdown is currently open. */
open: boolean;
/** Invoked when an outside interaction should close the dropdown. */
onClose: () => void;
/**
* Refs whose DOM subtree is considered "inside". Events originating inside
* any of these refs do NOT trigger close. Typically [triggerRef, dropdownRef].
*/
ignore: Array<RefObject<HTMLElement | null>>;
/**
* Close when the page scrolls OUTSIDE the dropdown. Scrolls that originate
* inside the dropdown's own scrollable list are ignored — otherwise touch-
* scrolling a long list would close the popup. Default: true.
*/
closeOnOutsideScroll?: boolean;
/** Close on window resize (position would be stale). Default: true. */
closeOnResize?: boolean;
/** Close on Escape key. Default: true. */
closeOnEscape?: boolean;
}
/**
* Standard outside-close behavior for portaled dropdowns. Mirrors the web
* implementation at ui/web/src/hooks/use-portal-dropdown-close.ts.
*
* - `pointerdown` unifies mouse + touch + pen.
* - `composedPath()` beats `Node.contains()` for portal subtrees.
* - Listener install is deferred one tick so the click that OPENED the
* dropdown cannot be re-delivered and immediately close it (touch bug).
* - Scroll listener ignores scrolls inside the dropdown (lets long lists
* scroll without self-closing).
*/
export function usePortalDropdownClose({
open,
onClose,
ignore,
closeOnOutsideScroll = true,
closeOnResize = true,
closeOnEscape = true,
}: Options) {
useEffect(() => {
if (!open) return;
const isInside = (event: Event): boolean => {
const nodes = ignore
.map((r) => r.current)
.filter((n): n is HTMLElement => n != null);
const path = typeof event.composedPath === "function" ? event.composedPath() : [];
if (path.length > 0) {
return path.some((n) => nodes.includes(n as HTMLElement));
}
const target = event.target as Node | null;
if (!target) return false;
return nodes.some((n) => n.contains(target));
};
const handlePointerDown = (event: PointerEvent) => {
if (!isInside(event)) onClose();
};
const handleEscape = (event: KeyboardEvent) => {
if (event.key === "Escape") onClose();
};
const handleScroll = (event: Event) => {
if (!isInside(event)) onClose();
};
const handleResize = () => onClose();
const installId = window.setTimeout(() => {
document.addEventListener("pointerdown", handlePointerDown);
if (closeOnEscape) document.addEventListener("keydown", handleEscape);
if (closeOnOutsideScroll) window.addEventListener("scroll", handleScroll, true);
if (closeOnResize) window.addEventListener("resize", handleResize);
}, 0);
return () => {
window.clearTimeout(installId);
document.removeEventListener("pointerdown", handlePointerDown);
document.removeEventListener("keydown", handleEscape);
window.removeEventListener("scroll", handleScroll, true);
window.removeEventListener("resize", handleResize);
};
}, [open, onClose, ignore, closeOnOutsideScroll, closeOnResize, closeOnEscape]);
}
@@ -0,0 +1,39 @@
/**
* Desktop hook for fetching TTS provider capabilities.
* Mirrors useTtsCapabilities from ui/web/src/api/tts-capabilities.ts
* but uses the desktop ApiClient pattern (no React Query).
*/
import { useState, useEffect, useCallback } from 'react'
import { getApiClient } from '../lib/api'
import { parseCapabilitiesResponse } from '../api/tts-capabilities'
import type { ProviderCapabilities } from '../api/tts-capabilities'
interface UseTtsCapabilitiesResult {
data: ProviderCapabilities[]
isLoading: boolean
error: string | null
}
export function useTtsCapabilities(): UseTtsCapabilitiesResult {
const [data, setData] = useState<ProviderCapabilities[]>([])
const [isLoading, setIsLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const fetch = useCallback(async () => {
setIsLoading(true)
setError(null)
try {
const res = await getApiClient().get<unknown>('/v1/tts/capabilities')
setData(parseCapabilitiesResponse(res))
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to load capabilities')
setData([])
} finally {
setIsLoading(false)
}
}, [])
useEffect(() => { void fetch() }, [fetch])
return { data, isLoading, error }
}
@@ -24,7 +24,8 @@
"openai": "OpenAI",
"elevenlabs": "ElevenLabs",
"edge": "Edge (Free)",
"minimax": "MiniMax"
"minimax": "MiniMax",
"gemini": "Google Gemini"
},
"autoModes": {
"off": "Off",
@@ -47,18 +48,80 @@
"apiKey": "API Key",
"apiBase": "API Base URL",
"model": "Model",
"voice": "Voice"
"voice": "Voice",
"speed": {
"label": "Speed",
"help": "Speech speed multiplier — 0.25 = slowest, 4.0 = fastest."
},
"responseFormat": {
"label": "Output Format",
"help": "Audio encoding returned by the API (mp3, opus, aac, flac, wav, pcm)."
},
"instructions": {
"label": "Instructions",
"help": "System-level style prompt — only supported on gpt-4o-mini-tts model."
}
},
"elevenlabs": {
"apiKey": "API Key",
"baseUrl": "Base URL",
"voiceId": "Voice ID",
"modelId": "Model ID"
"modelId": "Model ID",
"voiceSettings": {
"stability": {
"label": "Stability",
"help": "0 = more expressive variation, 1 = more consistent and stable."
},
"similarityBoost": {
"label": "Similarity Boost",
"help": "How closely output matches the original voice clone (0 = creative, 1 = similar)."
},
"style": {
"label": "Style Exaggeration",
"help": "Amplifies voice style. 0 = none; higher values add more expressiveness."
},
"useSpeakerBoost": {
"label": "Speaker Boost",
"help": "Enhances voice clarity and target speaker similarity."
},
"speed": {
"label": "Speed",
"help": "Speech speed — range 0.7 (slow) to 1.2 (fast)."
}
},
"applyTextNormalization": {
"label": "Text Normalization",
"help": "How numbers and abbreviations are expanded (auto, on, or off)."
},
"seed": {
"label": "Seed (Advanced)",
"help": "Set a fixed seed for reproducible output. 0 = random."
},
"optimizeStreamingLatency": {
"label": "Streaming Latency (Advanced)",
"help": "Latency optimization level: 0 = none, 4 = maximum."
},
"languageCode": {
"label": "Language Code",
"help": "ISO 639-1 language hint, e.g. 'en' or 'vi'. Advanced."
}
},
"edge": {
"voice": "Voice",
"speechRate": "Speech Rate",
"hint": "Edge TTS is free and requires no API key. Install with: pip install edge-tts"
"hint": "Edge TTS is free and requires no API key. Install with: pip install edge-tts",
"rate": {
"label": "Speech Rate",
"help": "Rate adjustment in percent. -50% = much slower, +100% = much faster."
},
"pitch": {
"label": "Pitch",
"help": "Pitch shift in Hz relative to neutral. Range: -50 Hz to +50 Hz."
},
"volume": {
"label": "Volume",
"help": "Volume adjustment in percent. -50% = quieter, +100% = louder."
}
},
"minimax": {
"apiKey": "API Key",
@@ -66,12 +129,91 @@
"groupIdPlaceholder": "Required for MiniMax",
"apiBase": "API Base",
"model": "Model",
"voiceId": "Voice ID"
"voiceId": "Voice ID",
"speed": {
"label": "Speed",
"help": "Speech speed multiplier — 0.5 = slowest, 2.0 = fastest."
},
"vol": {
"label": "Volume",
"help": "Volume multiplier — 0.01 = near-silent, 10.0 = loudest."
},
"pitch": {
"label": "Pitch",
"help": "Pitch shift in semitones. -12 = one octave down, +12 = one octave up."
},
"emotion": {
"label": "Emotion",
"help": "Emotional tone applied to the synthesized voice."
},
"textNormalization": {
"label": "Text Normalization",
"help": "Whether numbers and symbols are expanded before synthesis."
},
"audio": {
"format": {
"label": "Audio Format",
"help": "Output encoding: mp3, pcm, flac, or wav."
},
"sampleRate": {
"label": "Sample Rate",
"help": "Audio sample rate in Hz. Leave as Default for most use cases."
},
"bitrate": {
"label": "Bitrate (MP3 only)",
"help": "MP3 bit rate in bps. Only applies when format is mp3."
},
"channel": {
"label": "Channels",
"help": "1 = mono (smaller file), 2 = stereo."
}
}
},
"gemini": {
"apiKey": "API Key",
"apiBase": "API Base URL",
"voice": "Voice",
"model": "Model",
"multiSpeaker": "Multi-Speaker",
"multiSpeakerHint": "Up to 2 speakers supported",
"audioTags": "Audio Tags",
"audioTagsHint": "Insert expressive tags like [laughs] directly in your text",
"speakerName": "Speaker Name",
"speakerVoice": "Speaker Voice",
"addSpeaker": "Add Speaker",
"removeSpeaker": "Remove",
"maxSpeakersReached": "Maximum 2 speakers reached",
"filterAll": "All",
"filterEmotion": "Emotion",
"filterPacing": "Pacing",
"filterEffect": "Effect",
"filterVoiceQuality": "Voice Quality",
"searchPlaceholder": "Search tags…",
"insertTag": "Insert",
"multiSpeakerMode": {
"title": "Multi-Speaker Mode",
"addSpeaker": "Add speaker",
"speakerName": "Speaker name",
"speakerVoice": "Voice",
"removeSpeaker": "Remove",
"limitReached": "Maximum 2 speakers"
},
"audioTagsPanel": {
"title": "Audio Tags",
"search": "Search tags",
"categoryEmotion": "Emotion",
"categoryPacing": "Pacing",
"categoryEffect": "Effect",
"categoryVoiceQuality": "Voice quality",
"insertHint": "Click a tag to insert at caret"
},
"previewBadge": "Preview"
},
"voice_label": "Voice",
"voice_placeholder": "Select a voice…",
"voice_picker": {
"requires_provider": "Configure TTS provider first"
"requires_provider": "Configure TTS provider first",
"enter_voice_id": "Enter voice_id manually"
},
"voice_refresh": "Refresh",
"voice_preview": "Preview",
@@ -9,7 +9,7 @@
"configured": "Đã cấu hình",
"disabled": "Đã tắt",
"noProvider": "Chưa cấu hình provider TTS",
"activeProvider": "Provider đang hoạt động: {{provider}}, Tự động: {{auto}}"
"activeProvider": "Provider chính: {{provider}}, Tự động: {{auto}}"
},
"general": {
"title": "Cài đặt chung",
@@ -24,7 +24,8 @@
"openai": "OpenAI",
"elevenlabs": "ElevenLabs",
"edge": "Edge (Miễn phí)",
"minimax": "MiniMax"
"minimax": "MiniMax",
"gemini": "Google Gemini"
},
"autoModes": {
"off": "Tắt",
@@ -47,18 +48,80 @@
"apiKey": "API Key",
"apiBase": "API Base URL",
"model": "Model",
"voice": "Giọng nói"
"voice": "Giọng nói",
"speed": {
"label": "Tốc độ",
"help": "Hệ số tốc độ nói — 0.25 = chậm nhất, 4.0 = nhanh nhất."
},
"responseFormat": {
"label": "Định dạng đầu ra",
"help": "Định dạng mã hoá âm thanh trả về từ API (mp3, opus, aac, flac, wav, pcm)."
},
"instructions": {
"label": "Hướng dẫn",
"help": "Prompt phong cách hệ thống — chỉ hỗ trợ với model gpt-4o-mini-tts."
}
},
"elevenlabs": {
"apiKey": "API Key",
"baseUrl": "Base URL",
"voiceId": "Voice ID",
"modelId": "Model ID"
"modelId": "Model ID",
"voiceSettings": {
"stability": {
"label": "Độ ổn định",
"help": "0 = biến đổi nhiều hơn, 1 = nhất quán và ổn định hơn."
},
"similarityBoost": {
"label": "Độ tương đồng",
"help": "Mức độ bám sát giọng gốc — 0 = sáng tạo hơn, 1 = giống gốc hơn."
},
"style": {
"label": "Phóng đại phong cách",
"help": "Khuếch đại phong cách giọng. 0 = không; giá trị cao hơn = biểu cảm hơn."
},
"useSpeakerBoost": {
"label": "Tăng cường giọng",
"help": "Cải thiện độ rõ ràng và mức độ giống giọng đích."
},
"speed": {
"label": "Tốc độ",
"help": "Tốc độ nói — từ 0.7 (chậm) đến 1.2 (nhanh)."
}
},
"applyTextNormalization": {
"label": "Chuẩn hoá văn bản",
"help": "Cách xử lý số và viết tắt trước khi tổng hợp (auto, on hoặc off)."
},
"seed": {
"label": "Seed (Nâng cao)",
"help": "Đặt seed cố định để tái tạo kết quả giống nhau. 0 = ngẫu nhiên."
},
"optimizeStreamingLatency": {
"label": "Độ trễ streaming (Nâng cao)",
"help": "Mức tối ưu hoá độ trễ: 0 = không tối ưu, 4 = tối ưu tối đa."
},
"languageCode": {
"label": "Mã ngôn ngữ",
"help": "Gợi ý ngôn ngữ ISO 639-1, ví dụ: 'en' hoặc 'vi'. Nâng cao."
}
},
"edge": {
"voice": "Giọng nói",
"speechRate": "Tốc độ nói",
"hint": "Edge TTS miễn phí, không cần API key. Cài đặt: pip install edge-tts"
"hint": "Edge TTS miễn phí, không cần API key. Cài đặt: pip install edge-tts",
"rate": {
"label": "Tốc độ nói",
"help": "Điều chỉnh tốc độ theo phần trăm. -50% = chậm hơn nhiều, +100% = nhanh hơn nhiều."
},
"pitch": {
"label": "Cao độ",
"help": "Dịch chuyển cao độ theo Hz so với mức trung tính. Từ -50 Hz đến +50 Hz."
},
"volume": {
"label": "Âm lượng",
"help": "Điều chỉnh âm lượng theo phần trăm. -50% = nhỏ hơn, +100% = to hơn."
}
},
"minimax": {
"apiKey": "API Key",
@@ -66,12 +129,91 @@
"groupIdPlaceholder": "Bắt buộc cho MiniMax",
"apiBase": "API Base",
"model": "Model",
"voiceId": "Voice ID"
"voiceId": "Voice ID",
"speed": {
"label": "Tốc độ",
"help": "Hệ số tốc độ nói — 0.5 = chậm nhất, 2.0 = nhanh nhất."
},
"vol": {
"label": "Âm lượng",
"help": "Hệ số âm lượng — 0.01 = gần như không nghe được, 10.0 = to nhất."
},
"pitch": {
"label": "Cao độ",
"help": "Dịch chuyển cao độ theo nửa cung. -12 = thấp hơn một quãng tám, +12 = cao hơn một quãng tám."
},
"emotion": {
"label": "Cảm xúc",
"help": "Sắc thái cảm xúc áp dụng cho giọng đọc tổng hợp."
},
"textNormalization": {
"label": "Chuẩn hoá văn bản",
"help": "Có mở rộng số và ký hiệu trước khi tổng hợp hay không."
},
"audio": {
"format": {
"label": "Định dạng âm thanh",
"help": "Định dạng đầu ra: mp3, pcm, flac hoặc wav."
},
"sampleRate": {
"label": "Tần số lấy mẫu",
"help": "Tần số lấy mẫu tính bằng Hz. Để mặc định cho hầu hết trường hợp."
},
"bitrate": {
"label": "Bitrate (chỉ MP3)",
"help": "Tốc độ bit MP3 tính bằng bps. Chỉ áp dụng khi định dạng là mp3."
},
"channel": {
"label": "Kênh âm thanh",
"help": "1 = mono (file nhỏ hơn), 2 = stereo."
}
}
},
"gemini": {
"apiKey": "API Key",
"apiBase": "API Base URL",
"voice": "Giọng đọc",
"model": "Mô hình",
"multiSpeaker": "Nhiều người nói",
"multiSpeakerHint": "Hỗ trợ tối đa 2 người nói",
"audioTags": "Thẻ âm thanh",
"audioTagsHint": "Chèn thẻ biểu cảm như [laughs] trực tiếp vào văn bản",
"speakerName": "Tên người nói",
"speakerVoice": "Giọng đọc",
"addSpeaker": "Thêm người nói",
"removeSpeaker": "Xóa",
"maxSpeakersReached": "Đã đạt tối đa 2 người nói",
"filterAll": "Tất cả",
"filterEmotion": "Cảm xúc",
"filterPacing": "Nhịp độ",
"filterEffect": "Hiệu ứng",
"filterVoiceQuality": "Chất lượng giọng",
"searchPlaceholder": "Tìm thẻ…",
"insertTag": "Chèn",
"multiSpeakerMode": {
"title": "Chế độ nhiều người nói",
"addSpeaker": "Thêm người nói",
"speakerName": "Tên người nói",
"speakerVoice": "Giọng đọc",
"removeSpeaker": "Xóa",
"limitReached": "Tối đa 2 người nói"
},
"audioTagsPanel": {
"title": "Thẻ âm thanh",
"search": "Tìm thẻ",
"categoryEmotion": "Cảm xúc",
"categoryPacing": "Nhịp độ",
"categoryEffect": "Hiệu ứng",
"categoryVoiceQuality": "Chất lượng giọng",
"insertHint": "Nhấp vào thẻ để chèn tại vị trí con trỏ"
},
"previewBadge": "Xem trước"
},
"voice_label": "Giọng đọc",
"voice_placeholder": "Chọn giọng đọc…",
"voice_picker": {
"requires_provider": "Vui lòng cấu hình provider TTS trước"
"requires_provider": "Vui lòng cấu hình provider TTS trước",
"enter_voice_id": "Nhập voice_id thủ công"
},
"voice_refresh": "Làm mới",
"voice_preview": "Nghe thử",
@@ -1,6 +1,6 @@
{
"title": "语音合成",
"description": "配置 TTS Provider和自动应用设置",
"description": "配置 TTS Provider 和自动应用设置",
"save": "保存",
"saving": "保存中...",
"saveChanges": "保存更改",
@@ -24,7 +24,8 @@
"openai": "OpenAI",
"elevenlabs": "ElevenLabs",
"edge": "Edge(免费)",
"minimax": "MiniMax"
"minimax": "MiniMax",
"gemini": "Google Gemini"
},
"autoModes": {
"off": "关闭",
@@ -47,18 +48,80 @@
"apiKey": "API Key",
"apiBase": "API Base URL",
"model": "模型",
"voice": "语音"
"voice": "语音",
"speed": {
"label": "语速",
"help": "语速倍数 — 0.25 = 最慢,4.0 = 最快。"
},
"responseFormat": {
"label": "输出格式",
"help": "API 返回的音频编码格式(mp3、opus、aac、flac、wav、pcm)。"
},
"instructions": {
"label": "指令",
"help": "系统级风格提示词 — 仅 gpt-4o-mini-tts 模型支持。"
}
},
"elevenlabs": {
"apiKey": "API Key",
"baseUrl": "Base URL",
"voiceId": "Voice ID",
"modelId": "Model ID"
"modelId": "Model ID",
"voiceSettings": {
"stability": {
"label": "稳定性",
"help": "0 = 变化更多、更富表现力,1 = 更一致、更稳定。"
},
"similarityBoost": {
"label": "相似度增强",
"help": "与原始声音克隆的接近程度 — 0 = 更有创意,1 = 更相似。"
},
"style": {
"label": "风格夸张度",
"help": "放大声音风格特征。0 = 无;值越高表现力越强。"
},
"useSpeakerBoost": {
"label": "发言人增强",
"help": "提升声音清晰度和目标发言人相似度。"
},
"speed": {
"label": "语速",
"help": "语速范围 — 0.7(慢)到 1.2(快)。"
}
},
"applyTextNormalization": {
"label": "文本规范化",
"help": "数字和缩写词的展开方式(auto、on 或 off)。"
},
"seed": {
"label": "随机种子(高级)",
"help": "设置固定种子以获得可复现的输出。0 = 随机。"
},
"optimizeStreamingLatency": {
"label": "流式延迟优化(高级)",
"help": "延迟优化级别:0 = 不优化,4 = 最大优化。"
},
"languageCode": {
"label": "语言代码",
"help": "ISO 639-1 语言提示,如 'en' 或 'zh'。高级选项。"
}
},
"edge": {
"voice": "语音",
"speechRate": "语速",
"hint": "Edge TTS 免费且无需 API Key。安装方式:pip install edge-tts"
"hint": "Edge TTS 免费且无需 API Key。安装方式:pip install edge-tts",
"rate": {
"label": "语速",
"help": "语速百分比调整。-50% = 慢很多,+100% = 快很多。"
},
"pitch": {
"label": "音调",
"help": "相对于中性音调的 Hz 偏移。范围:-50 Hz 至 +50 Hz。"
},
"volume": {
"label": "音量",
"help": "音量百分比调整。-50% = 更小声,+100% = 更响。"
}
},
"minimax": {
"apiKey": "API Key",
@@ -66,12 +129,91 @@
"groupIdPlaceholder": "MiniMax 必填",
"apiBase": "API Base",
"model": "模型",
"voiceId": "Voice ID"
"voiceId": "Voice ID",
"speed": {
"label": "语速",
"help": "语速倍数 — 0.5 = 最慢,2.0 = 最快。"
},
"vol": {
"label": "音量",
"help": "音量倍数 — 0.01 = 近乎静音,10.0 = 最大音量。"
},
"pitch": {
"label": "音调",
"help": "音调偏移(半音)。-12 = 低一个八度,+12 = 高一个八度。"
},
"emotion": {
"label": "情感",
"help": "应用于合成语音的情感色彩。"
},
"textNormalization": {
"label": "文本规范化",
"help": "是否在合成前展开数字和符号。"
},
"audio": {
"format": {
"label": "音频格式",
"help": "输出编码格式:mp3、pcm、flac 或 wav。"
},
"sampleRate": {
"label": "采样率",
"help": "音频采样率(Hz)。大多数场景保持默认即可。"
},
"bitrate": {
"label": "比特率(仅 MP3)",
"help": "MP3 比特率(bps)。仅当格式为 mp3 时生效。"
},
"channel": {
"label": "声道",
"help": "1 = 单声道(文件较小),2 = 立体声。"
}
}
},
"gemini": {
"apiKey": "API Key",
"apiBase": "API Base URL",
"voice": "语音",
"model": "模型",
"multiSpeaker": "多人说话",
"multiSpeakerHint": "最多支持 2 位发言人",
"audioTags": "音频标签",
"audioTagsHint": "在文本中直接插入 [laughs] 等表情标签",
"speakerName": "发言人名称",
"speakerVoice": "语音",
"addSpeaker": "添加发言人",
"removeSpeaker": "删除",
"maxSpeakersReached": "已达到最多 2 位发言人",
"filterAll": "全部",
"filterEmotion": "情感",
"filterPacing": "节奏",
"filterEffect": "效果",
"filterVoiceQuality": "语音质量",
"searchPlaceholder": "搜索标签…",
"insertTag": "插入",
"multiSpeakerMode": {
"title": "多人说话模式",
"addSpeaker": "添加发言人",
"speakerName": "发言人名称",
"speakerVoice": "语音",
"removeSpeaker": "删除",
"limitReached": "最多 2 位发言人"
},
"audioTagsPanel": {
"title": "音频标签",
"search": "搜索标签",
"categoryEmotion": "情感",
"categoryPacing": "节奏",
"categoryEffect": "效果",
"categoryVoiceQuality": "语音质量",
"insertHint": "点击标签插入到光标位置"
},
"previewBadge": "预览版"
},
"voice_label": "声音",
"voice_placeholder": "选择声音…",
"voice_picker": {
"requires_provider": "请先配置 TTS Provider"
"requires_provider": "请先配置 TTS Provider",
"enter_voice_id": "手动输入 voice_id"
},
"voice_refresh": "刷新",
"voice_preview": "试听",
@@ -0,0 +1,122 @@
/**
* i18n TTS key parity tests.
*
* Ensures every leaf key present in the English (EN) tts.json exists in the
* Vietnamese (VI) and Simplified Chinese (ZH) translations. Prevents silent
* fallback to English in production caused by missing keys.
*
* Also covers desktop locale files — they have a distinct keyset from web but
* must themselves be internally consistent across EN/VI/ZH.
*/
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { resolve, dirname } from "path";
import { fileURLToPath } from "url";
const __dirname = dirname(fileURLToPath(import.meta.url));
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
type NestedRecord = { [key: string]: string | NestedRecord };
function loadLocale(relPath: string): NestedRecord {
const abs = resolve(__dirname, relPath);
return JSON.parse(readFileSync(abs, "utf-8")) as NestedRecord;
}
/**
* Recursively collect all leaf key paths from a nested object.
* Returns dot-separated paths, e.g. "openai.speed.label".
*/
function collectLeafPaths(obj: NestedRecord, prefix = ""): string[] {
const paths: string[] = [];
for (const [key, value] of Object.entries(obj)) {
const path = prefix ? `${prefix}.${key}` : key;
if (typeof value === "string") {
paths.push(path);
} else if (typeof value === "object" && value !== null) {
paths.push(...collectLeafPaths(value as NestedRecord, path));
}
}
return paths;
}
/**
* Resolves a dot-separated path in a nested object.
*/
function getByPath(obj: NestedRecord, path: string): unknown {
return path.split(".").reduce<unknown>((acc, segment) => {
if (acc !== null && typeof acc === "object") {
return (acc as NestedRecord)[segment];
}
return undefined;
}, obj);
}
/**
* Asserts every leaf key from `source` exists (as a string) in `target`.
* Reports all missing keys at once.
*/
function assertKeyParity(
source: NestedRecord,
target: NestedRecord,
targetLabel: string,
): void {
const leafPaths = collectLeafPaths(source);
const missing: string[] = [];
for (const path of leafPaths) {
const value = getByPath(target, path);
if (typeof value !== "string") {
missing.push(path);
}
}
expect(missing, `${targetLabel} is missing keys`).toEqual([]);
}
// ---------------------------------------------------------------------------
// Locale paths — relative to this test file (src/__tests__/)
// ---------------------------------------------------------------------------
const WEB_LOCALES = "../i18n/locales";
// ui/desktop/frontend: from ui/web/src/__tests__ go up 3 dirs to ui/, then into desktop/frontend
const DESKTOP_LOCALES = "../../../desktop/frontend/src/i18n/locales";
// ---------------------------------------------------------------------------
// Web locale parity
// ---------------------------------------------------------------------------
describe("Web TTS i18n parity", () => {
const webEN = loadLocale(`${WEB_LOCALES}/en/tts.json`);
const webVI = loadLocale(`${WEB_LOCALES}/vi/tts.json`);
const webZH = loadLocale(`${WEB_LOCALES}/zh/tts.json`);
it("VI contains every key present in EN", () => {
assertKeyParity(webEN, webVI, "web vi/tts.json");
});
it("ZH contains every key present in EN", () => {
assertKeyParity(webEN, webZH, "web zh/tts.json");
});
});
// ---------------------------------------------------------------------------
// Desktop locale parity
// ---------------------------------------------------------------------------
describe("Desktop TTS i18n parity", () => {
const desktopEN = loadLocale(`${DESKTOP_LOCALES}/en/tts.json`);
const desktopVI = loadLocale(`${DESKTOP_LOCALES}/vi/tts.json`);
const desktopZH = loadLocale(`${DESKTOP_LOCALES}/zh/tts.json`);
it("VI contains every key present in EN", () => {
assertKeyParity(desktopEN, desktopVI, "desktop vi/tts.json");
});
it("ZH contains every key present in EN", () => {
assertKeyParity(desktopEN, desktopZH, "desktop zh/tts.json");
});
});
@@ -0,0 +1,93 @@
import { describe, it, expect } from "vitest";
import {
parseCapabilitiesResponse,
buildCapabilitiesQueryOptions,
type ProviderCapabilities,
} from "../tts-capabilities";
// ---- parseCapabilitiesResponse ----
describe("parseCapabilitiesResponse", () => {
it("parses a valid response into typed array", () => {
const json = {
providers: [
{
provider: "openai",
display_name: "OpenAI TTS",
requires_api_key: true,
models: ["tts-1", "tts-1-hd"],
voices: [],
params: null,
custom_features: null,
},
],
};
const result = parseCapabilitiesResponse(json);
expect(result).toHaveLength(1);
expect(result[0]!.provider).toBe("openai");
expect(result[0]!.display_name).toBe("OpenAI TTS");
expect(result[0]!.requires_api_key).toBe(true);
expect(result[0]!.models).toEqual(["tts-1", "tts-1-hd"]);
});
it("returns empty array when providers field is empty", () => {
const result = parseCapabilitiesResponse({ providers: [] });
expect(result).toEqual([]);
});
it("throws on malformed input — missing providers key", () => {
expect(() => parseCapabilitiesResponse({})).toThrow();
});
it("throws on null input", () => {
expect(() => parseCapabilitiesResponse(null)).toThrow();
});
it("throws on non-array providers", () => {
expect(() => parseCapabilitiesResponse({ providers: "bad" })).toThrow();
});
});
// ---- buildCapabilitiesQueryOptions ----
describe("buildCapabilitiesQueryOptions", () => {
it("has cache key [tts, capabilities]", () => {
const opts = buildCapabilitiesQueryOptions(() => Promise.resolve({ providers: [] as ProviderCapabilities[] }));
expect(opts.queryKey).toEqual(["tts", "capabilities"]);
});
it("has staleTime of 5 minutes (300000 ms)", () => {
const opts = buildCapabilitiesQueryOptions(() => Promise.resolve({ providers: [] as ProviderCapabilities[] }));
expect(opts.staleTime).toBe(5 * 60_000);
});
it("queryFn resolves to parsed providers on success", async () => {
const fetcher = () =>
Promise.resolve({
providers: [
{
provider: "edge",
display_name: "Edge TTS",
requires_api_key: false,
models: [],
voices: [],
params: null,
custom_features: null,
} satisfies ProviderCapabilities,
],
});
const opts = buildCapabilitiesQueryOptions(fetcher);
// queryFn signature accepts a QueryFunctionContext; pass a minimal stub.
const result = await opts.queryFn({ signal: new AbortController().signal } as never);
expect(result).toHaveLength(1);
expect(result[0]!.provider).toBe("edge");
});
it("error fallback returns empty providers array on fetch failure", async () => {
const fetcher = () => Promise.reject(new Error("network error"));
const opts = buildCapabilitiesQueryOptions(fetcher);
// The queryFn should NOT throw — it catches errors and returns []
const result = await opts.queryFn({ signal: new AbortController().signal } as never);
expect(result).toEqual([]);
});
});
+125
View File
@@ -0,0 +1,125 @@
import { useQuery } from "@tanstack/react-query";
import { useHttp } from "@/hooks/use-ws";
// ---- Types ----
export interface EnumOption {
value: string;
label: string;
}
export interface Dependency {
field: string;
op: string;
value: unknown;
}
export type ParamType =
| "range"
| "number"
| "integer"
| "enum"
| "boolean"
| "string"
| "text";
export interface ParamSchema {
key: string;
type: ParamType;
label: string;
description?: string;
default?: unknown;
min?: number;
max?: number;
step?: number;
enum?: EnumOption[];
depends_on?: Dependency[];
}
export interface VoiceOption {
voice_id: string;
name: string;
language?: string;
gender?: string;
}
export interface ProviderCapabilities {
provider: string;
display_name: string;
requires_api_key: boolean;
models?: string[];
voices?: VoiceOption[];
params?: ParamSchema[] | null;
custom_features?: Record<string, unknown> | null;
}
interface CapabilitiesResponse {
providers: ProviderCapabilities[];
}
// ---- parseCapabilitiesResponse ----
/**
* Parses and validates a raw GET /v1/tts/capabilities JSON response.
* Throws if the shape is invalid.
*/
export function parseCapabilitiesResponse(raw: unknown): ProviderCapabilities[] {
if (raw === null || raw === undefined || typeof raw !== "object") {
throw new Error("capabilities response must be an object");
}
const obj = raw as Record<string, unknown>;
if (!("providers" in obj)) {
throw new Error("capabilities response missing 'providers' field");
}
if (!Array.isArray(obj.providers)) {
throw new Error("capabilities response 'providers' must be an array");
}
return obj.providers as ProviderCapabilities[];
}
// ---- buildCapabilitiesQueryOptions ----
type CapabilitiesFetcher = () => Promise<CapabilitiesResponse>;
/**
* Builds React Query options for the TTS capabilities query.
* The queryFn catches errors and returns an empty providers array as fallback.
*/
export function buildCapabilitiesQueryOptions(fetcher: CapabilitiesFetcher) {
return {
queryKey: ["tts", "capabilities"] as const,
staleTime: 5 * 60_000, // 5 minutes — catalog data changes rarely
queryFn: async (_ctx: { signal: AbortSignal }): Promise<ProviderCapabilities[]> => {
try {
const resp = await fetcher();
return parseCapabilitiesResponse(resp);
} catch {
return [];
}
},
};
}
// ---- useTtsCapabilities hook ----
export const ttsCapabilitiesKeys = {
all: ["tts", "capabilities"] as const,
};
/** React Query hook for fetching TTS provider capabilities. */
export function useTtsCapabilities() {
const http = useHttp();
return useQuery({
queryKey: ttsCapabilitiesKeys.all,
queryFn: async (): Promise<ProviderCapabilities[]> => {
try {
const resp = await http.get<CapabilitiesResponse>("/v1/tts/capabilities");
return parseCapabilitiesResponse(resp);
} catch {
return [];
}
},
staleTime: 5 * 60_000,
retry: 1,
});
}
@@ -0,0 +1,67 @@
import { describe, it, expect } from "vitest";
import {
filterTagsByCategory,
searchTags,
insertTagAtCaret,
GEMINI_AUDIO_TAGS,
} from "../../data/gemini-audio-tags";
describe("filterTagsByCategory", () => {
it("returns only emotion entries when category='emotion'", () => {
const results = filterTagsByCategory("emotion");
expect(results.length).toBeGreaterThan(0);
results.forEach((t) => expect(t.category).toBe("emotion"));
});
it("returns all tags when no category provided", () => {
expect(filterTagsByCategory()).toHaveLength(GEMINI_AUDIO_TAGS.length);
});
});
describe("searchTags", () => {
it("case-insensitive search for LAUGH returns laugh-related tags", () => {
const results = searchTags("LAUGH");
expect(results.length).toBeGreaterThan(0);
results.forEach((t) =>
expect(
t.tag.toLowerCase().includes("laugh") ||
t.description.toLowerCase().includes("laugh")
).toBe(true)
);
});
it("search 'whisper' returns whisper tags", () => {
const results = searchTags("whisper");
expect(results.length).toBeGreaterThan(0);
});
});
describe("insertTagAtCaret", () => {
it("inserts tag at caret position in middle of text", () => {
const text = "Hello world";
const caretPos = 5; // after "Hello"
const result = insertTagAtCaret(text, "[pause]", caretPos);
expect(result).toContain("[pause]");
expect(result.startsWith("Hello")).toBe(true);
});
it("inserts at position 0 without leading space", () => {
const result = insertTagAtCaret("world", "[pause]", 0);
expect(result.startsWith("[pause]")).toBe(true);
});
it("inserts at end without trailing space", () => {
const result = insertTagAtCaret("Hello", "[pause]", 5);
expect(result.endsWith("[pause]")).toBe(true);
});
});
describe("readonly guard (pure logic)", () => {
it("insertTagAtCaret when readonly should be caller responsibility — returns new text regardless", () => {
// The readonly guard is enforced at the component level (onInsert not called).
// Pure function always returns modified text; component skips calling it.
const text = "Hello world";
const result = insertTagAtCaret(text, "[pause]", 5);
expect(result).not.toBe(text);
});
});
@@ -0,0 +1,182 @@
/**
* Pure-logic tests for DynamicParamForm utilities.
* NO @testing-library/react — tests isolate state-shape transforms only.
* Covers: evaluateDependsOn, initializeDefaults, applyNestedChange, rendererDispatch, edge cases.
*/
import { describe, it, expect, vi } from "vitest";
import { evaluateDependsOn } from "../dynamic-param-form";
import {
initializeDefaults,
applyNestedChange,
getRendererName,
coerceNumericValue,
} from "../dynamic-param-form-logic";
import type { ParamSchema } from "@/api/tts-capabilities";
// ---- evaluateDependsOn ----
describe("evaluateDependsOn", () => {
it("returns true when deps is undefined", () => {
expect(evaluateDependsOn(undefined, {})).toBe(true);
});
it("returns true when deps is empty array", () => {
expect(evaluateDependsOn([], {})).toBe(true);
});
it("returns true when all conditions match (AND semantics)", () => {
const deps = [
{ field: "model", op: "eq", value: "x" },
{ field: "format", op: "eq", value: "mp3" },
];
const state = { model: "x", format: "mp3" };
expect(evaluateDependsOn(deps, state)).toBe(true);
});
it("returns false when one condition fails", () => {
const deps = [
{ field: "model", op: "eq", value: "x" },
{ field: "format", op: "eq", value: "mp3" },
];
const state = { model: "x", format: "wav" };
expect(evaluateDependsOn(deps, state)).toBe(false);
});
it("coerces values to string for comparison", () => {
const deps = [{ field: "enabled", op: "eq", value: true }];
expect(evaluateDependsOn(deps, { enabled: "true" })).toBe(true);
expect(evaluateDependsOn(deps, { enabled: true })).toBe(true);
expect(evaluateDependsOn(deps, { enabled: false })).toBe(false);
});
});
// ---- initializeDefaults ----
describe("initializeDefaults", () => {
it("sets Default for each schema key", () => {
const schemas: ParamSchema[] = [
{ key: "speed", type: "range", label: "Speed", default: 1.0 },
{ key: "format", type: "enum", label: "Format", default: "mp3" },
];
const out = initializeDefaults(schemas);
expect(out).toEqual({ speed: 1.0, format: "mp3" });
});
it("handles nested-key schemas (dot notation)", () => {
const schemas: ParamSchema[] = [
{ key: "voice_settings.stability", type: "range", label: "Stability", default: 0.5 },
{ key: "voice_settings.similarity_boost", type: "range", label: "Similarity", default: 0.75 },
];
const out = initializeDefaults(schemas);
expect(out).toEqual({
"voice_settings.stability": 0.5,
"voice_settings.similarity_boost": 0.75,
});
});
it("uses empty string when no default present", () => {
const schemas: ParamSchema[] = [
{ key: "instructions", type: "text", label: "Instructions" },
];
const out = initializeDefaults(schemas);
expect(out["instructions"]).toBe("");
});
it("handles empty schema array", () => {
expect(initializeDefaults([])).toEqual({});
});
});
// ---- applyNestedChange ----
describe("applyNestedChange", () => {
it("sets flat key", () => {
const result = applyNestedChange({}, "speed", 1.5);
expect(result).toEqual({ speed: 1.5 });
});
it("preserves other keys", () => {
const result = applyNestedChange({ speed: 1.0, format: "mp3" }, "speed", 1.5);
expect(result).toEqual({ speed: 1.5, format: "mp3" });
});
it("sets nested key (dot notation)", () => {
const result = applyNestedChange({}, "voice_settings.stability", 0.3);
expect(result).toEqual({ "voice_settings.stability": 0.3 });
});
it("does not mutate the original object", () => {
const original = { speed: 1.0 };
const result = applyNestedChange(original, "speed", 2.0);
expect(original.speed).toBe(1.0);
expect(result.speed).toBe(2.0);
});
});
// ---- getRendererName ----
describe("getRendererName", () => {
it("returns renderer name for each ParamType", () => {
const types = ["range", "number", "integer", "enum", "boolean", "string", "text"] as const;
for (const t of types) {
expect(getRendererName(t)).toBe(t);
}
});
});
// ---- coerceNumericValue ----
describe("coerceNumericValue (NaN edge case)", () => {
it("returns default when value is NaN", () => {
expect(coerceNumericValue(NaN, 1.0)).toBe(1.0);
});
it("returns value when it is a valid number", () => {
expect(coerceNumericValue(1.5, 1.0)).toBe(1.5);
});
it("returns 0 when value is NaN and no default provided", () => {
expect(coerceNumericValue(NaN, undefined)).toBe(0);
});
});
// ---- readonly suppresses onChange ----
describe("readonly suppresses onChange (logic-level check)", () => {
it("does not call onChange when readonly is true", () => {
const handler = vi.fn();
// Simulate the readonly guard logic from DynamicParamForm
const callHandlerIfNotReadonly = (readonly: boolean, val: unknown) => {
if (!readonly && handler) handler(val);
};
callHandlerIfNotReadonly(true, "new-value");
expect(handler).not.toHaveBeenCalled();
});
it("calls onChange when readonly is false", () => {
const handler = vi.fn();
const callHandlerIfNotReadonly = (readonly: boolean, val: unknown) => {
if (!readonly && handler) handler(val);
};
callHandlerIfNotReadonly(false, "new-value");
expect(handler).toHaveBeenCalledWith("new-value");
});
});
// ---- empty enum edge case ----
describe("empty enum handling", () => {
it("empty enum options array does not crash coerce", () => {
const schema: ParamSchema = {
key: "format",
type: "enum",
label: "Format",
enum: [],
default: "",
};
// EnumField with empty options — should render disabled without crash
// Logic-level: verify defaults apply correctly
const defaults = initializeDefaults([schema]);
expect(defaults["format"]).toBe("");
});
});
@@ -0,0 +1,70 @@
import { describe, it, expect } from "vitest";
import {
addSpeaker,
removeSpeaker,
updateSpeakerVoice,
MAX_GEMINI_SPEAKERS,
} from "../multi-speaker-editor";
import type { SpeakerVoice } from "../multi-speaker-editor";
const makeSpeakers = (...names: string[]): SpeakerVoice[] =>
names.map((name, i) => ({ speaker: name, voiceId: `Voice${i}` }));
describe("addSpeaker", () => {
it("increments count from 1 to 2", () => {
const result = addSpeaker(makeSpeakers("Joe"), { speaker: "Jane", voiceId: "Puck" });
expect(result).toHaveLength(2);
expect(result[1]!.speaker).toBe("Jane");
});
it("enforces Gemini limit of 2 speakers", () => {
const full = makeSpeakers("Joe", "Jane");
const result = addSpeaker(full, { speaker: "Bob", voiceId: "Kore" });
expect(result).toHaveLength(MAX_GEMINI_SPEAKERS);
expect(result).toEqual(full); // unchanged
});
});
describe("removeSpeaker", () => {
it("removes speaker at index 1", () => {
const speakers = makeSpeakers("Joe", "Jane");
const result = removeSpeaker(speakers, 1);
expect(result).toHaveLength(1);
expect(result[0]!.speaker).toBe("Joe");
});
it("removes speaker at index 0", () => {
const speakers = makeSpeakers("Joe", "Jane");
const result = removeSpeaker(speakers, 0);
expect(result).toHaveLength(1);
expect(result[0]!.speaker).toBe("Jane");
});
});
describe("updateSpeakerVoice", () => {
it("updates voice for given index without mutating others", () => {
const speakers = makeSpeakers("Joe", "Jane");
const result = updateSpeakerVoice(speakers, 0, "NewVoice");
expect(result[0]!.voiceId).toBe("NewVoice");
expect(result[1]!.voiceId).toBe(speakers[1]!.voiceId); // unchanged
});
it("returns a new array (no mutation)", () => {
const speakers = makeSpeakers("Joe");
const result = updateSpeakerVoice(speakers, 0, "X");
expect(result).not.toBe(speakers);
});
});
describe("MAX_GEMINI_SPEAKERS constant", () => {
it("is 2", () => {
expect(MAX_GEMINI_SPEAKERS).toBe(2);
});
});
describe("readonly guard", () => {
it("addSpeaker returns unchanged array when already at limit (readonly equivalent)", () => {
const full = makeSpeakers("Joe", "Jane");
expect(addSpeaker(full, { speaker: "Bob", voiceId: "Kore" })).toEqual(full);
});
});
+124
View File
@@ -0,0 +1,124 @@
/**
* AudioTagPicker — browse and insert Gemini audio style tags.
* Tags are inline markers inserted at caret position in a target textarea.
* Supports readonly prop for desktop viewer mode.
*/
import { useState } from "react";
import { useTranslation } from "react-i18next";
import { cn } from "@/lib/utils";
import {
GEMINI_AUDIO_TAGS,
filterTagsByCategory,
searchTags,
insertTagAtCaret,
type AudioTag,
} from "../data/gemini-audio-tags";
type Category = AudioTag["category"] | "all";
interface AudioTagPickerProps {
/** Called when user clicks a tag. Passes the tag string to insert. */
onInsert?: (tag: string) => void;
/** Current caret position in the target textarea (for onInsert context). */
caretPos?: number;
/** Current value of the target textarea (for direct insert). */
textValue?: string;
/** Called with the new full text value after tag insertion. */
onTextChange?: (newText: string) => void;
readonly?: boolean;
className?: string;
}
const CATEGORIES: { key: Category; labelKey: string }[] = [
{ key: "all", labelKey: "gemini.filterAll" },
{ key: "emotion", labelKey: "gemini.filterEmotion" },
{ key: "pacing", labelKey: "gemini.filterPacing" },
{ key: "effect", labelKey: "gemini.filterEffect" },
{ key: "voice_quality", labelKey: "gemini.filterVoiceQuality" },
];
export function AudioTagPicker({
onInsert,
caretPos = 0,
textValue = "",
onTextChange,
readonly = false,
className,
}: AudioTagPickerProps) {
const { t } = useTranslation("tts");
const [query, setQuery] = useState("");
const [activeCategory, setActiveCategory] = useState<Category>("all");
const tags: AudioTag[] =
query.trim()
? searchTags(query)
: activeCategory === "all"
? GEMINI_AUDIO_TAGS
: filterTagsByCategory(activeCategory);
const handleInsert = (tag: string) => {
if (readonly) return;
if (onTextChange && textValue !== undefined) {
onTextChange(insertTagAtCaret(textValue, tag, caretPos));
}
onInsert?.(tag);
};
return (
<div className={cn("space-y-2", className)}>
<p className="text-xs text-muted-foreground">{t("gemini.audioTagsHint")}</p>
{/* Search */}
<input
type="text"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder={t("gemini.searchPlaceholder")}
disabled={readonly}
className="w-full rounded-md border border-input bg-background px-3 py-1.5 text-sm text-base md:text-sm"
/>
{/* Category filter (hidden when searching) */}
{!query.trim() && (
<div className="flex flex-wrap gap-1">
{CATEGORIES.map(({ key, labelKey }) => (
<button
key={key}
type="button"
onClick={() => setActiveCategory(key)}
className={cn(
"rounded-full px-2 py-0.5 text-xs border transition-colors",
activeCategory === key
? "bg-primary text-primary-foreground border-primary"
: "border-border hover:bg-muted"
)}
>
{t(labelKey)}
</button>
))}
</div>
)}
{/* Tag grid */}
<div className="flex flex-wrap gap-1.5 max-h-40 overflow-y-auto">
{tags.map((tag) => (
<button
key={tag.tag}
type="button"
title={tag.description}
onClick={() => handleInsert(tag.tag)}
disabled={readonly}
className={cn(
"rounded px-2 py-0.5 text-xs border border-border font-mono transition-colors",
readonly
? "cursor-default opacity-60"
: "hover:bg-primary hover:text-primary-foreground hover:border-primary cursor-pointer"
)}
>
{tag.tag}
</button>
))}
</div>
</div>
);
}
+6 -14
View File
@@ -3,6 +3,7 @@ import { createPortal } from "react-dom";
import { useTranslation } from "react-i18next";
import { Bot, ChevronDown } from "lucide-react";
import { useHttp } from "@/hooks/use-ws";
import { usePortalDropdownClose } from "@/hooks/use-portal-dropdown-close";
import { useAuthStore } from "@/stores/use-auth-store";
import type { AgentData } from "@/types/agent";
@@ -49,20 +50,11 @@ export function AgentSelector({ value, onChange }: AgentSelectorProps) {
});
}, [open]);
useEffect(() => {
if (!open) return;
const handler = (e: MouseEvent) => {
const target = e.target as Node;
if (
containerRef.current && !containerRef.current.contains(target) &&
(!dropdownRef.current || !dropdownRef.current.contains(target))
) {
setOpen(false);
}
};
document.addEventListener("mousedown", handler);
return () => document.removeEventListener("mousedown", handler);
}, [open]);
usePortalDropdownClose({
open,
onClose: () => setOpen(false),
ignore: [containerRef, dropdownRef],
});
const selected = agents.find((a) => a.agent_key === value);
const selectedEmoji = selected ? agentEmoji(selected) : undefined;
@@ -0,0 +1,130 @@
import * as React from "react"
import type { ParamSchema, ParamType } from "@/api/tts-capabilities"
import { Slider } from "@/components/ui/slider"
import { Switch } from "@/components/ui/switch"
export type ParamValue = string | number | boolean
export interface FieldProps {
schema: ParamSchema
value: ParamValue
onChange: (val: ParamValue) => void
readonly: boolean
}
function RangeField({ schema, value, onChange, readonly }: FieldProps) {
const num = typeof value === "number" ? value : Number(value ?? schema.default ?? 0)
return (
<div className="flex items-center gap-3">
<Slider
min={schema.min ?? 0}
max={schema.max ?? 1}
step={schema.step ?? 0.01}
value={[num]}
onValueChange={readonly ? undefined : ([v = 0]) => onChange(v)}
disabled={readonly}
aria-label={schema.label}
className="flex-1"
/>
<span className="text-sm tabular-nums w-10 text-right">{num.toFixed(2)}</span>
</div>
)
}
function NumberField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="number"
className="border rounded px-2 py-1 text-sm w-full"
min={schema.min}
max={schema.max}
step={schema.step ?? 1}
value={value as number}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(Number(e.target.value))}
aria-label={schema.label}
/>
)
}
function IntegerField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="number"
className="border rounded px-2 py-1 text-sm w-full"
min={schema.min}
max={schema.max}
step={1}
value={value as number}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(Math.round(Number(e.target.value)))}
aria-label={schema.label}
/>
)
}
function EnumField({ schema, value, onChange, readonly }: FieldProps) {
return (
<select
className="border rounded px-2 py-1 text-sm w-full"
value={value as string}
disabled={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
>
{(schema.enum ?? []).map((opt) => (
<option key={opt.value} value={opt.value}>
{opt.label}
</option>
))}
</select>
)
}
function BooleanField({ schema, value, onChange, readonly }: FieldProps) {
return (
<Switch
checked={Boolean(value)}
onCheckedChange={readonly ? undefined : (v) => onChange(v)}
disabled={readonly}
aria-label={schema.label}
/>
)
}
function StringField({ schema, value, onChange, readonly }: FieldProps) {
return (
<input
type="text"
className="border rounded px-2 py-1 text-sm w-full"
value={value as string}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
/>
)
}
function TextField({ schema, value, onChange, readonly }: FieldProps) {
return (
<textarea
className="border rounded px-2 py-1 text-sm w-full resize-y min-h-[4rem]"
value={value as string}
readOnly={readonly}
onChange={readonly ? undefined : (e) => onChange(e.target.value)}
aria-label={schema.label}
/>
)
}
export const fieldRenderers: Record<ParamType, React.FC<FieldProps>> = {
range: RangeField,
number: NumberField,
integer: IntegerField,
enum: EnumField,
boolean: BooleanField,
string: StringField,
text: TextField,
}
export { StringField as DefaultField }
@@ -0,0 +1,60 @@
/**
* Pure logic helpers for DynamicParamForm.
* No React, no DOM — safe to import in unit tests without jsdom.
*
* Exported functions:
* initializeDefaults — build initial value map from schemas
* applyNestedChange — immutable update of value map by key
* getRendererName — returns ParamType string (for dispatch dispatch map testing)
* coerceNumericValue — NaN → default; used by number/integer/range fields
*/
import type { ParamSchema, ParamType } from "@/api/tts-capabilities";
export type ParamValueMap = Record<string, string | number | boolean>;
/**
* Builds an initial value object from an ordered list of ParamSchema.
* Each key maps to schema.default (or "" when default is absent).
* Keys are stored flat (dot-notation preserved as-is) — the wire format
* matches what the backend reads from opts.Params.
*/
export function initializeDefaults(schemas: ParamSchema[]): ParamValueMap {
const out: ParamValueMap = {};
for (const s of schemas) {
out[s.key] = s.default !== undefined ? (s.default as string | number | boolean) : "";
}
return out;
}
/**
* Returns a new map with key set to val — immutable (does not mutate src).
* Key may be a dot-notation string (e.g. "voice_settings.stability"); it is
* stored flat as-is to match the backend wire format.
*/
export function applyNestedChange(
src: ParamValueMap,
key: string,
val: string | number | boolean,
): ParamValueMap {
return { ...src, [key]: val };
}
/**
* Returns the renderer name for a ParamType (identity function).
* Exists so tests can assert the dispatch table without importing React.
*/
export function getRendererName(type: ParamType): ParamType {
return type;
}
/**
* Coerces a potentially-NaN numeric input to a safe value.
* Used by RangeField / NumberField / IntegerField onChange handlers.
*/
export function coerceNumericValue(
val: number,
defaultVal: number | undefined,
): number {
if (Number.isNaN(val)) return defaultVal ?? 0;
return val;
}
@@ -0,0 +1,85 @@
import type { ParamSchema } from "@/api/tts-capabilities"
import {
fieldRenderers,
DefaultField,
type FieldProps,
type ParamValue,
} from "./dynamic-param-form-fields"
export type { ParamValue, FieldProps }
export interface DynamicParamFormProps {
/** Ordered list of param schemas from ProviderCapabilities.params */
schema: ParamSchema[]
/** Current values keyed by ParamSchema.key */
value: Record<string, ParamValue>
/** Called when a field changes. Not fired when readonly=true. */
onChange?: (key: string, val: ParamValue) => void
/** When true all inputs are rendered in read-only / display mode. */
readonly?: boolean
}
/**
* Returns true when all DependsOn constraints are satisfied by formState (AND semantics).
* An empty DependsOn array means always visible.
*/
export function evaluateDependsOn(
deps: ParamSchema["depends_on"],
formState: Record<string, ParamValue>,
): boolean {
if (!deps || deps.length === 0) return true
return deps.every((d) => String(formState[d.field]) === String(d.value))
}
/**
* Renders a list of TTS provider params from a ParamSchema array.
* Each field type maps to a dedicated renderer. Visibility is gated by
* evaluateDependsOn. When readonly=true, no onChange callbacks fire.
*
* NOT mounted in tts-page.tsx in Phase A — exported for Phase C wiring.
*/
export function DynamicParamForm({
schema,
value,
onChange,
readonly = false,
}: DynamicParamFormProps) {
if (!schema || schema.length === 0) return null
return (
<div className="space-y-4">
{schema.map((param) => {
if (!evaluateDependsOn(param.depends_on, value)) return null
const Renderer = fieldRenderers[param.type] ?? DefaultField
const currentVal: ParamValue =
value[param.key] !== undefined
? value[param.key]!
: ((param.default as ParamValue) ?? "")
const handleChange = (val: ParamValue) => {
if (!readonly && onChange) {
onChange(param.key, val)
}
}
return (
<div key={param.key} className="space-y-1">
<label className="text-sm font-medium" htmlFor={`param-${param.key}`}>
{param.label}
</label>
{param.description && (
<p className="text-xs text-muted-foreground">{param.description}</p>
)}
<Renderer
schema={param}
value={currentVal}
onChange={handleChange}
readonly={readonly}
/>
</div>
)
})}
</div>
)
}
@@ -0,0 +1,134 @@
/**
* MultiSpeakerEditor — configures up to 2 Gemini TTS speakers.
* Exports pure-logic helpers (addSpeaker, removeSpeaker, updateSpeakerVoice)
* so they can be unit-tested without a DOM environment.
*/
import { useTranslation } from "react-i18next";
import { X, Plus } from "lucide-react";
import { cn } from "@/lib/utils";
// ---- Pure logic exports (testable without React) ----
export interface SpeakerVoice {
speaker: string;
voiceId: string;
}
/** Maximum speakers Gemini TTS supports in multi-speaker mode. */
export const MAX_GEMINI_SPEAKERS = 2;
/** Returns a new array with speaker appended, capped at MAX_GEMINI_SPEAKERS. */
export function addSpeaker(speakers: SpeakerVoice[], entry: SpeakerVoice): SpeakerVoice[] {
if (speakers.length >= MAX_GEMINI_SPEAKERS) return speakers;
return [...speakers, entry];
}
/** Returns a new array with the speaker at `index` removed. */
export function removeSpeaker(speakers: SpeakerVoice[], index: number): SpeakerVoice[] {
return speakers.filter((_, i) => i !== index);
}
/** Returns a new array with the voiceId at `index` replaced. */
export function updateSpeakerVoice(
speakers: SpeakerVoice[],
index: number,
voiceId: string
): SpeakerVoice[] {
return speakers.map((s, i) => (i === index ? { ...s, voiceId } : s));
}
// ---- React component ----
interface MultiSpeakerEditorProps {
speakers: SpeakerVoice[];
voices: string[]; // available voice IDs for the select
onChange: (speakers: SpeakerVoice[]) => void;
readonly?: boolean;
className?: string;
}
export function MultiSpeakerEditor({
speakers,
voices,
onChange,
readonly = false,
className,
}: MultiSpeakerEditorProps) {
const { t } = useTranslation("tts");
const atMax = speakers.length >= MAX_GEMINI_SPEAKERS;
const handleAdd = () => {
if (readonly || atMax) return;
const next = addSpeaker(speakers, { speaker: "", voiceId: voices[0] ?? "" });
onChange(next);
};
const handleRemove = (idx: number) => {
if (readonly) return;
onChange(removeSpeaker(speakers, idx));
};
const handleNameChange = (idx: number, name: string) => {
if (readonly) return;
onChange(speakers.map((s, i) => (i === idx ? { ...s, speaker: name } : s)));
};
const handleVoiceChange = (idx: number, voiceId: string) => {
if (readonly) return;
onChange(updateSpeakerVoice(speakers, idx, voiceId));
};
return (
<div className={cn("space-y-2", className)}>
<p className="text-xs text-muted-foreground">{t("gemini.multiSpeakerHint")}</p>
{speakers.map((s, idx) => (
<div key={idx} className="flex items-center gap-2">
<input
type="text"
value={s.speaker}
onChange={(e) => handleNameChange(idx, e.target.value)}
placeholder={t("gemini.speakerName")}
disabled={readonly}
className="flex-1 rounded-md border border-input bg-background px-3 py-1.5 text-sm text-base md:text-sm"
/>
<select
value={s.voiceId}
onChange={(e) => handleVoiceChange(idx, e.target.value)}
disabled={readonly}
className="flex-1 rounded-md border border-input bg-background px-2 py-1.5 text-sm text-base md:text-sm"
>
{voices.map((v) => (
<option key={v} value={v}>{v}</option>
))}
</select>
{!readonly && (
<button
type="button"
onClick={() => handleRemove(idx)}
className="shrink-0 rounded p-1 hover:bg-destructive/10 text-destructive"
title={t("gemini.removeSpeaker")}
>
<X className="h-4 w-4" />
</button>
)}
</div>
))}
{!readonly && (
<button
type="button"
onClick={handleAdd}
disabled={atMax}
className={cn(
"flex items-center gap-1 text-sm text-primary hover:underline",
atMax && "cursor-not-allowed opacity-50"
)}
>
<Plus className="h-4 w-4" />
{atMax ? t("gemini.maxSpeakersReached") : t("gemini.addSpeaker")}
</button>
)}
</div>
);
}
@@ -1,9 +1,10 @@
import { useMemo, useState, useRef, useEffect, useLayoutEffect } from "react";
import { useMemo, useState, useRef, useLayoutEffect } from "react";
import { createPortal } from "react-dom";
import { useTranslation } from "react-i18next";
import { X, ChevronDownIcon } from "lucide-react";
import { cn } from "@/lib/utils";
import { useSkills } from "@/pages/skills/hooks/use-skills";
import { usePortalDropdownClose } from "@/hooks/use-portal-dropdown-close";
interface SkillNameSelectProps {
value: string[];
@@ -50,20 +51,11 @@ export function SkillNameSelect({
});
}, [open, search]);
useEffect(() => {
if (!open) return;
const handler = (e: MouseEvent) => {
const target = e.target as Node;
if (
containerRef.current && !containerRef.current.contains(target) &&
(!dropdownRef.current || !dropdownRef.current.contains(target))
) {
setOpen(false);
}
};
document.addEventListener("mousedown", handler);
return () => document.removeEventListener("mousedown", handler);
}, [open]);
usePortalDropdownClose({
open,
onClose: () => setOpen(false),
ignore: [containerRef, dropdownRef],
});
const addSkill = (name: string) => {
if (!value.includes(name)) {
Loaded 100 of 123 files, more files were not shown because too many files have changed in this diff. Show more