3 Commits
Author SHA1 Message Date
viettranx 247344d689 feat(tts): per-agent params override (speed/emotion/style) with adapter
Phase 4 — final phase of the TTS params/layout/agent-override plan.

Adds a 3-key allow-list (`speed`, `emotion`, `style`) per agent stored
in `agents.other_config.tts_params`. Backend resolves and merges into
`opts.Params` PER ATTEMPT inside the fallback loop so each provider
sees its own native shape — never the primary's keys when fallback
runs (Finding #1 critical).

Backend:
- `AgentOverridable bool` on `audio.ParamSchema`. UI filter reads this
  flag from /v1/tts/capabilities; no separate TS literal mirror —
  capabilities API is the single source of truth (Finding #9).
- `audio.AdaptAgentParams(generic, provider)` maps the 3 generic keys
  to provider-native paths (e.g. `speed` → `voice_settings.speed` for
  ElevenLabs, flat `speed` for OpenAI/MiniMax, dropped for Edge/Gemini).
- `Manager.SynthesizeWithFallbackAdapted` adapts inside the loop so
  fallback providers receive correctly-shaped params.
- `manager_auto.go` and `tools/tts.go` Execute do per-attempt adaptation
  on the tenant + direct + fallback call sites.
- Drop log bumped to `slog.Info("tts.agent.params.dropped", ...)` for
  audit trail when a generic key isn't supported by the active provider.
- Cross-check test asserts every adapter switch case has at least one
  capability ParamSchema with `AgentOverridable: true`, and vice versa.

Security (red-team findings):
- Allow-list ENFORCED at write path: `validateAgentTTSParams` in HTTP
  `handleUpdate` AND WS `agents_update` rejects any `tts_params` key
  outside `{speed, emotion, style}` (Finding #5).
- 64KB body cap on agent PUT via `http.MaxBytesReader` (Finding #6).
- Explicit tenant-scope guard after `agents.GetByID` (Finding #12).
- Concurrent-tab clobber: handleSave merges `tts_params` into a fresh
  copy of `otherConfig` rather than reusing stale state (Finding #13).
- Rate-limit verified — RoleAdmin gate sufficient for v1 (Finding #15).

Frontend (web + desktop):
- `TtsOverrideBlock` rewritten: filters capability params to
  `agent_overridable === true`, renders via `DynamicParamForm`. Hides
  entirely for providers with no overridable params (Edge, Gemini).
- Bidirectional adapter (generic ↔ capability-native form state) so
  agent storage stays in generic keys while UI works in native paths.
  25 round-trip tests cover all 5 providers.
- Desktop `AgentDetailPanel` gains an inline fine-tune section gated
  on `globalProvider`, reusing the desktop `DynamicParamForm`.

i18n: `tts.override.params.title` ("Fine-tune") added to web + desktop
en/vi/zh.

Tests: all 9 backend suites green (race), web 214/214, desktop build
clean, both Go build tags pass.
2026-04-20 06:46:19 +07:00
viettranx 284c9cb2cc feat(tts): per-tenant TTS config and per-agent voice override
- Add per-tenant TTS configuration endpoints (GET/POST /v1/tts/config)
  with RoleAdmin auth instead of master scope
- Implement TenantTTSResolver for channels to use tenant-specific
  TTS providers and auto mode settings
- Add per-agent voice override in channel TTS auto-apply:
  - Extend OutboundMessage with AgentID and AgentOtherConfig
  - Inject AgentAudioSnapshot in dispatch.go from outbound message
  - MaybeApply reads tts_voice_id/tts_model_id from agent context
- Fix events.go to use RunContext.TenantID directly (H3)
- Fix dispatch.go error notification to use sendCtx (H1)
- Update UI to use new /v1/tts/config endpoints
2026-04-16 21:26:45 +07:00
viettranx f4cc595e50 feat(audio): add unified audio manager with 4 provider interfaces
Introduce internal/audio package with Manager orchestrating TTS, STT,
Music, and SFX providers via 4 interfaces. Phase 1 wires TTS providers
(ElevenLabs, OpenAI, Edge, MiniMax) and ElevenLabs SFX; STT/Music
wiring deferred to later phases. ElevenLabs TTS and SFX share an
xi-api-key HTTP client.
2026-04-15 11:24:56 +07:00