docs: phase 02 completion — voice/audio system docs + architecture updates

Update architecture overview with streaming TTS provider layer. Expand tools
system docs with voice/model resolution. Update HTTP and WebSocket RPC
documentation for voice endpoints. Record Phase 02 completion in changelog.
Update CLAUDE.md with voice picker and streaming TTS patterns.
This commit is contained in:
viettranx committed 2026-04-15 11:24:57 +07:00
1 parent a7c8170c4a
commit cf16cf53db
6 files changed
+181 -12

No files matched your search

+23
View File
@@ -475,6 +475,29 @@ Six distinct workspace scenarios:
- **L0 Auto-Injection**: Top-K vault entries injected into system prompt as "relevant context from vault"
- **Filesystem Sync**: Vault entries exported as `.md` files for manual editing, re-imported with change tracking
### Audio & Voice System (ElevenLabs + Streaming TTS)
**Provider architecture** (`internal/audio/`):
- **`Manager`**: Central orchestrator dispatching TTS/STT/Music/SFX requests to pluggable providers
- **`TTSProvider` interface**: Core text-to-speech contract (blocking, buffered response)
- **`StreamingTTSProvider` interface**: Optional interface for ElevenLabs `/stream` endpoint (chunked audio via `io.ReadCloser`)
- **Implementations**: ElevenLabs, OpenAI, Edge, MiniMax (phase-gated; STT/Music/SFX partial)
**Voice discovery & caching**:
- **Voice cache** (`internal/audio/voice_cache.go`): In-memory LRU (cap 1000 tenants, TTL 1h) shared by HTTP `/v1/voices` + WS `voices.list` handlers. Thread-safe with `sync.Mutex` (LRU updates require write lock)
- **Cache miss recovery**: HTTP handler auto-fetches from ElevenLabs; WS handler requires prior cache warm (provider resolution deferred to Phase 3)
- **Agent audio context** (`store.WithAgentAudio` / `AgentAudioFromCtx`): Immutable snapshot bundle (AgentID + OtherConfig JSONB) injected by dispatcher before tool dispatch; consumed by `TtsTool.Execute` for voice/model resolution
**Agent-level configuration** (`agents.other_config` JSONB):
- `tts_voice_id`: ElevenLabs voice ID (e.g., "pMsXgVXv3BLzUgSXRplE")
- `tts_model_id`: Model choice (eleven_v3, eleven_flash_v2_5, eleven_multilingual_v2, eleven_turbo_v2_5)
- Resolution precedence: CLI args → agent config → tenant override → provider default
**Web UI voice picker** (`ui/web/src/components/voice-picker.tsx`):
- Combobox with BM25 search, preview playback button (HTML `<audio>`)
- Handles preview CDN 403 (expiry) via `onError` → auto-refresh cache
- Embedded in PromptSettingsSection, bound to `other_config.tts_voice_id`
---
## Cross-References