Openedai Speech

WebUI speech synthesis provider for Agent Zero, backed by an external openedai-speech server (OpenAI-compatible POST /v1/audio/speech, XTTSv2).

Behavior

• Registers a TTS provider with the WebUI ttsService when enabled.
• Auto-speak (speech preference) and the per-message speak buttons use the provider automatically; spoken audio plays directly in the browser.
• The WebUI splits messages into chunks of at most 135 chars. The store packs consecutive chunks into groups of at most 246 chars (the proven a0_audio_player tts_speak regime) and synthesizes each group as a single request. Chunks covered by a group answer with null and reuse the group audio, so XTTSv2 sees long requests and its end-of-stream behavior stays reliable.
• Progressive prefetch: while a response streams, stable chunks (everything except the growing last chunk - measured content-stable against the real chunkText) pack into groups of at most 246 chars as they appear. A group becomes fireable when its packing closes - the next chunk no longer fits, or the response finished; its request text is final either way, so firing early never changes what the server synthesizes. The frontier group stays open until then, so boundaries match the finished-text grouping and tail chunks merge whenever they fit. A 200ms observer plus arrival- and pull-driven pumps fire the oldest unfired closed group within one group of playback, with at most one request in flight; after the response finishes the observer stops and pulls and arrivals alone keep the chain moving. The pull for a chunk answers from that cache, so playback stays gapless and the former pause between the first and second segment is gone. First-group minimum: when the first sentence is shorter than first_group_min_chars (default 100), the scheduler holds the first pull so the frontier group can absorb the next stable chunk. This prevents the audible pause that occurs when a short first group fires immediately and the next group's synthesis starts only after the first group's audio finishes playing. The hold releases when the group reaches the minimum, closes (overflow), the response finishes, or a 3s cap expires — whichever comes first. Set to 0 to disable. Watching rules (0.2.2 lesson): the timer only observes; dispatch is content-deduplicated (one request per group, keyed by text) and chain-limited, so ticks without new stable content are no-ops and the request count never exceeds one per group.
• Live auto-speak: progressive grouping only ever extends forward over stable content, and a chunk is only ever covered by a group whose audio fired at its start - nothing is silently skipped and nothing is spoken twice at the streaming-to-final transition. With progressive prefetch off, streaming chunks synthesize directly and the finished plan forms from the playback cursor onwards only (chunks already spoken during streaming are never regrouped).
• Defensive fallbacks: without a reliable stream mapping (or after a failed group request) chunks synthesize directly - the provider is never worse than plain per-chunk synthesis.
• Applies the same leading/trailing period trick as the a0_audio_player tts_speak tool so XTTSv2 pronounces umlauts correctly at boundaries.
• Optional tail trim (trim_tail) cuts trailing hallucinated speech from wav responses - a safety net against rare stochastic XTTSv2 end-of-stream misses that mostly affect short requests.
• Falls back to browser-native speechSynthesis when the plugin is disabled or the server is unreachable.

Config

Key	Default	Description
server	""	openedai-speech host:port (http, no protocol prefix) - set your server in the plugin settings
model	tts-1-hd	tts-1 (Piper) or tts-1-hd (XTTSv2)
voice	nora	Voice name configured on the server
response_format	wav	Audio format (wav, mp3, opus, aac, flac)
language	auto	Language hint for XTTSv2
speed	1.0	Playback speed multiplier (0.25-4.0)
timeout	60	Seconds per synthesis request (5-300)
trim_tail	true	Cut trailing hallucinated speech from wav responses
progressive_prefetch	true	Group and prefetch upcoming speech while the response streams (off = per-chunk synthesis until the response finishes)
first_group_min_chars	100	Minimum chars for the first speech group during progressive streaming; holds a short first sentence so the group can grow, preventing an audible pause before the next group (0 = disabled)

Routes

• POST /api/plugins/openedai_speech/status - enabled state and resolved config
• POST /api/plugins/openedai_speech/synthesize - synthesize text, returns base64 audio

Tests

Run with the framework venv from /a0 unless noted:

• tests/http_stack_predeploy.py - full Flask dispatch chain (CSRF gate, routing, handler loading, live synthesis). OS_SKIP_LIVE=1 skips the server requests.
• tests/e2e_predeploy.py - handler-level end-to-end checks
• tests/trim_regression.py - tail trim rule against the measured corpus in tests/fixtures (live server captures plus one full a0_audio_player pipeline output)
• tests/group_scheduler_test.mjs - node tests for the chunk group scheduler, including the live auto-speak cursor transition (node tests/group_scheduler_test.mjs)

Notes

• The integrated _kokoro_tts plugin should stay disabled while this provider is active so the provider selection stays unambiguous.

• hooks.py has no install-time work (aiohttp ships with the framework), but deploy.sh touches it on every deploy: hooks.py is one of the framework plugin watchdog patterns, so this reliably triggers the plugin-change pipeline (cache clear, usr.plugins module purge, browser reload push). Python changes go live without a framework restart.

• Licensed under the MIT License (SPDX: MIT) - see LICENSE.