Fix VoxCPM word-click audio cache leaks and build a labeled boundary benchmark
Session from read-along-ai-agent-traces on Hugging Face by kingkw1 · MIT · condensed by Coders Talk
Improve word-click playback accuracy for a read-along app by diagnosing cross-sentence audio cache leaks, testing VoxCPM pause-prompting variants, and building a benchmark against manually labeled word boundaries.
- +0Prompt · +0Implement forced-alignment-style word timestamps for VoxCPM sentence audio instead of proportional slicing, with fallback and tests.
- +0mAgent did · +0mAdded logging and UI status visibility (alignment vs proportional fallback) to app.py, plus tests; suite passes (22 tests).
- +5mPrompt · +5mReported that word clips leak across sentences (cached 'the' from 'the cat sat' bleeds into 'the dog ran fast'); asked for curriculum WAVs to manually timestamp and compare methods.
- +6mAgent did · +6mMade TTS_MEMORY_CACHE sentence-scoped and cleared on new prewarm to stop stale word clips leaking across sentences; added regression test; suite passes.
- +7mAgent did · +7mBuilt scripts/generate_curriculum_wavs.py to synthesize full-sentence WAVs per curriculum item with a manifest for manual annotation; fixed import path and Conda linker issue, generated 4 WAVs.
- +13mPrompt · +13mAsked whether VoxCPM could produce longer silences between words to make manual/automatic word separation easier.
- +17mPrompt · +17mAsked to generate multiple punctuation-prompting variants (comma, period, spaced-comma, natural) of each sentence for comparison.
- +17mAgent did · +17mExtended generator to produce natural/comma/period/spaced_comma variants, generated 16 WAVs total for listening comparison.
- +39mPrompt · +39mReported comma-separated variant sounded best; asked for the best way to manually label word start/stop times (Audacity labels?).
- +50mPrompt · +50mProvided manually labeled Audacity timestamp files for the comma variant; asked for a benchmark test and git strategy for committing the audio assets.
- +50mAgent did · +50mBuilt test_word_boundary_benchmark.py parsing Audacity labels, scoring boundary correctness against manual silence gaps, added proportional_word_timestamps baseline, and scoped .gitignore to allow committing the comma benchmark WAVs.
- +51mOutcome · +51mBenchmark suite passes (7 tests, 1 xfail) confirming proportional slicing fails the manual-label benchmark; sentence-scoped cache fix and 16 punctuation-variant WAVs plus labeled comma benchmark committed for a future alignment pass.
Next time I'd start by asking for a ground-truth labeling workflow and benchmark harness before building any alignment logic, since that's what actually let us tell good methods from bad ones. Pointing out the exact cross-sentence cache leak ('the cat sat' bleeding into 'the dog ran fast') was the correction that mattered — it revealed the real bug wasn't slicing precision but stale cached clips. The punctuation-variant experiment (comma vs period vs natural) was cheap to generate and gave us a clear winner before investing in alignment.