Fix stale word-audio cache bug and benchmark word-boundary splitter
Session from read-along-ai-agent-traces on Hugging Face by kingkw1 · MIT · condensed by Coders Talk
19msession
1interventions
GOAL
Fix a bug where cached TTS word audio leaked across sentences in a read-along app, then quantify and document how the current word-boundary detection method compares to the previous one.
- +0Prompt · +0Fix the bug where a word's cached audio from one sentence gets reused for the same word in a later sentence (e.g. 'the' in 'The cat sat' vs 'The dog ran fast').
- +0mAgent did · +0mTraced the issue to TTS_MEMORY_CACHE being keyed only by normalized word, so a word's audio from one sentence was reused in later sentences.
- +0mAgent did · +0mChanged the cache key to (sentence, word), kept the browser payload keyed by visible word, updated existing tests, added a regression test for the shared-word case, and confirmed the focused test suite passed.
- +1mAgent did · +1mFound and fixed a related race where an older background prewarm worker could briefly reactivate a previous sentence's status; added a regression test and reran the suite, which passed.
- +9mPrompt · +9mTell me how the current setup performs with our testing, now that we're in my local environment.
- +9mFail · +9mFull quiet pytest run exited with code 2 and no output, suggesting collection failed early.
- +9mAgent did · +9mIsolated the failure by running test files individually and via --collect-only, found focused suites all passed, fixed a quoting mistake in a one-off metrics script, and computed benchmark numbers directly from CASES: current splitter 9/9 (100% hit rate, 0 error) vs previous proportional splitter 5/9 (55.6%, ~93ms mean error).
- +13mPrompt · +13mDocument this benchmark in a data-science-style notebook covering the dataset, comma separation, and the improvement.
- +14mAgent did · +14mAdded notebooks/03_word_boundary_benchmark.ipynb with the synthetic benchmark dataset, splitter comparison, and results table; validated JSON and executed cells directly since nbconvert wasn't installed, confirming the 9/9 vs 5/9 numbers matched.
- +17mIntervention · +17mDeveloper questioned whether the benchmark used their actual recorded/labeled dataset, noticing unfamiliar words like 'elephant' and 'rhinoceros'.Why: It looked like the notebook was using made-up words instead of my recorded dataset, and didn't mention my labeled data at all, so I called that out before letting it stand as documentation.
- +17mAgent did · +17mConfirmed the benchmark used only synthetic fixtures, inspected the real labeled dataset (50 cleaned recordings, baseline_results.csv, train.jsonl), and patched the notebook to clarify the distinction and add a real labeled-data summary section, then re-validated by executing the notebook cells.
- +19mOutcome · +19mCache bug and status race fixed with passing regression tests; benchmark shows 9/9 (100%) vs 5/9 (55.6%) hit rate; notebook documented and corrected to distinguish synthetic benchmark data from the real 50-recording labeled dataset.
VERDICT
Next time I'd tell it upfront which dataset to use for any benchmark or notebook, since it defaulted to synthetic fixtures without flagging that clearly. Catching that it wasn't using my actual recorded/labeled data was the correction that mattered here — otherwise the notebook would have read as validated results on data it never touched. The cache and race-condition fix itself went smoothly with good regression tests added along the way.