Fix prompt-caching context-usage bug and harden /compact in Excel taskpane
Session from pi-for-excel-sessions on Hugging Face by thomasmustier · MIT · condensed by Coders Talk
Investigate why context % calculation seemed wrong and prompt caching appeared broken, causing a 'prompt too long' 400 error, then harden the /compact command to match Pi's own compaction approach.
- +0Prompt · +0We might have broken prompt caching, or context % calculation is off — got a 'prompt too long: 203094 tokens > 200000' error. Here's a session log to investigate.
- +0mAgent did · +0mParsed the JSON session log with node scripts to find token usage per turn, confirmed the last successful call was ~199,861 prompt tokens (mostly cacheWrite) before the next call overflowed 200k, then traced status-bar.ts's context % calc to find it undercounts because it ignores cacheRead/cacheWrite tokens.
- +11mAgent did · +11mEdited status-bar.ts to include cache tokens in the context % calculation and ran typecheck/lint.
- +31mIntervention · +31mDeveloper noticed 'git resotred' — the export.ts edits had vanished after a git operation; agent checked git status/log and found the working tree was clean, meaning prior edits were lost.Why: My compact.ts changes had disappeared and I wanted to know why before continuing.
- +37mPrompt · +37mCareful with hardcoded max tokens — check what Pi itself does. Also add a temporary debug indicator (behind a flag) showing I/O/cache tokens in the footer.
- +38mAgent did · +38mRead Pi's own compaction.js/utils.js source to find its reserveTokens/keepRecentTokens defaults and chars/4 estimation heuristic, then rewrote export.ts's /compact command and status-bar.ts to match, added a new /debug command and debug.ts flag module gated behind a toggle, and got typecheck/lint clean.
- +1h 11mAgent did · +1h 11mCommitted the compaction/debug work to perf/bundle-slim (8170efd), then cherry-picked and rebased it onto main via a git worktree, pushing both branches.
- +1h 40mFail · +1h 40mDeveloper pointed out the fix targeted the wrong UI element — the long warning was appearing in a toast, not a tooltip, so the tooltip wrapping change was irrelevant and needed reverting.
- +1h 42mAgent did · +1h 42mReverted the tooltip CSS change on both branches and instead fixed .pi-toast to wrap long messages (max-width, overflow-wrap), committing and pushing to perf/bundle-slim and main.
- +1h 48mAgent did · +1h 48mTraced a second compaction failure by comparing Pi's completeSimple call (no temperature set) against the app's own call (temperature: 0.2 alongside reasoning: 'high'), identified that Anthropic rejects temperature≠1 when thinking is enabled, removed the temperature param, and committed/pushed the fix to main (eea4bef).
- +2h 6mOutcome · +2h 6mDocumented compaction behaviour in docs/compaction.md (pushed to main as e41b018), and filed GitHub issue #40 for queueing messages/compact indicator UX follow-up work.
Next time I'd point straight at Pi's own compaction implementation as the reference from the start, since checking 'what does Pi do' twice (for token budgeting and for the temperature/reasoning conflict) was what actually resolved both the context % bug and the mysterious /compact 400 error. The correction that did the real work was catching that temperature: 0.2 combined with reasoning: 'high' is what Anthropic was rejecting — removing temperature fixed compaction outright. I'd also flag early which UI surface (toast vs tooltip) a bug is in, since I sent the agent down the wrong path fixing tooltips before realizing it was a toast.