Swap storyboard chat model to Nemotron-3-Nano-4B-FP8 on vLLM
Session from build-small-agent-trace on Hugging Face by drdavidtang · MIT · condensed by Coders Talk
Replace the storyboard app's chat model with nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 using FP8 KV-cache quantisation, prewarming the container on app startup while keeping MiniCPM for planning.
- +0Prompt · +0At app startup, also warm up the Nemotron container; swap in nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 with FP8 KV-cache quantisation for the storyboard chat function.
- +0mAgent did · +0mExplored the Modal container setup, backend chat/planner endpoints, and frontend to scope the change: point /api/chat at a dedicated Nemotron vLLM deployment with fp8 kv_cache_dtype, prewarm it on startup, and keep MiniCPM for planning.
- +1mAgent did · +1mPatched modal_app/nemotron.py, backend/engine.py, backend/providers/modal.py, backend/main.py, frontend/src/App.tsx, backend/test_main.py, scripts/modal_deploy.sh, .env.example, and README.md to wire in the new model and NEMOTRON_INFERENCE_URL.
- +1mAgent did · +1mRan pytest, npm build, and py_compile checks; worked around missing pytest/python on PATH and a broken Rollup native binary via uv and direct node invocations. TypeScript and syntax checks passed; pytest and vite build stayed unavailable locally.
- +2mPrompt · +2mI need to redeploy this right
- +7mFail · +7mModal build failed: pinned huggingface_hub==1.5.0 conflicted with vllm==0.15.1's requirement of huggingface-hub<1.0, making the dependency set unsatisfiable.
- +7mAgent did · +7mDiagnosed the conflict as isolated to the Nemotron image and repinned huggingface_hub to 0.36.0, then reran syntax and diff checks before confirming the fix.
- +12mPrompt · +12mAlso clean up the README and metadata/UI to reflect the new changes around Professor Wren.
- +12mAgent did · +12mRebranded the Space to 'Professor Wren's Story Rooms', updated README, index.html, and App.tsx to lead with Professor Wren, documented MiniCPM/Nemotron responsibilities, and removed stale 'Storyboard TUI'/'Narrative Terminal' branding.
- +29mPrompt · +29mOk git add, commit and push this.
- +29mOutcome · +29mCommitted 10 files (135 insertions, 103 deletions) as 493c08c 'Use Nemotron for Professor Wren chat' and pushed to main on the Hugging Face Space remote.
Next time I'd flag the huggingface_hub/vllm version compatibility upfront so the agent pins a working version before the first deploy attempt, since that's what actually blew up the build. The fix itself was quick once the error message pointed at the exact conflict, so letting the agent read the Modal build failure and repin huggingface_hub==0.36.0 did the real work. Local test/build tooling being broken (missing pytest, no execute perms, broken Rollup binary) meant I never got a real green signal before pushing, just static checks.