Push fine-tuned MiniCPM-V invoice model to HuggingFace and fix inference format
Session from kirana-detective-build-traces on Hugging Face by build-small-hackathon · MIT · condensed by Coders Talk
Push a fine-tuned MiniCPM-V 4.6 LoRA adapter from a Modal volume to the developer's personal HuggingFace namespace, update the professional model card, and make the model format compatible with the app's inference stack.
- +0Prompt · +0Push trained MiniCPM-V model from Modal storage to HuggingFace repo naazimsnh02 (org token lacked rights), and update the professional model card with training details.
- +0mAgent did · +0mInvestigated training script and existing model cards, wrote a dedicated push script targeting naazimsnh02, updated MODEL_CARD.md with real training metrics, and after fixing an f-string brace bug and a Windows encoding error, successfully pushed the LoRA adapter to HuggingFace.
- +9mPrompt · +9mCheck whether the model should be pushed as adapter or full/GGUF for the app's inference stack, per the PRD; then replace with GGUF like the other model.
- +9mAgent did · +9mFound the app uses llama-cpp-python not vLLM/transformers, so the LoRA adapter alone can't be used; wrote an export script to merge the LoRA and convert to GGUF, but hit repeated errors including CUDA build failures and finally that MiniCPM-V-4.6's architecture (MiniCPMV4_6Model) is not supported by llama.cpp's converter.
- +23mFail · +23mconvert_hf_to_gguf.py errors with 'Model MiniCPMV4_6Model is not supported' — llama.cpp has no converter support for this architecture at all.
- +38mIntervention · +38mDeveloper modified app.py and export_minicpm_v_gguf.py themselves to try an alternate approach (reusing OpenBMB's base GGUF + LoRA-to-GGUF conversion) and asked the agent to review it.Why: I wanted to try downloading the official OpenBMB GGUF and just converting the LoRA on top instead of the whole model, since full conversion was clearly blocked.
- +49mFail · +49mBoth the LoRA-to-GGUF conversion and a fresh attempt at converting the merged model to GGUF hit the same 'MiniCPMV4_6Model is not supported' wall, confirming no GGUF path exists for this architecture yet.
- +1h 2mAgent did · +1h 2mAbandoned GGUF entirely; switched invoice_extractor.py and app.py to load the merged HF model directly via transformers instead of llama-cpp-python, raised the inference timeout for CPU, and cleaned up repo references.
- +1h 4mPrompt · +1h 4mKeep naazimsnh02 for all creation/training scripts, but have the app and docs point to build-small-hackathon since models get moved there manually after creation; update all docs to match.
- +1h 4mAgent did · +1h 4mSwept app.py, all three agent files, README.md, MODEL_CARD.md, the PRD, and .kiro spec files, splitting references so creation scripts point to naazimsnh02 and runtime/docs point to build-small-hackathon; verified no stray references remained.
- +1h 7mOutcome · +1h 7mWrote a new professional model card and push script for the merged model repo; session ended with the developer needing to set HF_TOKEN on Windows before running the final push — card not yet confirmed pushed.
Next time I'd start by checking the app's actual inference stack (llama-cpp-python, not vLLM) before pushing any model format, since that assumption cost a lot of back-and-forth on GGUF conversion that turned out to be a dead end for MiniCPM-V-4.6. The correction that actually unblocked things was giving up on GGUF and switching the app to load the merged HF model via transformers directly — everything after that went smoothly.