Fine-tune ModernBERT on Jim Crow law dataset via HF Jobs
Session from agent-race-traces on Hugging Face by davanstrien · CC-BY-4.0 · condensed by Coders Talk
Explore the biglam/on_the_books dataset, fine-tune a text classifier to identify Jim Crow laws, train it via HF Jobs on GPU hardware, and push the model to a personal HF namespace.
- +0Prompt · +0Fine-tune a model on biglam/on_the_books to identify Jim Crow laws, train via hf jobs, push to davanstrien/jim-crow-laws-claude-code
- +0mAgent did · +0mExplored HF CLI and dataset with a UV script: 1,785 rows, binary jim_crow label, section_text field; decided on ModernBERT-base for its 8K context
- +2mAgent did · +2mWrote train_jim_crow.py training script (202 lines), checked available job hardware, and linted with ruff
- +2mIntervention · +2mInterrupted after seeing ruff lint errors (unused imports), then said 'continue' to let it fix themWhy: I wanted to make sure the lint errors got fixed before moving on to submitting the job.
- +7mFail · +7mFirst HF Jobs training run failed with exit code 1: Trainer no longer accepts tokenizer= argument in newer transformers
- +7mAgent did · +7mFixed tokenizer= to processing_class= and resubmitted the job on l4x1 hardware
- +12mAgent did · +12mMonitored resubmitted job logs until training completed and model weights were pushed to the Hub
- +15mAgent did · +15mRewrote the auto-generated model card with proper training/dataset context and uploaded README.md to the Hub repo
- +16mOutcome · +16mModel fine-tuned and live at davanstrien/jim-crow-laws-claude-code: ModernBERT-base trained on 1,428/357 train/eval split, ~4.5 min training on L4 GPU after one failed run, with an improved model card pushed
Next time I'd mention up front that the transformers API had renamed tokenizer= to processing_class=, since that's what caused the one failed job run. Letting it auto-fix the ruff lint errors after I interrupted just to check them worked fine, that intervention was more a sanity check than a real correction. Overall it moved fast: dataset exploration, script writing, one failed job, a fix, and a successful run with a properly written model card, all in under 20 minutes.