Simplify evaluation dataset generation for Homeroom Copilot (Gradio/pandas)
Session from homeroom-copilot-open-traces on Hugging Face by ravi2505 · MIT · condensed by Coders Talk
Replace the over-engineered UI-driven evaluation export with a simple, developer-only workflow to produce 8 fixed evaluation cases (2 per risk level) without exposing any export tooling to end users.
- +0Prompt · +0Simplify evaluation: remove UI export button/workflow, keep evaluation/ docs and case files, add a one-time developer script to generate 8 fixed risk cases without touching scoring/root-cause/retrieval logic.
- +0mAgent did · +0mRemoved the Export Evaluation Dataset button and its callback wiring from app.py, deleted ~371 lines of export helper functions, and added scripts/generate_evaluation_cases.py as the manual one-time generator.
- +3mAgent did · +3mRestored/created evaluation/HUMAN_EVALUATION_GUIDE.md, GPT4_JUDGE_PROMPT.md, blank human_scores.csv/gpt_scores.json/evaluation_results.csv, ran syntax checks, and confirmed no export UI remained in app.py.
- +18mPrompt · +18mChange of plan: instead of a manual script, auto-generate the evaluation dataset once on app startup (background, no UI) using the deployed model, only if evaluation_dataset.json doesn't already exist.
- +18mAgent did · +18mAdded a background daemon thread (start_evaluation_generation_thread) in app.py that checks for evaluation_dataset.json and generates all 8 cases via the existing Qwen action-plan pipeline if missing; restored the guide/prompt docs.
- +48mPrompt · +48mChange of plan again: do NOT auto-generate AI Action Plans via the model. Only extract existing student profile/risk/root-cause/intervention data into case files with a placeholder for manually-added action plans; remove the startup generation hook.
- +48mIntervention · +48mDeveloper stopped the background LLM-calling generation approach after realizing it would invoke Qwen automatically on every deploy, and redirected to a placeholder-only extraction script.Why: I didn't want the app silently calling the model on startup and risking cost/inconsistent output — I only wanted existing data extracted, with action plans pasted in manually from the HF Space.
- +49mAgent did · +49mRemoved the startup evaluation thread and generate_ai_action_plan-dependent code from app.py (import cleanup, ~243 lines removed), then added scripts/export_student_evaluation_data.py which extracts profile/risk/root-cause/intervention data for 8 named students into case_00X.md files with an action-plan placeholder, verified no Qwen/generate_text calls remain.
- +1h 7mPrompt · +1h 7mFix a UI bug: Root Cause Analysis panel content doesn't scroll correctly when it overflows; fix overflow/CSS only, no redesign, must work on desktop and mobile.
- +1h 7mAgent did · +1h 7mTraced root-cause panel CSS, added a .root-cause-section flex container with min-height:0, fixed .analysis-main-scroll flex/overflow rules, hid horizontal overflow, adjusted mobile CSS for a fixed scrollable height; syntax check passed but live browser verification was not possible since no server was running on port 7861.
- +1h 9mOutcome · +1h 9mDelivered: app.py with export UI and auto-generation removed, scripts/export_student_evaluation_data.py for manual data extraction, evaluation/ docs restored, and CSS fix for Root Cause Analysis scrolling — verified only via static syntax/CSS checks, not a live browser run.
Next time I'd specify upfront that action plans must be pasted in manually and that nothing should call the model automatically — I went through three different generation strategies (UI export removal, background auto-generation via Qwen, then placeholder-only extraction) before landing on what I actually wanted. The correction that mattered was stopping the background thread that silently called generate_ai_action_plan() on startup; that's the one that could have caused real cost/consistency problems if shipped.