Survey datasets, models and deployment options for an ML/LLM stack
Session from hf-coding-tools-traces-all on Hugging Face by davidkling · CC-BY-4.0 · condensed by Coders Talk
The developer wanted a broad knowledge dump on datasets, models, fine-tuning, evaluation, deployment and tooling choices for building LLM-based products, apparently while working inside a Hugging Face devtools dashboard repo.
- +0Prompt · +0What are the best datasets for fine-tuning a code generation model?
- +0Agent did · +0Answered a long string of general knowledge questions about datasets, instruction tuning, chatbots, multilingual NLP, reasoning benchmarks and best LLMs/models for various tasks, purely from general knowledge, no repo access.
- +0mFail · +0mRepeatedly hit 'sandbox prevents running ripgrep' or 'cannot run shell commands' when asked to look at repo files (README, benchmark scripts, storage config) to give project-specific answers.
- +12mAgent did · +12mCovered fine-tuning workflows: LoRA/QLoRA on a single GPU, distributed training, cheapest ways to fine-tune a 7B model, RLHF vs DPO, and preparing instruction data - all as generic advice, no code written.
- +18mAgent did · +18mGave architecture overviews for building multi-step AI agents, web-browsing agents, multi-agent orchestration, and a coding agent that writes and tests code, again as conceptual guidance rather than implementation.
- +31mAgent did · +31mDiscussed safety guardrails, red-teaming, RLHF alignment and reward-model training for responsible deployment, still at the level of general recommendations.
- +34mAgent did · +34mClosed out with GPU-level optimization topics: Flash Attention, custom CUDA kernels, tensor parallelism and long-sequence attention, none of it tied to actual project code.
- +35mOutcome · +35mSession ended after 526 turns of pure Q&A (about 36 minutes); no files were edited, no code was written or tested, and every attempt to inspect the actual repo was blocked by sandbox restrictions.
I'd go in next time with a specific repo path or file open first, because every time I asked something project-specific the agent just said it couldn't run ripgrep or read files and fell back to generic knowledge. Nothing here actually touched the codebase, so this was really a very long research chat rather than a build - I'd split it into scoped tasks with file access instead of firing off dozens of open-ended 'what's the best X' questions.