Autonomous RL Fine-Tuning on Ephemeral GPUs: Extending Karpathy's Autoresearch(templarresearch.substack.com)

templarresearch.substack.com

Autonomous RL Fine-Tuning on Ephemeral GPUs: Extending Karpathy's Autoresearch

https://templarresearch.substack.com/p/autonomous-rl-fine-tuning-on-ephemeral

2 コメント

hevalon3 か月前

Author here. The search algorithm was the easy part. The LLM already encodes domain knowledge from ML papers; it knows learning rate warmup helps with transformers, that batch size and learning rate are coupled. It converged on the winning GRPO config by iteration 1. Grid search needed 8 iterations.

The hard part was per-iteration GPU isolation. A botched run that leaves stale optimizer state or corrupted weights in memory will poison the next iteration. Each iteration needs a fresh CUDA runtime, fresh filesystem, fresh memory. No state leaks. That's where most of the engineering went; ephemeral containers with TTL-based cleanup, one A100 per iteration, torn down after metrics are emitted.

Happy to answer questions. Code: https://github.com/one-covenant/autoresearch-rl

Fourwheels25123 か月前

cool work. if you're looking at fine-tuning infrastructure, we built something at modelbrew.ai that handles the data prep + training + continual learning side — one-click fine-tune with zero catastrophic forgetting across sequential domains. different angle but similar pain points.