Upskiller

Upskiller is an RL environment that trains LLMs to agentically invoke skills — deciding which to load, when to unload, and when to submit under a context budget. SkillsBench shows skill invocation fails over half the time; we built Upskiller to close that gap. The environment uses synthetic, fictional skills that can't exist in training data, forcing generalization. Tasks are dynamically generated to prevent memorization. Rewards are deterministic and rules-based — code execution, structural verification, multi-part checks, no LLM judge. Our 5-signal reward trains correctness, precision, recall, context hygiene, and token efficiency. Trained with GRPO + LoRA (r=128) across Qwen 3 1.7B, 4B, and 8B using HF TRL. Built on OpenEnv 0.2.1, deployed on HF Spaces.