George Sung

Home robots will have to plan in houses they have never seen, alongside people doing their own share of the work. I took Muse Glimmer 30B, Meta's open-weights model, made it the high-level planner in Meta's PARTNR benchmark (Habitat 3.0), and improved it with reinforcement learning in simulation: no human demonstrations or labels, only whether the task got done. A robot can't practice thousands of chores with real people, so Glimmer practices with a simulated partner played by a copy of itself. The two agents plan separately from their own partial views and can't message each other, and only the partner can clean, fill or switch things on, so they have to work out how to split the job. During the hackathon I built the whole pipeline: Glimmer as a native tool-calling PARTNR planner, online GRPO with LoRA on a 30B model, and live adapter updates into vLLM with speculative decoding. On 1000 tasks in 12 houses never seen in training, the success rate rose from 782 (original Muse Glimmer 30B) to 810 (my fine-tuned version for this hackathon) out of 1000. Code, adapter and all 1000 episodes with their reasoning are public. Project page: https://muse-habitat.georgesung.com/ Github: https://github.com/georgesung/muse-habitat LoRA adapter on HuggingFace: https://huggingface.co/georgesung/Muse-Glimmer-30B-Habitat-PARTNR-Lora