MOLLI

MOLLI is a vision-based AI agent that learns a programming game by looking at the screen, then writes and runs its own code. It plays The Farmer Was Replaced, where you control a farming drone with a Python-like language. MOLLI has no game API or game-state access. It works only from screen captures and real mouse and keyboard input, as a person would. Its loop is: read the screen with OCR and a vision model, open the in-game manual and save the functions it finds to a dictionary, write code, have that code reviewed, type it into the editor, run it, and check the screen for changes and errors. Most automation depends on an API or on scripts written for one specific interface. MOLLI explores a different problem: can an agent operate software it was never integrated with, learning the rules from the same screen and documentation a human would use? Currently, the agent can identify screens, write code, run it and detect errors. Reading resource counts reliably and buying upgrades are still in progress.