Recipe Overview

Some tasks require controlling software or machines directly. A computer control agent equips an LLM with interfaces (like mouse/keyboard APIs) to operate computers. The problem it solves is automating GUI or system interactions. For example, if asked to gather stock data from a website, the agent can open a browser and click through the site under LLM guidance. This enables automation of tasks that lack APIs, allowing agents to work with any software a human could use. The pattern is powerful for legacy systems or complex multi-step processes that cross application boundaries.