AMD Patents a System That Watches Your Screen and Writes Its Own Code to Run Apps
AMD is patenting a chip-level system that can watch what's happening on your computer, figure out what it needs to do next, and write the code to do it, all without touching a single API.
What AMD's screen-reading automation chip actually does
You're trying to automate a repetitive task across three different apps, but none of them expose a way for other software to control them. Normally, that's a dead end. AMD's patent describes a circuit built directly into the processor that can bypass that problem entirely by reading what's on screen and figuring out the current state of any software just by looking at it.
From there, the system breaks a goal down into smaller steps, writes working code for each step, and then executes that code to actually do the thing. It's closer to having a co-pilot inside your chip than a traditional automation tool you install on top of your operating system.
The key detail is the phrase "independent of APIs." Most automation today depends on software makers opening specific doors for other programs to walk through. AMD's approach doesn't wait for that invitation. It observes, plans, and acts on its own.
extracting, independent of one or more application programming interfaces associated with a computational environment, application state data representing a current state of the computational environment …
Translation: It reads your screen directly without needing special tools built into the apps.
How AMD's circuit reads state, sets goals, and generates code
The patent describes an automation circuit built into a processing system that operates in three stages: observe, plan, and act.
In the observe stage, the circuit extracts application state data (a snapshot of what the software is currently doing and displaying) without relying on APIs (the standard hooks that developers normally build in to let other programs talk to theirs). This means it can work on apps that were never designed to be automated.
In the plan stage, the circuit uses that state data to identify one or more objectives and then decomposes (breaks down) those objectives into smaller, manageable sub-objectives. This is the machine learning layer: the system infers what needs to happen next rather than following a fixed script.
In the act stage, the circuit generates executable code for each sub-objective and runs that code directly inside the computational environment. The result is that the chip can take real actions inside software, filling forms, clicking buttons, navigating menus, without a human touching the keyboard.
- No API dependency: works on any software regardless of whether it was designed for automation
- ML-driven goal decomposition: breaks complex tasks into steps dynamically
- On-chip code generation and execution: the whole loop runs at the processor level
… generate executable code corresponding to the plurality of sub-objectives, and execute the executable code to perform one or more actions …
Translation: It writes its own software instructions on the fly to carry out your tasks.
What this means for AI agents running on AMD hardware
The practical problem AMD is attacking here is real: most AI agent systems today hit a wall when they encounter software that doesn't expose clean programmatic controls. Enterprise workflows are full of legacy apps, proprietary tools, and web interfaces that were never built with automation in mind. A system that can observe and act on any software state, without waiting for developer cooperation, would make AI agents far more broadly useful in real work environments.
AMD's angle is also strategic. As AI workloads move increasingly onto the processor itself rather than into cloud services, having an on-chip automation engine positions AMD silicon as a more capable host for autonomous agents. The latest Big Tech patents in the chip and AI-agent space show a clear push toward embedding inference and decision-making closer to the hardware, and this filing sits squarely in that current.
The problem AMD is targeting, software that resists automation because it never opened its doors to other programs, is one of the most concrete friction points slowing down real AI agent adoption at work. Most agent demos look great in controlled environments and fall apart the moment they hit a closed enterprise tool. A chip-level system that reads screen state and generates its own action code is a direct, proportionate answer to that gap. Whether AMD can make this work reliably across the enormous variety of real-world software interfaces is the hard question, and the patent doesn't answer it, but the problem it names is exactly the right one to be solving.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
7 drawing sheets from US 2026/0233100 A1 · click any drawing to enlarge
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →