This storyline collects filings on AI agents that plan actions, control apps, parse tool calls, read images, remember past chats, listen to ambient sound, and get checked by privacy and compliance guardrails. Together they show Big Tech building assistants that act on your behalf while also policing what those agents see, hear, and do.
53 filings · tracking since May 2026 · latest Jul 2026 · updates automatically as new filings publish
US 2026/0187170 A1
Apps pre-register their executable actions with Google Search, letting the system invoke commands directly rather than routing users through links and app navigation layers.
US 2026/0195146 A1
Parsing visual layouts to locate buried cancellation options lets agents navigate sites designed to prevent user action without human intervention.
US 2026/0196188 A1
The storyline has focused on agents that parse tool calls and read images; this filing shows the agent deciding which tools to invoke based on visual input and learned user patterns, automating the decision layer that sits between perception and action.
US 2026/0197255 A1
Sequencing data operations automatically sidesteps the need for users to know statistical methods or tool syntax. The system maps dependencies between steps, then executes them in order while validating intermediate results.
US 2026/0194361 A1
Parsing map images as structured spatial data lets the agent build itineraries by understanding actual geography rather than collating web results, solving how to ground travel planning in visual layout instead of text.
US 2026/0195390 A1
Parsing user intent against site structure lets the agent bypass dark patterns by building a direct execution path, confirming the need for adversarial site navigation in the agent stack.
US 2026/0189649 A1
Coordinating actions across multiple apps requires Siri to parse which tool calls succeeded and failed, then report back on outcomes rather than leaving users guessing whether each handoff actually executed.
US 2026/0187941 A1
Ambient monitoring moves from passive observation to active prediction: the agent watches object trajectories and alerts you to hazards before your attention catches them.
US 2026/0189616 A1
Listening to spoken references during calls and auto-fetching the mentioned documents cuts the manual steps agents currently need to take when retrieving files on behalf of users.
US 2026/0186865 A1
Agents need to route requests across multiple services without human code. Nvidia's filing shows how to parse a plain-English query and map it to the right cloud API automatically.
US 2026/0188057 A1
Where agents need human judgment: Tesla's system moves the diagnostic filter upstream, letting the car decide what's worth reporting to service before a human ever enters the loop.
US 2026/0188312 A1
Your AI assistant can understand "check my flight" but can't phone the airline. Amazon's patent solves the translation layer that converts that intent into an actual API call.
US 2026/0186876 A1
Right now, when one app wants to talk to another, a developer has to write exact, unforgiving instructions in a specific format. Google is patenting a system that lets apps just describe what they want, and an AI figures out the rest.
US 2026/0186841 A1
Abstracting lab equipment behind code interfaces lets agents call physical instruments the way they'd invoke software functions, removing setup friction from experiment workflows.
US 2026/0187529 A1
Within the agent planning layer, this filing shows how to generate complete task maps upfront rather than step-by-step. The action graph approach lets agents evaluate alternative paths before committing to actions, reducing errors when controlling apps.
US 2026/0186801 A1
The storyline tracks how agents parse tool calls and control apps. This filing shows agents can now synthesize UI components by reading plain requests, then assemble live data feeds into custom layouts without pre-built templates.
US 2026/0189528 A1
Ambient sound capture in group settings confirms the storyline's direction toward agents that monitor multiple concurrent speakers rather than wait for explicit user queries to trigger searches.
US 2026/0186870 A1
Config files let non-programmers define when AI agents should run and what they should do with results, removing the coding barrier between user intent and multi-step automation workflows.
US 2026/0187143 A1
Where agents need to parse what's happening on screen in real time, Samsung adds automatic object recognition in video frames. This lets an agent watching ambient content identify search targets without waiting for explicit descriptions from the user.
US 2026/0187132 A1
The timeline so far shows AI agents moving from understanding requests to executing them. This filing shows they first need to extract the actual request from messy human communication, parsing intent before an agent can act on it.
US 2026/0187648 A1
Getting an AI to search through fragmented customer history across multiple channels and formats solves a basic blocker: agents can't act on incomplete context. Without this assembly work automated, AI stays locked out of real customer service decisions.
US 2026/0189421 A1
The storyline needs agents that extract consensus from group chats and execute the resulting decisions. This filing shows how to parse multi-party conversations for agreement signals, then route actions to the right apps.
US 2026/0188319 A1
Where agents need guardrails before executing: Samsung maps preconditions like device state and data availability upfront, catching dead-end requests before the assistant wastes cycles trying to fulfill them.
US 2026/0175868 A1
When autonomous vehicles hit edge cases, the filing shows AI chat systems as an alternative to human remote operators, letting the car query for guidance and keep moving rather than freeze.
US 2026/0175417 A1
Robots need to run basic navigation constantly while swapping in specialized skills for specific tasks. Amazon's approach layers a fixed movement controller under interchangeable task modules, so a robot can grab objects without forgetting how to walk.
US 2026/0179014 A1
Mining past support chats to auto-generate task workflows cuts the manual labor of coding agent instructions, letting the system learn execution patterns from what actually worked before.
US 2026/0169684 A1
Within the agents storyline, this filing shows how to collapse multi-step workflows into a single input stream, pointing and speaking at once so the AI can bind what you see to what you want, then route to the right app without intermediate steps.
US 2026/0169977 A1
A deliberately error-prone AI trains a second AI to spot mistakes before output reaches users, solving the confidence problem where agents deliver plausible-sounding wrong information in enterprise workflows.
US 2026/0170430 A1
Your support agent gets a ready-made action plan instead of searching docs or guessing. The real gain: the AI marks which steps it invented versus which came straight from company policy, so agents can spot and fix bad suggestions before executing them.
US 2026/0170437 A1
Where agents improvise solutions, this filing shows AI generating a structured resolution plan upfront. Moves the assistant from reactive helper to planner that sketches the path before human hands touch the case.
US 2026/0170067 A1
Keeping AI assistants from inventing answers requires training them to spot unanswerable questions before they confabulate. Salesforce's system flags gaps in available data so agents can defer to humans instead of acting on made-up information.
US 2026/0170021 A1
Running multiple competing viewpoints through one model before committing to action reduces the risk that an AI agent will execute flawed plans, a key requirement if these systems are meant to autonomously handle real customer problems.
US 2026/0169842 A1
Your AI agent could complete multi-app workflows without human inspection, since the system catches its own connection errors before data flows break things downstream.
US 2026/0172743 A1
Predicting speech intent from ambient sound alone removes the need for wake words, letting agents respond to natural conversation without explicit activation signals.
US 2026/0161273 A1
Getting an AI to reliably identify and interact with UI elements across different apps requires mapping what's clickable on screen in real time, which this patent addresses through numbered annotation.
US 2026/0161970 A1
Automatically executing actions tied to message replies collapses the gap between composing a response and acting on its implications, letting AI infer and perform the logical next steps without separate user commands.
US 2026/0161678 A1
Your support agent gets answers surfaced while the customer is still typing, cutting the search-and-paste step entirely. That shrinks the gap between understanding what someone needs and acting on it.
US 2026/0161723 A1
Your agent needs to know which buttons to click and which fields to fill on unfamiliar websites. Google's approach lets the browser itself figure out what's clickable and what matters, so you don't have to pre-teach it every site you visit.
US 2026/0161265 A1
Where agents need to move from understanding requests to executing them, this filing solves the mechanical problem of translating natural language commands into specific app interactions by having the AI visually parse and sequentially navigate UI elements.
US 2026/0155148 A1
Users can't act on AI recommendations without knowing the reasoning behind them. This filing proposes making that reasoning visible in plain language so people can verify whether the AI actually understood their situation correctly.
US 2026/0155136 A1
Real-time tone shifts based on conversational cues let agents match user state without explicit commands, moving assistants from static personas toward dynamic responsiveness that reads frustration or impatience and adjusts output accordingly.
US 2026/0148739 A1
Where agents need grounding: this patent solves the reference problem that keeps voice commands generic. An AI that knows what's on screen can act on spoken commands that point to specific objects without requiring users to name them explicitly.
US 2026/0140695 A1
Your agent could infer what you need from overheard conversation and offer ready-made actions, skipping the step where you figure out what app to open or command to give.
US 2026/0141406 A1
A monitoring agent runs parallel to deployed assistants, detecting policy violations in real time and revoking their access to sensitive data or functions before damage occurs. This moves compliance from post-deployment audits into active runtime enforcement.
US 2026/0141261 A1
Your agents can now run unsupervised at scale: Salesforce's system lets a compliance-focused AI monitor and halt other agents in real time, rather than waiting for human audits after decisions are made.
US 2026/0141896 A1
Understanding what action to take on extracted text requires analyzing context clues in images themselves, not just the words.
US 2026/0141903 A1
The storyline so far assumes agents need permission and oversight to act safely. This patent flips that: it stops agents from even receiving sensitive data in the first place, building a filter layer before the agent sees what you said.
US 2026/0141917 A1
Smart speakers currently can't distinguish direct speech from room echoes. Amazon's system maps physical objects in a space, then uses that geometry to filter out reflections and pinpoint where sound actually originated.
US 2026/0140980 A1
Storing conversations as retrievable chunks lets agents reference months of prior context without reloading entire chat histories, making multi-session tasks like trip planning actually continuous rather than perpetually restarted.
US 2026/0134866 A1
When an AI needs to act, it first has to plan what to do. Google's patent makes that planning step visible and trainable, so the system can learn better reasoning before it touches any tool.
US 2026/0127411 A1
A routing layer that separates intent-parsing from execution lets each app-specific agent operate independently, reducing the complexity of building a single AI that knows how to control dozens of different interfaces.
US 2026/0127385 A1
Agents with better judgment about which tasks to attempt reduces failed executions when AI moves from answering questions to actually executing commands on your behalf.
US 2026/0127232 A1
Shared devices need AI that weighs multiple people's needs instead of defaulting to the asker or splitting the difference. Google's patent adds a detection layer that identifies who's present and what they prefer, then adjusts recommendations accordingly.
Most of the filings here come from Google, and they map out a layered architecture: one model figures out what a user wants, another decides which app or tool to call, and a filtering step cleans up the prompt before it reaches the model. Other Google filings extend that reasoning to images, letting an assistant read text in a photo and act on it, and to memory, using embeddings so an agent recalls earlier conversations instead of starting fresh each time.
Amazon and Salesforce fill in the parts of the story about control and safety. Amazon patents cover a privacy monitor that flags Alexa skills collecting sensitive data, a system that fuses object maps with audio to locate the real speaker instead of an echo, and a way to tie voice commands to whatever is on screen so 'add that to my cart' resolves correctly. Salesforce contributes guardrail frameworks where one agent watches others for compliance violations and can shut down a rogue deployment mid-run.
Read together, the filings point toward assistants that act first and explain later, so watch for more patents on permissioning, since an agent that can control apps, read your screen, and remember your history needs clear limits on what it is allowed to do without asking. The guardrail and privacy filings suggest that policing these agents is becoming its own patent category, not an afterthought.
Not necessarily. A patent filing describes an invention a company wants to protect, not a shipped feature. Some ideas in this storyline, like app control layers or ambient listening, may show up in products later, get shelved, or never leave the lab. The filings tell us where engineering effort is going, not what you'll use next year.
Google files the most patents in this collection, covering intent detection, app control, prompt filtering, tool-call reasoning, image understanding, and memory. That spread suggests Google is building out a full stack for its assistants rather than one feature. It does not mean Google's approach will win, only that its patent output on this topic is currently the densest.
The recurring problems are figuring out what a user actually wants, deciding which app or tool to call, keeping a filtered and accurate prompt, recalling past interactions, and reacting to what a camera or microphone picks up. Amazon and Salesforce filings add privacy checks and compliance guardrails on top, since an assistant that acts needs limits on what it can do.
Salesforce's guardrail patents describe one AI agent watching others for compliance violations and blocking a rogue deployment before it causes harm. Paired with Amazon's privacy monitor for Alexa skills, they show that as assistants gain the ability to act, companies are also patenting ways to supervise and shut down agents that misbehave.
Which company should we read for you?
We track 17 companies here. Pro is the same weekly breakdown for any company you choose, delivered privately. Type a name and we'll scope it and send you a quote.