Microsoft Files Patent for Turning Whiteboard Sketches Into Working Data Systems
Microsoft has filed a patent application for a tool that would read a photo of your whiteboard sketch, or listen to you describe a plan out loud, and build the working software behind it.

- Publication number
- US 2026/0367963 A1(US20260367963A1)
- Applicant
- Microsoft Technology Licensing, LLC
- Filed · Published
- Jun 25, 2025 · Nov 19, 2026
- Inventors
- Wee Hyong TOK, Abhishek NARAIN, Ting Wei LEE, Xuping ZHOU, Gust VERBRUGGEN, Clint SIMON, Vu Minh LE, Fang LIU, Wang ZHANG, Xin YANG
- Status when we published
- Waiting for an examiner
- Document
- 20 claims
- US classification
- 718/102
- Parent application
- Claims priority from a provisional application 63805220 (filed May 13, 2025)
What Microsoft's whiteboard-to-pipeline patent does
You're at a whiteboard with a marker, sketching how your company's customer data should move from one system to another: pull it from storage, filter out the inactive accounts, load what's left into a database. The meeting ends, someone snaps a photo, and then a person has to rebuild the whole thing by hand in a software tool.
Microsoft's patent application describes a way to skip that rebuild. You hand the system a photo, a voice recording, or a video of someone walking through the plan. AI reads the boxes and arrows, or listens for words like "then" and "after that," and works out what the steps are and what order they go in.
The result is a draft of the working setup that you can edit before anything runs.
… analyzing, by the computing system, the media input using one or more artificial intelligence models to detect pipeline components; determining, by the computing system, an arrangement of the pipeline components based on relationships inferred from the media input; …
Translation: AI inspects the drawing to spot individual tools and figures out how they connect to each other.
How the AI reads boxes, arrows and spoken steps
The filing covers a method with four steps. A computer system receives a media input (an image, an audio recording, or a video) that shows a data processing workflow. AI models then detect the pipeline components, meaning the individual steps, such as a data source, a transformation, a condition, or a destination. Next, the system works out the arrangement: the order, the dependencies, and which steps branch or loop. Finally, it generates a pipeline definition, a structured file that a data platform can run.
Each type of input gets its own reading:
- Images: vision models find boxes, arrows and labels, and text recognition reads the words written on them.
- Audio: speech recognition transcribes the recording, then language models look for ordering words like "next" or "if it fails" (which signals a branch).
- Video: both of the above, lined up by timestamp. If a narrator describes a filtering step while pointing at a labeled box, the system gains confidence in that link.
The filing also describes showing the draft in an editor so a person can fix it, comparing it against a library of common design templates and adjusting it (for example, adding error-handling steps), and sending it to a platform to run. Missing details can be filled with placeholders. The system can also log how many components it found and whether the user accepted, edited or deployed the result.
… generating, by the computing system, a data pipeline definition comprising the pipeline components and their arrangement for execution within a processing environment.
Translation: The system builds a working software workflow file from your sketch that computers can run.
Why Microsoft's data tools could care about this
The practical payoff is for teams that build data workflows. Plans often start on a whiteboard, in a recorded meeting, or on a screen share, and the filing says rebuilding them by hand wastes time and loses context from the brainstorm. A tool like this would let the first draft come straight from the meeting itself.
The filing also points to moving off older systems: a screenshot of a legacy tool could be enough to reconstruct its workflow without access to the original code. It names Microsoft Fabric Data Factory and Azure Data Factory among example platforms the output could run on, so the idea fits neatly beside Microsoft's own cloud data products. It is a patent application, though, and nothing here says a product is coming.
That makes this Microsoft's 38th filing we've tracked since July in our AI agents that act for you watchlist, adding to earlier applications like rewriting its own app notes and searches returned as written reports.
Source: US 2026/0367963 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings · 9 sheets
9 drawing sheets from US 2026/0367963 A1 · click any drawing to enlarge

No comments yet