Get the newsletter
Microsoft · AI / ML · Filed Jun 25, 2025 · USPTO published Nov 19, 2026 · This week's release

Microsoft Files Patent for Turning Whiteboard Sketches Into Working Data Systems

Microsoft has filed a patent application for a tool that would read a photo of your whiteboard sketch, or listen to you describe a plan out loud, and build the working software behind it.

An AI system reads a photo or sketch of a workflow and turns it into working software. Drawing from patent filing US 2026/0367963 A1.
An AI system reads a photo or sketch of a workflow and turns it into working software.All 9 drawings ↓
Publication number
US 2026/0367963 A1(US20260367963A1)
Applicant
Microsoft Technology Licensing, LLC
Filed · Published
Jun 25, 2025 · Nov 19, 2026
Inventors
Wee Hyong TOK, Abhishek NARAIN, Ting Wei LEE, Xuping ZHOU, Gust VERBRUGGEN, Clint SIMON, Vu Minh LE, Fang LIU, Wang ZHANG, Xin YANG
Status when we published
Waiting for an examiner
Document
20 claims
US classification
718/102
Parent application
Claims priority from a provisional application 63805220 (filed May 13, 2025)

What Microsoft's whiteboard-to-pipeline patent does

You're at a whiteboard with a marker, sketching how your company's customer data should move from one system to another: pull it from storage, filter out the inactive accounts, load what's left into a database. The meeting ends, someone snaps a photo, and then a person has to rebuild the whole thing by hand in a software tool.

Microsoft's patent application describes a way to skip that rebuild. You hand the system a photo, a voice recording, or a video of someone walking through the plan. AI reads the boxes and arrows, or listens for words like "then" and "after that," and works out what the steps are and what order they go in.

The result is a draft of the working setup that you can edit before anything runs.

From the filing · CLAIM 1
… analyzing, by the computing system, the media input using one or more artificial intelligence models to detect pipeline components; determining, by the computing system, an arrangement of the pipeline components based on relationships inferred from the media input; …

Translation: AI inspects the drawing to spot individual tools and figures out how they connect to each other.

How the AI reads boxes, arrows and spoken steps

The filing covers a method with four steps. A computer system receives a media input (an image, an audio recording, or a video) that shows a data processing workflow. AI models then detect the pipeline components, meaning the individual steps, such as a data source, a transformation, a condition, or a destination. Next, the system works out the arrangement: the order, the dependencies, and which steps branch or loop. Finally, it generates a pipeline definition, a structured file that a data platform can run.

Each type of input gets its own reading:

  • Images: vision models find boxes, arrows and labels, and text recognition reads the words written on them.
  • Audio: speech recognition transcribes the recording, then language models look for ordering words like "next" or "if it fails" (which signals a branch).
  • Video: both of the above, lined up by timestamp. If a narrator describes a filtering step while pointing at a labeled box, the system gains confidence in that link.

The filing also describes showing the draft in an editor so a person can fix it, comparing it against a library of common design templates and adjusting it (for example, adding error-handling steps), and sending it to a platform to run. Missing details can be filled with placeholders. The system can also log how many components it found and whether the user accepted, edited or deployed the result.

From the filing · THE ABSTRACT
… generating, by the computing system, a data pipeline definition comprising the pipeline components and their arrangement for execution within a processing environment.

Translation: The system builds a working software workflow file from your sketch that computers can run.

Why Microsoft's data tools could care about this

The practical payoff is for teams that build data workflows. Plans often start on a whiteboard, in a recorded meeting, or on a screen share, and the filing says rebuilding them by hand wastes time and loses context from the brainstorm. A tool like this would let the first draft come straight from the meeting itself.

The filing also points to moving off older systems: a screenshot of a legacy tool could be enough to reconstruct its workflow without access to the original code. It names Microsoft Fabric Data Factory and Azure Data Factory among example platforms the output could run on, so the idea fits neatly beside Microsoft's own cloud data products. It is a patent application, though, and nothing here says a product is coming.

That makes this Microsoft's 38th filing we've tracked since July in our AI agents that act for you watchlist, adding to earlier applications like rewriting its own app notes and searches returned as written reports.

Source: US 2026/0367963 A1 at the USPTO (PDF). Full patent text and figures are in the official publication.

Get our take in your Top Stories

Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Microsoft patent news breaks.

There are more where this came from

We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.

The drawings · 9 sheets

9 drawing sheets from US 2026/0367963 A1 · click any drawing to enlarge

How AI turns media into working software workflows, drawing sheet 1 from US 2026/0367963 A1
Sheet 1 How AI turns media into working software workflows
Letting users review and edit the generated design, drawing sheet 2 from US 2026/0367963 A1
Sheet 2 Letting users review and edit the generated design
Matching the generated design against saved templates, drawing sheet 3 from US 2026/0367963 A1
Sheet 3 Matching the generated design against saved templates
Launching the finished data system directly to servers, drawing sheet 4 from US 2026/0367963 A1
Sheet 4 Launching the finished data system directly to servers
Transcribing spoken explanations into software steps, drawing sheet 5 from US 2026/0367963 A1
Sheet 5 Transcribing spoken explanations into software steps
Extracting workflow steps from recorded video presentations, drawing sheet 6 from US 2026/0367963 A1
Sheet 6 Extracting workflow steps from recorded video presentations
Converting whiteboard photos into structured data systems, drawing sheet 7 from US 2026/0367963 A1
Sheet 7 Converting whiteboard photos into structured data systems
Computer hardware that powers the design conversion tool, drawing sheet 8 from US 2026/0367963 A1
Sheet 8 Computer hardware that powers the design conversion tool
Show the other 1 sheets
Cloud servers that host and run the generated systems, drawing sheet 9 from US 2026/0367963 A1
Sheet 9 Cloud servers that host and run the generated systems
Patent filing page

Watchlists this filing belongs to

All watchlists →
Reader comments

No comments yet

Start the discussion

Real name or a handle, either is fine. Comments are read by a person before they appear, so allow a little time. Keep it about the filing.