Nvidia Patents a Risk Score for AI Models Downloaded From Untrusted Sources
An AI model downloaded from a stranger is really a program, and it can hide code that attacks the computer running it. Nvidia's filing describes a way to test-drive the model first and hand it a risk score.
What Nvidia's AI model risk score actually does
Every time a company downloads an AI model from an outside source, it is trusting a stranger's work to run on its own computers. Those model files can hide instructions that leak data, copy the model, or grind a server to a halt. Most security tools were built to catch viruses, so judging an AI model is a gap.
Nvidia's filing describes a checkpoint that sits in front of the door. It opens the model's files, picks out the individual building blocks inside, and runs them in a practice environment to see how they behave. Then it hands each one a damage likelihood score, graded high, medium or low.
The score can change over time. If researchers later report a new attack on a similar building block, or the model acts oddly once it's live, the score goes up and an alert can fire. A high score can trigger steps like locking the model in a restricted area or suspending its access.
… scanning the one or more executable files implementing the AI model to identify a target attribute of the AI model …
Translation: It inspects the downloaded code to spot specific features inside the artificial intelligence model.
How the system scans, tests and scores a model
The system takes in the executable files that make up an AI model (the packaged code and settings that let the model run), often in the ONNX format. A scanner parses the files and lists the model's attributes: its operators (the individual operations the model performs), shapes, instructions and operation codes.
For each attribute, it runs tests in a sandbox (an isolated practice environment tied to the computer that will host the model) and records behavior such as unexpected system calls, memory spikes and odd outputs. The score then blends three things:
- Behavior seen during testing
- History: how similar attributes showed up in past security incidents
- Resource impact: whether the model contains attributes that could be used to exhaust a machine's resources
Each piece gets a weight that can be tuned to where the model will be deployed. In the filing's example scale, 0 to 1, above 0.7 is high risk, 0.4 to 0.7 is medium, and below 0.4 is low. In the worked example, a text processing operator sitting at 0.5 jumps to 0.95 after a prompt injection flaw (tricking an AI into following hidden instructions) is reported against a very similar operator.
Once the model is live, usage data keeps adjusting the score. Past a threshold, the system raises an alert and can take a remedial action.
… generating, based on the test data, a damage likelihood score associated with the target attribute of the AI model.
Translation: It calculates a safety score that predicts how much harm the model might cause.
Why checking AI downloads matters for companies
If your employer builds products on AI models pulled from public libraries or outside vendors, a checkpoint like this could sit between the download and the live system. A bad model can put at risk not just the company but also the customers who use its apps. A single score and a risk label gives security teams a fast way to decide what to block or review.
Nvidia is best known for the chips that run AI models, so a tool that vets those models fits naturally beside its data center work. Keep in mind this is an application, and the examiner's first review rejected the claims as filed, which is routine and usually followed by amendments. The formulas and weights in the filing are examples, so a real product could look quite different.
Nvidia's 12th filing we've tracked in our AI guardrails work since July follows one scanning blueprints for security holes and a chatbot speech slider.
Nothing here needs new hardware. A sandbox, a file scanner and a scoring formula are all software, and a company that already checks uploaded models could add this as one more step in that routine.
The hard part is the homework. The score leans on a library of past attack cases and a feed of fresh vulnerability reports, and someone has to build both and keep them current. The weights in the filing's formulas are examples, so a real team would spend months tuning them against its own systems.
The shortest route to a product is a scanning step inside an existing AI platform, since the claims cover only a scan, a test and a score. That keeps the distance to a working feature short, though the quality of the data decides whether the scores mean anything.
Get our take in your Top Stories
Liked this breakdown? Add Patentlyze as a preferred source on Google, and our plain-English take shows up more often in your Top Stories the next time Nvidia patent news breaks.
There are more where this came from
We read every patent application Big Tech publishes and send you the ones worth knowing. Plain English, free, every week.
The drawings
9 drawing sheets from US 2026/0350174 A1 · click any drawing to enlarge
Show the other 1 sheets
Want this weekly breakdown for a company we don't cover? Patentlyze Pro →
Be the first to weigh in