Protect AI model scanning: Guardian is gone, ModelScan is not

Protect AI used to be a company you could buy model scanning from. Palo Alto Networks announced the acquisition on 28 April 2025 and completed it on 22 July 2025, so what is left to review is not a vendor but two artefacts: Prisma AIRS AI Model Security, the paid scanner that Guardian became, and ModelScan, an Apache-2.0 command-line tool that still installs from PyPI. The position of this review is that scanning a model file is a CI gate rather than a procurement category: the paid tier buys policy, provenance and audit trail, and nothing in the published material shows it detecting what the free tools miss.

A model file is executable code wearing a data-science extension, because loading one with PyTorch runs whatever the pickle inside asks for. The scanner therefore sits between the registry and the deploy step, at the same intake point as a marketplace tool or a package from a public index; the reasoning in the MCP security checklist applies to weights as well. The free competition is ModelAudit and Picklescan, and on the commercial side JFrog's research team published the PickleScan bypasses described further down.

What it is

ModelScan is a Python package with a single command: it reads a file byte by byte, looks for serialisation constructs that lead to code execution, and reports them by severity without ever loading the model. Guardian was the hosted and enterprise layer over that idea, scanning models in CI and in registries, enforcing policies and keeping an audit trail, and it no longer exists under that name. The capability is sold as Prisma AIRS AI Model Security inside the Prisma AIRS platform, which also carries runtime protection, red teaming and posture management.

• Vendor: Protect AI, now inside Palo Alto Networks; the acquisition completed on 22 July 2025
• Free component: ModelScan, Apache-2.0, PyPI version 0.8.8 released on 18 February 2026, Python 3.10 to 3.12
• Paid component: Prisma AIRS AI Model Security, 35+ file types and 25+ threat categories, price only on request
• Detection: unsafe serialisation constructs, embedded malicious code, backdoors and structural anomalies, rated CRITICAL to LOW
• Interface: modelscan -p PATH, console or JSON reporting, exit codes 0 to 4
• Placement: scans run inside your own environment so model files and IP stay local, according to the product page
• Guardian is gone: protectai.com/guardian returns 404, and the Hugging Face documentation page that describes it still links that dead address

For a team already paying for Prisma AIRS, model scanning is a checkbox on a platform it owns. For a team that is not, the same gate can be assembled from ModelScan and a CI step at no cost. That asymmetry, rather than any detection benchmark, is what this review turns on.

How it works

Static scanning avoids the very trap it defends against: nothing in the file is ever deserialised. The ModelScan README describes reading a file a byte at a time, like a string, looking for unsafe code signatures, which bounds both the runtime and the risk, since a scan takes about as long as reading the file from disk. Findings arrive as CRITICAL, HIGH, MEDIUM or LOW, and the process exit code carries the verdict into CI.

The paid tier adds what a file-by-file tool cannot see. Palo Alto Networks states that scans validate models against Advanced WildFire threat intelligence and findings from the huntr researcher community, that analysis runs in the customer's own environment, and that build systems integrate through an API; January 2026 added Artifactory and GitLab sources. Those are claims about intelligence, provenance and workflow, and none of them says the parser reads the file more accurately.

Static, not behavioural

Neither tier watches the model run. No scanner can tell you that a layer activates only for a particular input distribution, or that accuracy was quietly degraded for one class of user; that requires evaluation against your own data and, for agents, runtime controls of the kind described in the AI agent sandbox checklist. Static analysis answers one question: does this file contain constructs that execute when it is loaded. Answering it well is worth a great deal, as long as nobody reads no findings as safe.

Supported formats

Format coverage is where the free tool and the paid product diverge most, and where the free tool is at least honest about its limits.

The gap that matters is not a missing format but an unlisted one: a scanner cannot scan what it does not recognise, and an unrecognised file has to fail closed rather than pass quietly. ModelScan handles that correctly by returning exit code 3 when it is given no supported files. The question to ask of any vendor's format list is what happens to the thirty-sixth file type.

Getting started

The free path takes two commands: install the package, point it at a directory. What matters in CI is not the text output but the exit code, because 0 means clean, 1 means findings, and 2, 3 and 4 mean the scan did not really happen.

# CI gate: scan every model before it reaches the registry
pip install "modelscan[tensorflow,h5py]"

modelscan --path ./models --reporting-format json --output-file scan.json
status=$?

# 0 clean, 1 findings, 2 scan failed, 3 no supported files, 4 usage error
if [ "$status" -ne 0 ]; then
  echo "modelscan returned $status: refusing to publish"
  exit 1
fi

Treat 2, 3 and 4 as failures. A scanner that crashes on a malformed archive, or skips a file type it does not know, prints something very close to a clean run unless the pipeline is built to notice. The PickleScan bypasses published by JFrog had exactly this shape: a file crafted so that the scanner errored or took another path, while PyTorch loaded it without complaint.

What survived

State it plainly: Guardian does not exist as a product any more. protectai.com/guardian returns 404, protectai.com itself now serves the Prisma AIRS pages, and the only first-party description left is a Hugging Face documentation page that still links the dead URL. The acquisition was announced on 28 April 2025 and completed on 22 July 2025, and the scanning capability now appears as Prisma AIRS AI Model Security, with Artifactory, GitLab and cloud-storage sources added in January 2026.

ModelScan survived intact. The repository still sits under the protectai organisation under Apache-2.0 with 780 stars, and version 0.8.8 on PyPI is dated 18 February 2026. Its README still advises readers to consider Guardian and links to the dead page, which is as good a signal as any about the project's maintenance temperature: functional, quietly neglected, not abandoned.

The commercial side has the same shape at a different address. There is no public price, only a demo request, and a number that arrives after a sales conversation. That is ordinary for security software; for an engineer costing a CI gate it means the paid tier has to beat free tools by enough to justify a procurement, and no published measurement shows that.

Where it shingles

The weaknesses first. ModelScan covers three format families while free competition covers dozens. Reading a file statically says nothing about behaviour. There is no independent coverage measurement from the vendor, and the one independent benchmark available is unflattering: an August 2026 study of 135 labelled pickle and PyTorch families found ModelScan reached a definitive verdict on 49.6% of them, against 100% for ModelAudit and 81.5% for Fickling. When ModelScan did decide, its precision, recall and F1 were all 100%. And the paid tier has no price you can look up.

Read that third column as a failure-mode warning rather than a league table. A tool that returns a verdict on half of what it is shown, and is perfect on that half, is a tool that stays silent the rest of the time, and silence is what a CI gate reports as success. The same benchmark found that in the 48 malicious families ModelScan failed to analyse, ModelAudit and Fickling both detected the payload, while Fickling found no true positives beyond what the other two already covered. The conclusion is to run two scanners, not to buy a fifth.

Verdict

Protect AI's scanning is worth buying in its paid form only if Prisma AIRS is already on the purchase order; bought alone, it is a policy and audit layer priced like a platform. In its free form it is a narrow, useful CI gate that should never be the only one. The engineering judgement this review commits to is that model scanning is infrastructure, like a linter: it should be cheap, offline, exit-code driven and boring. A product whose main argument is a threat-intelligence feed is arguing about a different layer.

1. Use ModelScan if you need a free, offline gate over pickle, TensorFlow and Keras files and you treat any non-zero exit code as a build failure
2. Use Prisma AIRS AI Model Security if provenance, policy enforcement and an audit trail are things your auditors already ask about; detection alone does not justify the price
3. Do not adopt either as your only control: the coverage numbers say a second scanner and a fail-closed rule matter more than the brand on the box
4. Do not plan around Guardian: the product name is retired, the URL is dead, and any migration advice that mentions it is out of date
5. Prefer safetensors and other non-executable formats where you can, because the strongest model-scanning policy is not needing the deserialiser at all

A scanner that finishes only half of what it is shown is not a gate. It is a speed bump with a green light.

Sources

1. Palo Alto Networks: completes acquisition of Protect AI
2. Prisma AIRS AI Model Security product page
3. Prisma AIRS platform page served at protectai.com
4. ModelScan repository and README
5. ModelScan on PyPI
6. ModelAudit repository and README
7. promptfoo documentation: model scanning
8. Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners
9. JFrog: three zero-day PickleScan vulnerabilities
10. PickleScan repository and README
11. Hugging Face Hub docs: third-party scanner Protect AI