MLSecOps: Securing Your Models and Their Data, Not Just Your Code

Your AppSec pipeline scans code, dependencies, and containers. It does not see the 7-billion-parameter model your team downloaded from Hugging Face last week. Or the fine-tuning dataset sitting in an S3 bucket with no access controls. The attack surface of your AI systems extends beyond what your current tooling can observe.

The attack surface extends beyond code

When your product depends on an LLMLarge Language ModelA neural network trained on large volumes of text that generates, summarizes, and transforms natural language., its behavior is determined by three things: the code, the model weights, and the training data. Your AppSec covers only the first.

MITRE ATLASAdversarial Threat Landscape for Artificial-Intelligence SystemsA knowledge base of adversary tactics and techniques targeting machine learning systems, maintained by MITRE Corporation. has been mapping this surface since 2022. Version 5.5.0 (March 2026) contains 167 attack techniques, 35 mitigations, and 57 real-world case studies. In October 2025, it listed 66. The documented surface more than doubled in five months. The 14 techniques added for agentic systems show that the threat is evolving faster than the frameworks documenting it.

The artifacts that AppSec ignores are production artifacts: weight files (.safetensors, .gguf, .pt), training and fine-tuning datasets, prompt templates, inference pipeline configurations. A backdoored model in production is as serious as a compromised dependency. Except nobody scans for it.

The model supply chain is the new critical vector

In March 2026, an attacker compromised the PyPI credentials of the LiteLLM maintainer — an inference proxy present in 36% of cloud environments with 95 million downloads per month. Two malicious versions (1.82.7 and 1.82.8) were published. The payload operated in three stages: credential harvesting (OpenAI, Anthropic, Azure, AWS keys), Kubernetes lateral movement, and a persistent backdoor. The incident lasted two hours. It was discovered because the payload contained a bug that triggered a fork bomb.

I have seen teams treat downloading a Hugging Face model with the same level of diligence as an npm install — that is, none. Since 2024, over 1,400 malicious models have been removed from the platform, which hosts 1.2 million. In May 2026, a fake repository “Open-OSS/privacy-filter” reached #1 trending with 244,000 downloads in 18 hours. It contained a PowerShell infostealer in its loader.py.

Data poisoning is the other vector. Anthropic, the UK AI Safety Institute, and the Alan Turing Institute showed in October 2025 that 250 documents are enough to inject a backdoor into an LLM, regardless of its size (from 600 million to 13 billion parameters). The number is nearly constant. It is not a percentage of the dataset. It is an absolute threshold.

95% traditional AppSec, 5% that changes everything

Caleb Sima, president of the AI Safety Initiative at the Cloud Security Alliance, frames it this way: a model is a file in an S3 bucket. Access controls, identity management, encryption at rest and in transit, dependency scanning — these are the same practices we have been applying for 15 years. Only about 5% of security controls are AI-specific.

I think he is right on the fundamentals. That is the message CISOs need to hear: you do not need to reinvent everything. But the remaining 5% are the 5% nobody covers — and those are the ones that change everything.

Those 5% cover model provenance (where does this .safetensors come from, who trained it, on what data), training data integrity (250 poisoned documents are enough), and the non-deterministic behavior of a system whose output depends on a prompt. You cannot write a unit test for an LLM the way you do for a REST API. The failure mode is different: it is not a crash, it is a drift.

HiddenLayer (2025) reports that 74% of surveyed organizations suffered an AI-related breach in the past year. And 83% of companies have no automated controls on their AI supply chain (CISO Marketplace, 2025). The gap between massive adoption and near-absent security is the window attackers are exploiting.

The building blocks of a trust chain exist

The tooling is young, but the primitives are there.

CycloneDXCycloneDXAn OWASP standard for software bill of materials that supports component inventory, vulnerability reporting, and VEX. ML-BOMMachine Learning Bill of MaterialsAn extension of the software bill of materials concept that inventories ML models, training datasets, and inference dependencies. (v1.7) extends the SBOMSoftware Bill of MaterialsA machine-readable inventory of the components and dependencies included in a software product. format to models, datasets, and ML dependencies (Why Your SBOM Does Not Protect You explores the four missing layers between inventory and a security decision). OWASPOpen Web Application Security ProjectA nonprofit foundation that produces standards, tools, and guidance for application security, including the OWASP Top 10 vulnerability rankings. has published an open-source AI BOM Generator that produces BOMs from Hugging Face. Adoption remains low: DarkReading calls AI BOMs “largely aspirational” in 2026. That is true. But traditional SBOMs were in the same state five years ago.

OpenSSFOpen Source Security FoundationA Linux Foundation project that coordinates cross-industry efforts to improve open-source software security. Model Signing, with contributions from Google, HiddenLayer, and NVIDIA, extends code signing to ML artifacts. The principle is the same as for signed container images: verify that the model you are running is the one you think you are running.

The EU AI Act (Article 15) requires that high-risk AI systems resist data poisoning, adversarial examples, and confidentiality attacks. The obligations take effect on August 2, 2026. The market is already moving: $844 million in MLSecOps acquisitions in 2024–2025. Palo Alto Networks acquired Protect AI, Cisco acquired Robust Intelligence for $400M, F5 acquired CalypsoAI.

Start with inventory, not with a framework

I give the same advice to every team that asks me where to begin.

Take inventory. Which models are running in production. Where they came from. Who downloaded them. Who validated them. If you cannot answer those four questions in under an hour, you have a blind spot that your AppSec does not cover.

Then apply the controls you already know: access controls on model buckets, dependency scanning on the inference pipeline’s Python packages, integrity verification of weight files. Those are Caleb Sima’s 95%. They are necessary and you know how to do them. The same evidence-chain logic applies to traditional artifacts (Building a DevSecOps Evidence Chain details the pattern).

For the remaining 5%, start with provenance. An internal registry of approved models, with verification hashes and traceable sources, covers more risk than a full MITRE ATLAS deployment that nobody will maintain.

Take the next model your team plans to deploy. Ask where the weights come from, what data it was trained on, and who validated that it does not contain a payload. If nobody can answer, you just found your first MLSecOps project.