Skill v1.0.1
currentAutomated scan100/100+4 new
name: model-supply-chain-security description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion. category: security risk: critical source: https://github.com/BagelHole/DevOps-Security-Agent-Skills source_repo: BagelHole/DevOps-Security-Agent-Skills source_type: community date_added: '2026-09-20' license: MIT license_source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/blob/main/LICENSE compatibility: Requires the relevant security tooling (scanners, vault CLIs) and an authorized scope for any active assessment. Docs-only; helper scripts and templates not bundled. metadata: author: devops-skills version: '1.0'
Model Supply Chain Security
Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.
When to Use This Skill
Use this skill when:
- Pulling pretrained models from public registries (Hugging Face, TensorFlow Hub)
- Building model-serving containers for production deployment
- Establishing trust policies for ML artifact promotion across environments
- Responding to supply chain incidents affecting ML dependencies
- Meeting SLSA or SOC2 compliance requirements for AI systems
Prerequisites
cosignv2+ installed for signing and verificationsyftfor SBOM generation of model-serving imagescraneorskopeofor OCI image inspection- Container registry with signature support (GHCR, ECR, ACR, Artifact Registry)
- CI/CD pipeline with provenance generation capability
Threats
- Poisoned pretrained weights or adapters
- Malicious model conversion tools or loaders
- Compromised build pipelines and registries
- Insecure runtime images with critical CVEs
- Typosquatting on model registries
- Deserialization attacks via pickle or custom loaders
Control Objectives
- Verify artifact integrity end-to-end
- Prove provenance for every promoted model
- Detect vulnerable dependencies before deploy
- Restrict execution to trusted signed artifacts
Model Signing with Cosign
Sign a Model Artifact
# Generate a keypair (store private key securely)cosign generate-key-pair# Sign an OCI-packaged model imagecosign sign --key cosign.key ghcr.io/acme/ml-models/sentiment:v2.1.0# Keyless signing with Sigstore (uses OIDC identity)cosign sign ghcr.io/acme/ml-models/sentiment:v2.1.0# Verify the signaturecosign verify --key cosign.pub ghcr.io/acme/ml-models/sentiment:v2.1.0# Keyless verification (requires certificate identity)cosign verify \--certificate-identity=ci-bot@acme.iam.gserviceaccount.com \--certificate-oidc-issuer=https://accounts.google.com \ghcr.io/acme/ml-models/sentiment:v2.1.0
Sign Model Weight Files Directly
# For model files stored as blobs (not OCI images)# Compute digest and signsha256sum model-weights.safetensors > model-weights.sha256cosign sign-blob --key cosign.key model-weights.safetensors \--output-signature model-weights.sig \--output-certificate model-weights.crt# Verify blob signaturecosign verify-blob --key cosign.pub \--signature model-weights.sig \model-weights.safetensors
SLSA for ML Pipelines
SLSA Level Requirements for Model Builds
# slsa-requirements.yamlslsa_levels:level_1:- Build process is scripted (not manual)- Provenance document generated automaticallylevel_2:- Build runs on hosted CI service- Provenance is authenticated (signed)- Source is version controlledlevel_3:- Build environment is ephemeral and isolated- Provenance is non-falsifiable (hardened builder)- Source integrity verified (two-person review)
Generate SLSA Provenance for Model Training
# .github/workflows/model-build-slsa.ymlname: Model Build with SLSA Provenanceon:push:tags: ['model-v*']jobs:train-and-package:runs-on: ubuntu-latestpermissions:id-token: writecontents: readpackages: writesteps:- uses: actions/checkout@v4- name: Train modelrun: python train.py --config configs/production.yaml- name: Package model as OCI artifactrun: |oras push ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }} \model-weights.safetensors:application/vnd.acme.model.safetensors \model-config.json:application/json- name: Generate SBOM for training environmentrun: |syft dir:. -o cyclonedx-json > training-sbom.json- name: Sign and attestrun: |cosign sign ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}cosign attest --predicate training-sbom.json \--type cyclonedx \ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}# Provenance must be generated by a supported reusable workflow invoked at jobs.<job>.uses,# not as a step. Add a separate dependent job with the actual immutable digest output# from an identified push step and the selected generator's required inputs.
Model Cards for Provenance
# model-card.yamlmodel_details:name: "sentiment-classifier-v2.1.0"version: "2.1.0"type: "text-classification"framework: "pytorch"license: "Apache-2.0"provenance:training_data:source: "s3://acme-datasets/sentiment-v3/"hash: "sha256:abc123..."data_card_ref: "https://internal.acme.com/data-cards/sentiment-v3"training_config:source: "git://github.com/acme/ml-models@abc123"hyperparameters:learning_rate: 0.00005epochs: 10batch_size: 32build_environment:builder: "github-actions"runner: "ubuntu-22.04"python: "3.11.7"torch: "2.1.2"cuda: "12.1"build_id: "gh-actions-12345"commit_sha: "abc123def456"build_timestamp: "2025-01-15T10:30:00Z"signed_by: "ci-bot@acme.iam.gserviceaccount.com"performance:accuracy: 0.94f1_score: 0.93evaluation_dataset: "s3://acme-datasets/sentiment-eval-v3/"evaluation_hash: "sha256:def456..."security:vulnerability_scan: "clean"sbom_ref: "ghcr.io/acme/ml-models/sentiment:v2.1.0.sbom"last_security_review: "2025-01-10"known_limitations:- "May produce biased outputs for underrepresented languages"- "Not evaluated for adversarial robustness"
Registry Scanning
# Scan model-serving image for CVEstrivy image ghcr.io/acme/ml-models/sentiment-serving:v2.1.0# Generate SBOM for the serving containersyft ghcr.io/acme/ml-models/sentiment-serving:v2.1.0 -o spdx-json > serving-sbom.json# Scan SBOM for vulnerabilitiesgrype sbom:serving-sbom.json --fail-on critical# Check for known-malicious model files (pickle scanning)pip install ficklingfickling --check model.pkl
Automated Registry Scan Pipeline
# .github/workflows/registry-scan.ymlname: Nightly Registry Scanon:schedule:- cron: '0 2 * * *'jobs:scan:runs-on: ubuntu-lateststrategy:matrix:image:- ghcr.io/acme/ml-models/sentiment-serving:latest- ghcr.io/acme/ml-models/embedding-serving:latest- ghcr.io/acme/ml-models/rag-api:lateststeps:- name: Scan imagerun: |trivy image --severity CRITICAL,HIGH \--exit-code 1 \--format json \--output scan-$(echo ${{ matrix.image }} | tr '/:' '-').json \${{ matrix.image }}- name: Verify signatures are still validrun: |cosign verify \--certificate-identity=ci-bot@acme.iam.gserviceaccount.com \--certificate-oidc-issuer=https://accounts.google.com \${{ matrix.image }}
Promotion Policy Enforcement
#!/usr/bin/env python3"""model_promotion_gate.py - Verify model meets all promotion criteria."""import subprocessimport jsonimport sysdef check_signature(image: str) -> bool:result = subprocess.run(["cosign", "verify", "--certificate-identity=ci-bot@acme.iam.gserviceaccount.com","--certificate-oidc-issuer=https://accounts.google.com", image],capture_output=True, text=True,)return result.returncode == 0def check_vulnerabilities(image: str) -> bool:result = subprocess.run(["trivy", "image", "--severity", "CRITICAL", "--exit-code", "1","--quiet", image],capture_output=True, text=True,)return result.returncode == 0def check_sbom_exists(image: str) -> bool:result = subprocess.run(["cosign", "verify-attestation", "--type", "cyclonedx","--certificate-identity=ci-bot@acme.iam.gserviceaccount.com","--certificate-oidc-issuer=https://accounts.google.com", image],capture_output=True, text=True,)return result.returncode == 0def check_model_card(image: str) -> bool:result = subprocess.run(["cosign", "verify-attestation", "--type", "custom","--certificate-identity=ci-bot@acme.iam.gserviceaccount.com","--certificate-oidc-issuer=https://accounts.google.com", image],capture_output=True, text=True,)return result.returncode == 0def main():image = sys.argv[1]checks = {"signature_valid": check_signature(image),"no_critical_cves": check_vulnerabilities(image),"sbom_attached": check_sbom_exists(image),"model_card_present": check_model_card(image),}all_passed = all(checks.values())for name, passed in checks.items():status = "PASS" if passed else "FAIL"print(f" [{status}] {name}")if not all_passed:print("Promotion BLOCKED: not all checks passed.")sys.exit(1)print("Illustrative checks passed; policy/content validation and authorized promotion still required.")if __name__ == "__main__":main()
Runtime Hardening
- Run inference containers as non-root.
- Apply egress restrictions to prevent unauthorized downloads.
- Mount model volumes read-only when possible.
- Alert on unsigned artifact pull attempts.
- Prefer safetensors weights to avoid pickle execution during weight decoding; custom model code, dependencies and parsers still require review.
# kubernetes deployment hardeningapiVersion: apps/v1kind: Deploymentmetadata:name: model-servingspec:template:spec:securityContext:runAsNonRoot: truerunAsUser: 1000fsGroup: 1000containers:- name: inferenceimage: ghcr.io/acme/ml-models/sentiment-serving:v2.1.0securityContext:readOnlyRootFilesystem: trueallowPrivilegeEscalation: falsecapabilities:drop: ["ALL"]volumeMounts:- name: model-weightsmountPath: /modelsreadOnly: trueresources:limits:memory: "4Gi"nvidia.com/gpu: "1"volumes:- name: model-weightspersistentVolumeClaim:claimName: model-weights-pvcreadOnly: true
Kyverno Policy for Admission Control
apiVersion: kyverno.io/v1kind: ClusterPolicymetadata:name: require-signed-model-imagesspec:validationFailureAction: Enforcerules:- name: verify-model-image-signaturematch:any:- resources:kinds: ["Pod"]namespaces: ["ml-serving"]verifyImages:- imageReferences: ["ghcr.io/acme/ml-models/*"]attestors:- entries:- keyless:subject: "ci-bot@acme.iam.gserviceaccount.com"issuer: "https://accounts.google.com"
Troubleshooting
| Problem | Cause | Solution | |
|---|---|---|---|
cosign verify fails with "no matching signatures" | Image was pushed without signing | Quarantine and investigate missing signatures; do not sign an untrusted artifact merely to make verification pass | |
| Provenance attestation missing | SLSA generator not configured | Add slsa-github-generator to the build workflow | |
| Trivy reports CVEs in base image | Stale base image | Update FROM image in Dockerfile; rebuild and re-sign | |
| Pickle deserialization warning | Model saved in unsafe format | Do not load suspicious pickle to convert it on a trusted host; obtain a verified safe source or use an isolated analysis environment | |
| Keyless verification fails | Wrong OIDC issuer or identity | Check --certificate-identity and --certificate-oidc-issuer flags | |
| Model card not found for artifact | Attestation not attached to digest | Attach with cosign attest --predicate model-card.yaml --type custom IMAGE |
Related Skills
- sbom-supply-chain (
sbom-supply-chain) - Generate SBOM and provenance evidence - container-hardening (
container-hardening) - Harden runtime container posture - model-registry-governance (
model-registry-governance) - Controlled lifecycle and approvals
Limitations
- Apply guidance only within authorized scope; test destructive steps in non-production first.
- Docs-only import: upstream scripts and templates not bundled.
Example
# Read-only first: inventory before any active step.which <tool> && <tool> --help | head -n 20
Adapted from BagelHole/DevOps-Security-Agent-Skills (MIT); frontmatter, When to Use/Limitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.
All artifacts, identities and scores are fictional examples. Resolve mutable tags to a digest once and verify, scan and promote that exact digest; otherwise checks can inspect different artifacts. Signatures prove signer/integrity, not safe weights or training data. Attestation existence is insufficient: decode and validate predicate schema, subject digest, builder/source/dependency policy, and substantive model-card contents. A generic custom attestation is not proof of a model card, and YAML is not automatically the JSON predicate required by a tool. The training workflow needs reviewed tool installation, registry login, actual push outputs and jobs-level reusable provenance wiring; the Kubernetes manifest is a partial fragment lacking selector/template labels. SLSA requirements depend on the spec version and track; two-person review is not a universal Build L3 requirement. Directory SBOM scans do not inventory every installed training dependency. Pin reviewed tools/actions and bound scanner/network/resource use.