Skip to content

Opt-in weight integrity verification at load time (pinned SHA-256 hashes / manifest) #14784

Description

@UniversePeak

Is your feature request related to a problem? Please describe.

from_pretrained loads whatever weight files are on disk without any integrity verification at load time. Transport is content-addressed (LFS etag / Xet) and hf cache verify (huggingface_hub v0.32+) can re-check a cache directory against what the Hub currently serves, but both of those guarantees end before the actual load: a checkpoint that was tampered with after verification, a modified snapshot in a shared cache, or any local directory passed directly to from_pretrained is loaded as-is, and there is no way to pin approved weight hashes that get enforced when the model loads.

This matters more as automated pipelines and agents consume public checkpoints without a human in the loop. Recent work shows the threat is practical: backdoored latent world-model checkpoints can hijack downstream control while passing pre-deployment checks (arXiv 2609.15781, "When the World Lies", Sep 2026), and scanner-evading deserialization attacks on model files keep evolving (arXiv 2607.17503, "ShadowPickle", Jul 2026). Pinning a revision protects identity ("this is what was uploaded at that commit") but does not let a user detect that the bytes being loaded today differ from the ones they approved, and hf cache verify is a manual, cache-only step outside the load path.

Describe the solution you'd like.

An opt-in integrity check in the existing load path, e.g.:

pipe = DiffusionPipeline.from_pretrained(
    "org/model",
    expected_hashes={"unet/diffusion_pytorch_model.safetensors": "<sha256>", ...},
)

or equivalently integrity_manifest="path/to/manifest.json" (or a manifest discovered inside the local directory). Proposed behavior:

  • Before weight materialization, hash each weight file with SHA-256 and compare against the pinned value.
  • On mismatch, raise a clear dedicated error naming the file and both hashes; an optional flag could downgrade this to a warning for soft adoption.
  • While the files are open anyway, surface tensor-shape mismatches between checkpoint and config as a clear early error instead of a deep load-time failure.
  • Default behavior unchanged: no kwarg, no verification, no extra I/O.

Scope note: this verifies your approved bytes are the bytes being loaded (tampering, substitution, cache drift). It cannot certify that an upload you approved is benign — that would need external attestation and is out of scope.

Describe the alternatives you've considered.

  • Revision pinning — necessary but not sufficient: no detection of post-download tampering, and no contract for local directories.
  • hf cache verify — great post-hoc audit, but manual, cache-directory-only, checks against the Hub's current state rather than hashes the user pinned, and not wired into loading.
  • Verification in huggingface_hub itself — plausible home, but users express trust at from_pretrained, so surfacing the contract in diffusers would match where the decision is actually made; happy to be told the Hub is the better layer.
  • safetensors — removes arbitrary-code-execution risk on load, but says nothing about whether the weights are the ones the user expects.

Additional context.

Happy to work on this if maintainers think it fits. Concrete plan: a small additive change on the existing checkpoint-loading path plus regression tests — a known-clean checkpoint must load unchanged, and a tampered checkpoint (one flipped byte → SHA-256 mismatch) must raise the dedicated error; also report the false-positive rate on clean checkpoints.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions