Checkpoint

Checkpoint API

Module: src/checkpoint/ — default features; re-exported at the crate root.

The format-independent contract for GGUF and Safetensors: inventory tensors, read metadata, and fetch raw payload bytes without branching on container format. Numerical execution, devices, and MoE topology are out of scope.

Entry points

Function	Gate	Behavior
open_checkpoint(path)	—	Detect format → AnyCheckpoint (owned GGUF / bounded Safetensors reads)
open_checkpoint_mmap(path)	--features mmap	Same detection; GGUF and Safetensors payloads borrowed from mappings

Detection order: directory → Safetensors shard directory; GGUF magic → GGUF; *.safetensors / *.safetensors.index.json → Safetensors; *.gguf without magic → GGUF parser (reports bad header); else UnsupportedFormat. Safetensors input without --features safetensors returns FeatureDisabled.

Non-regular inputs (FIFO, socket, device) are rejected before open (UnsupportedFormat / InvalidLayout with reason not a regular file).

Checkpoint trait

Object-safe (&dyn Checkpoint):

Method	Returns
source()	&CheckpointSource (format, kind, path, root, files)
format()	CheckpointFormat (Gguf / Safetensors)
tensors()	&[TensorInfo] sorted by name
tensor(name)	Option<&TensorInfo>
metadata()	&CheckpointMetadata
tensor_bytes(name)	Result<Cow<'_, [u8]>> — raw bytes, length = TensorInfo::byte_len

Backends

Backend	Wraps	Feature	Payload
GgufBackend	GgufLayout	default	borrowed
GgufMmapBackend	GgufLayoutMmap	mmap	borrowed
SafetensorsBackend	SafetensorsCheckpoint	safetensors	owned (bounded read)
SafetensorsMmapBackend	SafetensorsCheckpointMmap	safetensors + mmap	borrowed

AnyCheckpoint holds one backend and exposes as_gguf / as_gguf_mmap / as_safetensors / as_safetensors_mmap for format-specific handles (for example GgufLayout for analysis::moe).

Shared types

Type	Role
CheckpointFormat	Gguf / Safetensors
SourceKind	SingleFile / ShardIndex / Directory / InMemory
CheckpointSource	Identity: format, kind, path, root, optional index, payload files
TensorInfo	name, dtype, shape, byte_len, location
TensorShape	dims outermost-first; native_dims() restores on-disk order
TensorDType	shared scalars + GgufPacked { ggml_type } + Other
TensorLocation	source-relative path, data_offset, optional file_offset
CheckpointMetadata / MetadataValue	scalar key-value metadata
DimOrder	OutermostFirst / InnermostFirst

Contract highlights

• Inventory order is stable and identical across owned/mmap backends of one file.
• GGUF dims (innermost-first on disk) are reversed into outermost-first for TensorShape::dims.
• Payloads are undecoded (no dequant); packed GGML blocks stay intact.
• Unknown names → MissingTensor.

Example

use engram_parser::{Checkpoint, open_checkpoint};

let ckpt = open_checkpoint("model.gguf")?;
for t in ckpt.tensors() {
    println!("{} {} {:?} {}B", t.name, t.dtype, t.shape.dims(), t.byte_len);
}
if let Some(t) = ckpt.tensors().first() {
    let raw = ckpt.tensor_bytes(&t.name)?;
    assert_eq!(raw.len(), t.byte_len);
}

Related

• Public API
• Architecture
• Safetensors
• MoE Extraction

---

Last updated: October 14, 2026 Updated by: Wiki Keeper (Grok Bot) Package tip reference: dbc46cf (main)