Skip to content

Add Apple MPS NF4 benchmark - #2057

Open
hamedrabah wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
hamedrabah:hamedrabah/mps-nf4-benchmark
Open

Add Apple MPS NF4 benchmark#2057
hamedrabah wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
hamedrabah:hamedrabah/mps-nf4-benchmark

Conversation

@hamedrabah

Copy link
Copy Markdown

Summary

Adds a standalone Apple MPS benchmark for the now-supported NF4 path. It
measures:

  • NF4 quantization and dequantization median latency
  • packed tensor plus absmax metadata size relative to fp32
  • round-trip quantization error
  • Linear4bit latency and output error relative to dense fp16

The benchmarking README includes the command and the script exposes tensor
size, layer size, warm-up, and repetition controls. Results are reported rather
than asserted so the benchmark remains useful across Apple Silicon variants.

Validation

Tested on Apple M4 / macOS 15.7.9 with PyTorch 2.13.0 against the current
checkout (bitsandbytes 0.50.2.dev0):

$ PYTHONPATH=. python benchmarking/mps/nf4_benchmark.py --repeats 3 --warmups 1
device: Apple MPS (arm64)
torch: 2.13.0; bitsandbytes: 0.50.2.dev0
NF4 packed+absmax storage: 0.141x of fp32 (16.0 MiB -> 2.2 MiB)
quantize median: 7.950 ms
dequantize median: 1.110 ms
round-trip MAE: 0.072869; max error: 0.654016
dense fp16 GEMV median: 0.870 ms
NF4 Linear4bit median: 1.996 ms
dense-vs-NF4 output MAE: 3.361328

$ ruff check benchmarking/mps/nf4_benchmark.py
All checks passed!

$ ruff format --check benchmarking/mps/nf4_benchmark.py
1 file already formatted

The existing 4-bit test selection was also exercised on MPS: 252 passed, 36
skipped, 60 xfailed, and 426 deselected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant