Skip to content

gemv: coalesce batched DMA into a single iterated BD per column#127

Open
atassis wants to merge 4 commits into
amd:develfrom
atassis:gemv-coalesce
Open

gemv: coalesce batched DMA into a single iterated BD per column#127
atassis wants to merge 4 commits into
amd:develfrom
atassis:gemv-coalesce

Conversation

@atassis

@atassis atassis commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Replaces the GEMV operator's per-batch host DMA descriptor unroll with a single iterated descriptor per column.

GEMM already folds many transfers into one multi-dimensional iterated descriptor (via TensorTiler2D); GEMV did not, so a batched GEMV emitted num_batches separate fill/drain descriptors (plus a per-batch task-group wait) per AIE column, linear in the batch count. The test suite also had no num_batches > 1 GEMV coverage, so the batched path shipped untested.

Added

  • A num_batches > 1 golden reference (generate_golden_reference_batched) and a parametrized device test (test_gemv_batched) covering: the coalesced path with large num_batches (the size-uncapped descriptor dimension) and a multi-dimension run split; a run that requires an alignment-aware (even) inner split; the per-batch fallback (a batch stride over the limit); and an attention-style shape with tile_size_input > 1 and num_batches = a head count.

Changed

  • The GEMV runtime now coalesces the batched A-fill / C-drain into one iterated descriptor per column by default. num_batches is placed in the size-uncapped descriptor dimension and the contiguous per-batch run is split across the two wrap dimensions (each <= 1023), per the AIE shim BD limits (AIEXDialect.cpp verifyStridesWraps). The run split keeps the innermost size and the strides 4-byte-aligned (the shim's address granularity, which is enforced even for linear transfers).
  • The per-batch drain wait is replaced by ObjectFifo backpressure. Depth 2 is sufficient (rather than depth num_batches): the A-fill (MM2S) and C-drain (S2MM) run on separate shim channels and the pipeline is paced by the fifo locks, so the producer cannot overrun the consumer; this is a streaming pipeline, not a store-all buffer. The depth invariant is asserted.
  • The coalesced descriptor accesses the exact same DRAM elements in the same order as the previous unroll. num_batches == 1, and any configuration that cannot be coalesced (a run with no aligned split under the wrap cap, or a batch stride that is too large or not aligned), fall back to the existing per-batch path and are byte-identical. The coalesced path therefore introduces no new failure modes relative to the unroll.

Removed

  • Nothing (behavior for existing callers is unchanged).

Motivation and measurements

This is a descriptor-count / build-time and correctness change, not a runtime speedup: the transfer is access-equivalent (same bytes, same order), so it does not change data-movement time.

The descriptor count for a batched GEMV drops from O(num_batches) to O(1) per column, which shrinks the descriptor-bound lowering phase of the build. Measured cold compile of a single GEMV (shape M=448, K=64, columns=8, build only, no device run):

num_batches unrolled coalesced
192 5.99s 4.72s
1536 52.37s 3.54s

The coalesced build is roughly constant in num_batches while the unrolled build grows linearly, so the saving scales with the batch count.

Validation

  • Offline access-equivalence of the coalesced vs unrolled descriptor for the tested and fallback configurations, including the alignment-aware split.
  • num_batches == 1 generates byte-identical lowered output to the previous code.
  • The new device tests pass on NPU2 at num_batches of 4, 8, 32, 100, and 192, plus the aligned-split shape (M=1026), the fallback shape (batch stride > limit), and an attention-style shape (tile_size_input=4); the existing num_batches == 1 tests are unchanged.
  • Exercised end to end: the in-repo llama_3.2_1b application, which calls GEMV with num_batches = n_heads, builds and generates coherent output on NPU2 with the coalesced path.
  • In-repo GEMV consumers were checked: swiglu_decode uses num_batches=1 (byte-identical, unaffected); the llama_3.2_1b application calls GEMV with num_batches = n_heads for the attention scores/context GEMVs - those now coalesce, and the coalesced descriptor is legal and access-equivalent at those shapes, so behavior is unchanged.

PR Merge Checklist

  1. The PR is rebased on the latest devel commit and pointing to devel.
  2. Your PR has been reviewed and approved.
  3. All checks are passing.

andrej
andrej requested changes Jun 29, 2026
View reviewed changes

@andrej andrej left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looking good! Reducing the number of runtime sequence instructions seems useful and the performance gain seems significant.

I only have some questions about the tensor access patterns. I have a feeling / hope that this could be radically simplified if indeed they are just linear transfers.

Comment thread iron/operators/gemv/design.py Outdated
Comment on lines +219 to +220
# Backpressure replaces the per-batch drain wait, so the A/C ObjectFifos must
# be deep enough (>=2) for the producer not to overrun the consumer.

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by the producer overrunning the consumer? Proper lock synchronization and backpressure should i.m.o. lead to at most a deadlock, and "overrunning" (buffers?) sounds concerning.

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, "overrun" is a bad word for it, sorry. The ObjectFifo lock backpressures, so a fast producer just blocks (worst case it stalls), it can't corrupt anything. depth>=2 is only there to overlap the fill with compute, so it's a perf thing, not correctness. Reworded the comment.

Comment thread iron/operators/gemv/design.py Outdated
# dims have a size cap (1023) while one dim is size-uncapped; TAP sizes are
# outermost-first and the verifier reverses them, so [1, num_batches, run_hi,
# run_lo] places num_batches in the uncapped dim and the contiguous run in the two
# wrap dims. Access-equivalent to the unroll (covered by test_gemv_batched).

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# wrap dims. Access-equivalent to the unroll (covered by test_gemv_batched).
# wrap dims.

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Also dropped the duplicated sentence.

Comment thread iron/operators/gemv/design.py
Comment on lines +186 to +194
def split_run(run, lim=MAX_WRAP, gran=GRAN_ELEMS):
"""Factor a contiguous run into (hi, lo), both <= lim and lo a multiple of gran
(the address-granularity-aligned inner size), lo maximal. None if no such
split exists (caller then falls back to the per-batch path)."""
lo_start = (lim // gran) * gran
for lo in range(lo_start, 0, -gran):
if run % lo == 0 and (run // lo) <= lim:
return (run // lo, lo)
return None

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atassis Does this split up a purely linear, contiguous transfer? There is a special case to not use data layout transformation dimensions at all. If left off, you should be able to specify only transfer lengths, and no dimensions.

@hunhoffe My understanding is this splits up a contiguous transfer into multiple dimensions. Is this logic still required after MLIR-AIE PRs #2924 and #3036?

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So within a batch the run is contiguous, but the batch stride is the whole matrix (M*K) and each column only takes its (M//cols)*K slice, so for cols>1 the batches have gaps between them - it's linear only when cols==1. On #2924/#3036: yeah those would let me drop the inner split, but we pin mlir_aie e4f35d6 (Mar 31) and both landed after that (Apr 7 / May 11), so on this pin the split is still needed. I left a note that it can become [num_batches, A_run] once the pin moves past #3036.

@andrej andrej Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atassis Just updated the pinned MLIR-AIE. Can that simplify this?

Comment thread iron/operators/gemv/design.py
Comment on lines +182 to +184
MAX_WRAP = 1023
MAX_STRIDE = (1 << 20) - 1 # conservative element-stride bound for the wrap dims
GRAN_ELEMS = 2 # 4-byte shim granularity / 2-byte bf16 element

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
MAX_WRAP = 1023
MAX_STRIDE = (1 << 20) - 1 # conservative element-stride bound for the wrap dims
GRAN_ELEMS = 2 # 4-byte shim granularity / 2-byte bf16 element
MAX_WRAP = 1023
MAX_STRIDE = (1 << 20) - 1 # conservative element-stride bound for the wrap dims
GRAN_ELEMS = 2 # 4-byte shim granularity / 2-byte bf16 element
# FIXME: Pull this information in to the target model in MLIR-AIE; also used in https://github.com/Xilinx/mlir-aie/blob/58ad59e8b3a601f540a653d1222d1dfc6303d3b0/lib/Dialect/AIEX/IR/AIEXDialect.cpp#L191-L199

(No need to do in this PR but let's leave the comment as a future to do)

Reference

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the FIXME pointing at verifyStridesWraps in AIEXDialect.cpp. Kept the constants, still needed on this pin.

Comment thread iron/operators/gemv/design.py
Comment on lines +211 to +216
return TensorAccessPattern(
tensor_dims=L3_ty.__args__[0],
offset=col_off,
sizes=[1, num_batches, run_hi, run_lo],
strides=[0, bstride, run_lo, 1],
)

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't spend too much time thinking through this but at a glance this looks like it might just be a linear transfer (if bstride==run_hi*run_lo). If that's the case, a lot of this code can be deleted and you should be able to just not supply any TAP to the rt.fill at all, which will equal a linear transfer.

If the transfer is more complicated, please explain in a comment! Thanks

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I checked that - bstride is M*K and the run is (M//cols)*K, so they're equal only when cols==1. For cols>1 it's a real strided gather, that's why the batch dim has to stay. The run_hi/run_lo split is just to get under the 1023 wrap cap (more in the other thread). So I explained it in a comment instead of deleting.

Comment thread iron/operators/gemv/design.py Outdated
Comment on lines +258 to +276
# Fallback (also the num_batches==1 path): stock per-batch unroll.
for batch in range(num_batches):
tg_ac = rt.task_group()
for col in range(cols):
rt.fill(
A_L3L1_fifos[col].prod(),
A,
A_taps[col][batch],
task_group=tg_ac,
)
for col in range(cols):
rt.drain(
C_L1L3_fifos[col].cons(),
C,
C_taps[col][batch],
task_group=tg_ac,
wait=True,
)
rt.finish_task_group(tg_ac)

@andrej andrej Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is essentially identical to the if-branch above except for number of wait/finish_task_group calls, could we reduce code duplication by instead looping over a new variable num_waits, which in this else-case would be num_waits==num_batches and in the if-case would be num_waits==1, and then also swap out the TAPs as appropriate.

@atassis atassis Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - one loop over num_waits now (1 for coalesced, num_batches otherwise), and swap the tap.

@andrej andrej Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! thanks, I prefer that

@github-actions

github-actions Bot commented Jun 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CI Test Results

3ab7ee2 (2026_06_29_17_34_01)

IRON - CI Summary

Examples

iron/applications/llama_3.2_1b
Test Krackan Status Krackan Phoenix Status Phoenix
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40] ✅ - - -

Small

iron/operators/axpy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] ✅ 149.48 ✅ 330.04
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] ✅ 169.60 ✅ 541.48
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] ✅ 188.22 ✅ 742.10
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0] ✅ 235.98 - -
iron/operators/dequant
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] ✅ 169.22 ✅ 418.92
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] ✅ 187.02 ✅ 389.32
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] ✅ 178.00 ✅ 449.10
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] ✅ 212.26 ✅ 350.90
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] ✅ 206.92 ✅ 785.72
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] ✅ 206.00 ✅ 550.26
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32] ✅ 225.78 - -
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32] ✅ 240.78 - -
iron/operators/elementwise_add
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 145.86 ✅ 388.78
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 172.22 ✅ 547.24
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 174.38 ✅ 403.74
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 243.62 - -
iron/operators/elementwise_mul
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 185.88 ✅ 379.36
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 186.52 ✅ 515.16
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 199.26 ✅ 511.58
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 212.04 - -
iron/operators/gelu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 151.44 ✅ 413.60
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 156.54 ✅ 466.42
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 154.18 ✅ 360.06
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 152.82 ✅ 488.12
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 185.50 ✅ 427.60
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 191.46 ✅ 522.22
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 190.82 - -
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 226.44 - -
iron/operators/gemm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1] ✅ 2287.60 - -
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 235.84 ✅ 797.08
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 252.76 ✅ 577.68
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 48728.78 ✅ 83147.14
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 28685.16 ✅ 24942.66
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 7808.30 - -
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] ✅ 2459.14 ✅ 3247.08
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] ✅ 3514.02 ✅ 5654.94
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1] ✅ 1537.88 - -
iron/operators/gemv
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] ✅ 0.23 ✅ 0.09
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] ✅ 12.66 ✅ 3.67
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] ✅ 23.66 ✅ 6.18
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] ✅ 39.66 ✅ 10.03
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256] ✅ 41.45 - -
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] ✅ 12.50 ✅ 3.68
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] ✅ 24.21 ✅ 6.40
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] ✅ 38.54 ✅ 8.79
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024] ✅ 40.98 - -
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2] ✅ - ✅ -
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2] ✅ - ✅ -
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4] ✅ - ✅ -
test_gemv_batched[M_256-K_128-num_aie_columns_8-tile_size_input_1-tile_size_output_32-num_batches_100] ✅ - - -
test_gemv_batched[M_448-K_64-num_aie_columns_8-tile_size_input_1-tile_size_output_56-num_batches_192] ✅ - - -
test_gemv_batched[M_512-K_64-num_aie_columns_8-tile_size_input_4-tile_size_output_64-num_batches_32] ✅ - - -
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8] ✅ - ✅ -
iron/operators/layer_norm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 151.42 ✅ 354.44
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 158.80 ✅ 341.88
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 194.46 ✅ 389.20
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 179.48 ✅ 378.54
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 160.08 ✅ 680.74
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 207.24 ✅ 492.38
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 172.18 - -
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 187.20 - -
iron/operators/mem_copy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] ✅ 164.20 ✅ 359.64
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128] ✅ 208.76 - -
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] ✅ 147.66 ✅ 411.70
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] ✅ 171.88 ✅ 393.60
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] ✅ 183.08 ✅ 555.62
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] ✅ 179.70 ✅ 363.40
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256] ✅ 187.84 - -
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] ✅ 204.72 ✅ 507.60
iron/operators/mha
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0] ✅ 40876.74 - -
iron/operators/relu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 149.02 ✅ 374.32
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 173.82 ✅ 711.18
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 173.10 ✅ 522.32
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 154.94 ✅ 464.42
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 182.70 ✅ 439.34
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 176.94 ✅ 576.10
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 168.96 - -
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 225.12 - -
iron/operators/rms_norm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False] ✅ 161.54 ✅ 426.86
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True] ✅ 193.96 ✅ 422.84
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False] ✅ 168.26 ✅ 462.26
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True] ✅ 189.44 ✅ 477.58
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False] ✅ 154.40 ✅ 368.66
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True] ✅ 200.76 ✅ 532.22
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False] ✅ 164.48 ✅ 419.42
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True] ✅ 232.00 ✅ 508.66
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] ✅ 178.80 ✅ 441.26
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True] ✅ 174.28 ✅ 472.16
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False] ✅ 188.88 ✅ 819.16
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True] ✅ 213.42 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False] ✅ 216.90 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True] ✅ 214.74 - -
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False] ✅ 276.28 - -
iron/operators/rope
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0] ✅ 176.58 ✅ 358.70
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0] ✅ 192.38 ✅ 435.78
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0] ✅ 182.02 ✅ 338.24
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0] ✅ 176.80 - -
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0] ✅ 202.18 ✅ 463.44
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0] ✅ 176.78 ✅ 710.66
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0] ✅ 184.58 ✅ 529.46
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0] ✅ 236.48 - -
iron/operators/sigmoid
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 176.34 ✅ 335.46
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 197.28 ✅ 313.90
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 185.52 ✅ 321.72
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 198.70 ✅ 480.42
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 165.06 ✅ 319.42
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 232.86 ✅ 460.54
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 244.26 - -
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 259.84 - -
iron/operators/silu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 199.20 ✅ 469.18
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 198.46 ✅ 289.16
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 196.48 ✅ 345.10
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 223.80 - -
iron/operators/softmax
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024] ✅ 166.14 ✅ 439.10
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048] ✅ 159.66 ✅ 385.22
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 170.16 ✅ 349.20
iron/operators/swiglu_decode
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] ✅ 4017.51 ✅ 16040.39
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] ✅ 3696.35 ✅ 13460.43
iron/operators/swiglu_prefill
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] ✅ 10503.01 ✅ 22942.67
iron/operators/tanh
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 153.36 ✅ 1133.22
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 154.70 ✅ 407.74
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 186.32 ✅ 394.52
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 160.72 ✅ 452.24
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 197.30 ✅ 526.54
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 163.84 ✅ 369.56
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 181.66 - -
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 221.58 - -
iron/operators/transpose
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1] ✅ 143.16 ✅ 668.90
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2] ✅ 175.42 ✅ 1120.96
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1] ✅ 142.58 ✅ 504.30
Krackan - Small

IRON

Tested on 2026_06_29_17_34_01 at commit 3ab7ee2.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5149.480.08n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5169.600.07n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5188.220.07n/a
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]✅ 5/5235.980.06n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5169.220.03n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5187.020.03n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5178.000.03n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5212.260.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5206.920.03n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5206.000.03n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]✅ 5/5225.780.02n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]✅ 5/5240.780.02n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5145.860.09n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5172.220.07n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5174.380.07n/a
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5243.620.05n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5185.880.07n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5186.520.07n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5199.260.06n/a
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5212.040.06n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5151.440.05n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5156.540.05n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5154.180.05n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5152.820.05n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5185.500.05n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5191.460.04n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5190.820.04n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5226.440.04n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]✅ 5/52287.604.111617.51
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5235.841.1047.13
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5252.760.9038.53
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/548728.780.52352.56
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/528685.160.88598.96
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/57808.303.222201.34
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/52459.143.31867.79
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/53514.020.3820.26
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]✅ 5/51537.884.591419.33
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.230.22
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a12.6612.66
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a23.6623.64
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a39.6639.64
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]✅ 5/5n/a41.4541.43
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a12.5012.50
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a24.2124.20
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a38.5438.52
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a40.9840.95
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]✅ 5/5n/an/an/a
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]✅ 5/5n/an/an/a
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]✅ 5/5n/an/an/a
test_gemv_batched[M_256-K_128-num_aie_columns_8-tile_size_input_1-tile_size_output_32-num_batches_100]✅ 5/5n/an/an/a
test_gemv_batched[M_448-K_64-num_aie_columns_8-tile_size_input_1-tile_size_output_56-num_batches_192]✅ 5/5n/an/an/a
test_gemv_batched[M_512-K_64-num_aie_columns_8-tile_size_input_4-tile_size_output_64-num_batches_32]✅ 5/5n/an/an/a
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]✅ 5/5n/an/an/a
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5151.420.06n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5158.800.05n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5194.460.04n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5179.480.05n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5160.080.05n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5207.240.04n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5172.180.05n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5187.200.04n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5164.200.05n/a
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]✅ 5/5208.760.04n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5147.660.06n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5171.880.05n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5183.080.04n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5179.700.05n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]✅ 5/5187.840.04n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5204.720.04n/a
iron/operators/mha
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]✅ 5/540876.740.21n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5149.020.06n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5173.820.05n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5173.100.05n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5154.940.05n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5182.700.05n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5176.940.05n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5168.960.05n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5225.120.04n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/5161.540.05n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5193.960.06n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/5168.260.05n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5189.440.06n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/5154.400.06n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5200.760.05n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5164.480.05n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5232.000.04n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5178.800.05n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/5174.280.05n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5188.880.04n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]✅ 5/5213.420.04n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]✅ 5/5216.900.04n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]✅ 5/5214.740.04n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]✅ 5/5276.280.03n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5176.580.61n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5192.380.52n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5182.020.57n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]✅ 5/5176.800.56n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5202.180.40n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5176.780.42n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5184.580.42n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]✅ 5/5236.480.35n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5176.340.05n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5197.280.04n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5185.520.05n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5198.700.04n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5165.060.05n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5232.860.04n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5244.260.03n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5259.840.03n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5199.200.04n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5198.460.04n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5196.480.04n/a
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5223.800.04n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5166.140.88n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5159.660.85n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5170.160.88n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/54017.510.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/53696.350.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/510503.010.21n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5153.360.05n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5154.700.05n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5186.320.05n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5160.720.05n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5197.300.05n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5163.840.05n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5181.660.05n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5221.580.04n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5143.163.68n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/5175.425.99n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5142.583.70n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.10 (+7.29%)0.08 (+5.48%)0.08 (+7.12%)0.07 (+0.86%)0.01 (+34.56%)174.80 (-0.85%)149.48 (-4.44%)152.50 (-6.67%)120.60 (-6.80%)22.92 (+26.05%)
9c70ba8 — 2026-06-29 16:51:590.09 (n/a)0.08 (n/a)0.08 (n/a)0.07 (n/a)0.01 (n/a)176.30 (n/a)156.42 (n/a)163.40 (n/a)129.40 (n/a)18.18 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.10 (+21.04%)0.07 (+9.21%)0.07 (+5.82%)0.06 (+16.67%)0.01 (+40.08%)198.80 (-14.27%)169.60 (-7.93%)168.10 (-5.51%)127.70 (-17.40%)27.07 (-5.44%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.01 (n/a)231.90 (n/a)184.20 (n/a)177.90 (n/a)154.60 (n/a)28.63 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.10 (+3.20%)0.07 (-0.11%)0.07 (+17.32%)0.05 (-12.41%)0.02 (+12.04%)245.60 (+14.18%)188.22 (+1.36%)175.30 (-14.74%)127.20 (-3.12%)45.40 (+20.65%)
9c70ba8 — 2026-06-29 16:51:590.09 (n/a)0.07 (n/a)0.06 (n/a)0.06 (n/a)0.02 (n/a)215.10 (n/a)185.70 (n/a)205.60 (n/a)131.30 (n/a)37.63 (n/a)

test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.09 (+21.56%)0.06 (+4.93%)0.05 (-12.19%)0.03 (-18.66%)0.02 (+71.49%)356.90 (+22.94%)235.98 (+3.74%)264.00 (+13.89%)131.50 (-17.71%)91.77 (+66.27%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.06 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)290.30 (n/a)227.48 (n/a)231.80 (n/a)159.80 (n/a)55.19 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-11.99%)0.03 (-5.52%)0.03 (-0.57%)0.03 (-6.14%)0.00 (-39.86%)200.70 (+6.53%)169.22 (+4.60%)167.90 (+0.54%)152.20 (+13.67%)19.23 (-25.86%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)188.40 (n/a)161.78 (n/a)167.00 (n/a)133.90 (n/a)25.94 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (+3.88%)0.03 (+10.00%)0.03 (+15.23%)0.02 (+14.65%)0.00 (-8.04%)219.90 (-12.77%)187.02 (-9.56%)180.80 (-13.20%)159.00 (-3.75%)24.52 (-22.00%)
9c70ba8 — 2026-06-29 16:51:590.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)252.10 (n/a)206.80 (n/a)208.30 (n/a)165.20 (n/a)31.44 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-13.53%)0.03 (-6.96%)0.03 (-4.73%)0.03 (-2.67%)0.00 (-35.30%)198.60 (+2.74%)178.00 (+6.69%)177.80 (+4.96%)156.50 (+15.67%)17.26 (-22.37%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.00 (n/a)193.30 (n/a)166.84 (n/a)169.40 (n/a)135.30 (n/a)22.24 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-18.37%)0.02 (-6.36%)0.02 (-4.19%)0.02 (+3.22%)0.00 (-57.36%)243.50 (-3.14%)212.26 (+3.99%)213.70 (+4.35%)190.50 (+22.51%)20.78 (-49.79%)
9c70ba8 — 2026-06-29 16:51:590.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)251.40 (n/a)204.12 (n/a)204.80 (n/a)155.50 (n/a)41.39 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.04 (+5.78%)0.03 (-5.23%)0.02 (-7.36%)0.02 (+3.63%)0.01 (+8.83%)238.70 (-3.52%)206.92 (+5.81%)215.00 (+7.93%)141.40 (-5.42%)39.91 (-0.54%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)247.40 (n/a)195.56 (n/a)199.20 (n/a)149.50 (n/a)40.13 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-18.24%)0.03 (-12.70%)0.03 (-13.29%)0.02 (-1.26%)0.00 (-52.71%)240.30 (+1.26%)206.00 (+12.01%)202.00 (+15.36%)179.20 (+22.32%)22.19 (-40.68%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)237.30 (n/a)183.92 (n/a)175.10 (n/a)146.50 (n/a)37.41 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-20.95%)0.02 (-19.31%)0.02 (-21.68%)0.02 (-8.19%)0.00 (-31.69%)282.50 (+8.95%)225.78 (+21.77%)223.20 (+27.69%)178.30 (+26.54%)46.11 (-5.68%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)259.30 (n/a)185.42 (n/a)174.80 (n/a)140.90 (n/a)48.88 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.03 (-25.33%)0.02 (-17.54%)0.02 (-18.78%)0.02 (+14.61%)0.00 (-54.54%)267.80 (-12.74%)240.78 (+15.22%)251.20 (+23.14%)185.90 (+33.93%)32.16 (-48.76%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)306.90 (n/a)208.98 (n/a)204.00 (n/a)138.80 (n/a)62.76 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.11 (n/a)0.09 (n/a)0.08 (n/a)0.07 (n/a)0.01 (n/a)167.30 (n/a)145.86 (n/a)145.40 (n/a)116.60 (n/a)19.92 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.08 (n/a)0.07 (n/a)0.08 (n/a)0.06 (n/a)0.01 (n/a)202.60 (n/a)172.22 (n/a)162.50 (n/a)159.30 (n/a)18.07 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.09 (n/a)0.07 (n/a)0.07 (n/a)0.06 (n/a)0.02 (n/a)218.90 (n/a)174.38 (n/a)173.70 (n/a)129.70 (n/a)40.53 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.06 (n/a)0.03 (n/a)0.01 (n/a)400.70 (n/a)243.62 (n/a)204.40 (n/a)192.30 (n/a)88.73 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.08 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.01 (n/a)225.70 (n/a)185.88 (n/a)179.90 (n/a)147.20 (n/a)37.11 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.10 (n/a)0.07 (n/a)0.06 (n/a)0.05 (n/a)0.02 (n/a)227.70 (n/a)186.52 (n/a)191.90 (n/a)126.90 (n/a)42.59 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)225.80 (n/a)199.26 (n/a)194.90 (n/a)171.20 (n/a)25.70 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.09 (n/a)0.06 (n/a)0.05 (n/a)0.05 (n/a)0.02 (n/a)266.80 (n/a)212.04 (n/a)223.60 (n/a)137.60 (n/a)49.55 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)176.00 (n/a)151.44 (n/a)156.30 (n/a)129.60 (n/a)18.43 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)177.60 (n/a)156.54 (n/a)168.00 (n/a)130.00 (n/a)22.36 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)171.10 (n/a)154.18 (n/a)154.50 (n/a)132.30 (n/a)16.20 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)172.60 (n/a)152.82 (n/a)166.00 (n/a)119.80 (n/a)24.24 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)257.70 (n/a)185.50 (n/a)172.20 (n/a)120.50 (n/a)50.98 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.04 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)219.90 (n/a)191.46 (n/a)181.70 (n/a)167.90 (n/a)24.47 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.04 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)226.70 (n/a)190.82 (n/a)178.70 (n/a)153.60 (n/a)30.60 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.04 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)299.80 (n/a)226.44 (n/a)213.70 (n/a)197.60 (n/a)41.63 (n/a)
iron/operators/gemm

test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:214.21 (-14.69%)4.11 (-6.81%)4.11 (-2.12%)4.03 (+0.68%)0.07 (-83.36%)2330.70 (-0.68%)2287.60 (+6.57%)2288.80 (+2.16%)2233.10 (+17.22%)38.90 (-80.51%)1656.63 (-14.69%)1617.51 (-6.81%)1616.26 (-2.12%)1587.21 (+0.68%)27.65 (-83.36%)
9c70ba8 — 2026-06-29 16:51:594.94 (n/a)4.41 (n/a)4.20 (n/a)4.01 (n/a)0.42 (n/a)2346.60 (n/a)2146.64 (n/a)2240.30 (n/a)1905.10 (n/a)199.58 (n/a)1941.82 (n/a)1735.67 (n/a)1651.26 (n/a)1576.47 (n/a)166.20 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:212.01 (+48.51%)1.10 (-3.48%)1.03 (-20.39%)0.65 (-4.36%)0.55 (+89.32%)341.30 (+4.56%)235.84 (+13.86%)213.80 (+25.62%)110.30 (-32.66%)95.35 (+37.22%)85.58 (+48.51%)47.13 (-3.48%)44.13 (-20.39%)27.65 (-4.36%)23.36 (+89.32%)
9c70ba8 — 2026-06-29 16:51:591.35 (n/a)1.14 (n/a)1.30 (n/a)0.68 (n/a)0.29 (n/a)326.40 (n/a)207.14 (n/a)170.20 (n/a)163.80 (n/a)69.49 (n/a)57.63 (n/a)48.83 (n/a)55.44 (n/a)28.91 (n/a)12.34 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:211.03 (-21.61%)0.90 (-15.81%)0.97 (-19.44%)0.63 (-3.61%)0.16 (-45.64%)351.70 (+3.75%)252.76 (+13.91%)228.40 (+24.13%)215.00 (+27.60%)56.07 (-23.49%)43.90 (-21.61%)38.53 (-15.81%)41.32 (-19.44%)26.83 (-3.61%)6.75 (-45.64%)
9c70ba8 — 2026-06-29 16:51:591.31 (n/a)1.07 (n/a)1.20 (n/a)0.65 (n/a)0.29 (n/a)339.00 (n/a)221.90 (n/a)184.00 (n/a)168.50 (n/a)73.29 (n/a)56.00 (n/a)45.76 (n/a)51.29 (n/a)27.83 (n/a)12.42 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:210.52 (-0.86%)0.52 (-0.39%)0.52 (-0.23%)0.52 (-0.38%)0.00 (-52.01%)48853.90 (+0.38%)48728.78 (+0.39%)48706.20 (+0.23%)48629.70 (+0.87%)90.29 (-51.38%)353.28 (-0.86%)352.56 (-0.39%)352.72 (-0.23%)351.66 (-0.38%)0.65 (-52.01%)
9c70ba8 — 2026-06-29 16:51:590.52 (n/a)0.52 (n/a)0.52 (n/a)0.52 (n/a)0.00 (n/a)48668.40 (n/a)48539.04 (n/a)48595.00 (n/a)48211.30 (n/a)185.70 (n/a)356.35 (n/a)353.94 (n/a)353.53 (n/a)353.00 (n/a)1.36 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:210.89 (+0.92%)0.88 (-0.09%)0.88 (-0.26%)0.87 (-0.21%)0.01 (+94.77%)28925.50 (+0.21%)28685.16 (+0.10%)28754.10 (+0.26%)28250.90 (-0.92%)276.01 (+93.50%)608.12 (+0.92%)598.96 (-0.09%)597.48 (-0.26%)593.94 (-0.21%)5.80 (+94.77%)
9c70ba8 — 2026-06-29 16:51:590.88 (n/a)0.88 (n/a)0.88 (n/a)0.87 (n/a)0.00 (n/a)28863.80 (n/a)28656.32 (n/a)28678.50 (n/a)28511.80 (n/a)142.64 (n/a)602.55 (n/a)599.53 (n/a)599.05 (n/a)595.20 (n/a)2.98 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:213.32 (-3.20%)3.22 (-1.61%)3.20 (-3.28%)3.15 (+0.21%)0.08 (-32.41%)7995.60 (-0.21%)7808.30 (+1.57%)7857.40 (+3.39%)7572.40 (+3.31%)197.96 (-30.61%)2268.76 (-3.20%)2201.34 (-1.61%)2186.46 (-3.28%)2148.67 (+0.21%)56.12 (-32.41%)
9c70ba8 — 2026-06-29 16:51:593.43 (n/a)3.28 (n/a)3.31 (n/a)3.14 (n/a)0.12 (n/a)8012.50 (n/a)7687.26 (n/a)7600.00 (n/a)7330.00 (n/a)285.31 (n/a)2343.79 (n/a)2237.32 (n/a)2260.51 (n/a)2144.14 (n/a)83.03 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:213.65 (-5.16%)3.31 (-2.40%)3.47 (+4.61%)2.86 (-1.90%)0.35 (-5.15%)2813.80 (+1.93%)2459.14 (+2.44%)2322.20 (-4.41%)2207.10 (+5.44%)272.19 (+2.81%)957.77 (-5.16%)867.79 (-2.40%)910.29 (+4.61%)751.27 (-1.90%)92.44 (-5.15%)
9c70ba8 — 2026-06-29 16:51:593.85 (n/a)3.39 (n/a)3.32 (n/a)2.92 (n/a)0.37 (n/a)2760.40 (n/a)2400.60 (n/a)2429.40 (n/a)2093.20 (n/a)264.75 (n/a)1009.89 (n/a)889.15 (n/a)870.16 (n/a)765.79 (n/a)97.46 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:210.50 (+7.32%)0.38 (-0.39%)0.33 (-3.40%)0.27 (-15.84%)0.10 (+67.10%)4573.20 (+18.82%)3514.02 (+4.34%)3760.00 (+3.51%)2506.00 (-6.83%)919.25 (+77.62%)26.78 (+7.32%)20.26 (-0.39%)17.85 (-3.40%)14.67 (-15.84%)5.59 (+67.10%)
9c70ba8 — 2026-06-29 16:51:590.46 (n/a)0.38 (n/a)0.34 (n/a)0.32 (n/a)0.06 (n/a)3848.80 (n/a)3367.78 (n/a)3632.40 (n/a)2689.60 (n/a)517.52 (n/a)24.95 (n/a)20.34 (n/a)18.48 (n/a)17.44 (n/a)3.34 (n/a)

test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:216.28 (-2.11%)4.59 (-8.55%)4.70 (-0.33%)3.21 (-19.09%)1.25 (+35.90%)2074.60 (+23.59%)1537.88 (+13.21%)1414.50 (+0.33%)1058.70 (+2.15%)420.73 (+77.50%)1941.17 (-2.11%)1419.33 (-8.55%)1452.91 (-0.33%)990.67 (-19.09%)385.40 (+35.90%)
9c70ba8 — 2026-06-29 16:51:596.42 (n/a)5.02 (n/a)4.72 (n/a)3.96 (n/a)0.92 (n/a)1678.60 (n/a)1358.40 (n/a)1409.90 (n/a)1036.40 (n/a)237.03 (n/a)1982.97 (n/a)1552.03 (n/a)1457.75 (n/a)1224.38 (n/a)283.59 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:210.27 (+1.24%)0.23 (-1.75%)0.24 (+7.22%)0.18 (-8.55%)0.04 (+21.16%)0.27 (+1.24%)0.22 (-1.75%)0.23 (+7.22%)0.18 (-8.55%)0.03 (+21.16%)
9c70ba8 — 2026-06-29 16:51:590.27 (n/a)0.23 (n/a)0.22 (n/a)0.19 (n/a)0.03 (n/a)0.26 (n/a)0.23 (n/a)0.22 (n/a)0.19 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2113.28 (+0.52%)12.66 (+1.20%)12.58 (-3.31%)12.03 (+10.46%)0.50 (-47.55%)13.27 (+0.52%)12.66 (+1.20%)12.57 (-3.31%)12.03 (+10.46%)0.50 (-47.55%)
9c70ba8 — 2026-06-29 16:51:5913.21 (n/a)12.51 (n/a)13.01 (n/a)10.89 (n/a)0.96 (n/a)13.21 (n/a)12.51 (n/a)13.00 (n/a)10.89 (n/a)0.96 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2125.21 (+0.01%)23.66 (-3.40%)23.99 (-2.76%)20.85 (-11.36%)1.71 (+160.89%)25.19 (+0.01%)23.64 (-3.40%)23.98 (-2.76%)20.83 (-11.36%)1.71 (+160.89%)
9c70ba8 — 2026-06-29 16:51:5925.21 (n/a)24.49 (n/a)24.67 (n/a)23.52 (n/a)0.65 (n/a)25.19 (n/a)24.47 (n/a)24.66 (n/a)23.50 (n/a)0.65 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2141.57 (-2.98%)39.66 (-1.15%)40.01 (-0.09%)37.46 (+2.39%)1.52 (-37.03%)41.54 (-2.98%)39.64 (-1.15%)39.98 (-0.09%)37.43 (+2.39%)1.52 (-37.03%)
9c70ba8 — 2026-06-29 16:51:5942.84 (n/a)40.12 (n/a)40.04 (n/a)36.58 (n/a)2.41 (n/a)42.82 (n/a)40.10 (n/a)40.02 (n/a)36.56 (n/a)2.41 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2143.17 (-5.07%)41.45 (-3.58%)41.68 (-0.94%)38.00 (-5.17%)2.06 (-11.88%)43.15 (-5.07%)41.43 (-3.58%)41.65 (-0.94%)37.98 (-5.17%)2.06 (-11.88%)
9c70ba8 — 2026-06-29 16:51:5945.48 (n/a)42.99 (n/a)42.07 (n/a)40.07 (n/a)2.34 (n/a)45.45 (n/a)42.97 (n/a)42.05 (n/a)40.05 (n/a)2.34 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2113.50 (+0.26%)12.50 (-0.38%)12.49 (-3.45%)11.12 (+3.85%)0.91 (-17.24%)13.49 (+0.26%)12.50 (-0.38%)12.48 (-3.45%)11.11 (+3.85%)0.91 (-17.24%)
9c70ba8 — 2026-06-29 16:51:5913.46 (n/a)12.55 (n/a)12.94 (n/a)10.71 (n/a)1.10 (n/a)13.46 (n/a)12.54 (n/a)12.93 (n/a)10.70 (n/a)1.10 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2124.61 (-1.10%)24.21 (-0.35%)24.16 (-1.94%)23.74 (+2.51%)0.36 (-48.60%)24.59 (-1.10%)24.20 (-0.35%)24.14 (-1.94%)23.72 (+2.51%)0.36 (-48.60%)
9c70ba8 — 2026-06-29 16:51:5924.88 (n/a)24.30 (n/a)24.64 (n/a)23.16 (n/a)0.69 (n/a)24.87 (n/a)24.28 (n/a)24.62 (n/a)23.14 (n/a)0.69 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2140.31 (-5.28%)38.54 (-4.58%)39.72 (+0.48%)32.83 (-14.62%)3.20 (+78.11%)40.28 (-5.28%)38.52 (-4.58%)39.70 (+0.48%)32.81 (-14.62%)3.20 (+78.11%)
9c70ba8 — 2026-06-29 16:51:5942.56 (n/a)40.39 (n/a)39.53 (n/a)38.45 (n/a)1.80 (n/a)42.53 (n/a)40.37 (n/a)39.51 (n/a)38.43 (n/a)1.80 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:28:2143.40 (-4.17%)40.98 (-7.32%)42.11 (-6.00%)35.44 (-17.05%)3.23 (+197.54%)43.37 (-4.17%)40.95 (-7.32%)42.08 (-6.00%)35.42 (-17.05%)3.23 (+197.54%)
9c70ba8 — 2026-06-29 16:51:5945.29 (n/a)44.22 (n/a)44.80 (n/a)42.73 (n/a)1.09 (n/a)45.26 (n/a)44.19 (n/a)44.77 (n/a)42.70 (n/a)1.08 (n/a)

test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]

No metrics available.

test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]

No metrics available.

test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]

No metrics available.

test_gemv_batched[M_256-K_128-num_aie_columns_8-tile_size_input_1-tile_size_output_32-num_batches_100]

No metrics available.

test_gemv_batched[M_448-K_64-num_aie_columns_8-tile_size_input_1-tile_size_output_56-num_batches_192]

No metrics available.

test_gemv_batched[M_512-K_64-num_aie_columns_8-tile_size_input_4-tile_size_output_64-num_batches_32]

No metrics available.

test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]

No metrics available.

iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (n/a)0.06 (n/a)0.06 (n/a)0.04 (n/a)0.01 (n/a)200.20 (n/a)151.42 (n/a)126.30 (n/a)116.00 (n/a)39.86 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)174.60 (n/a)158.80 (n/a)154.70 (n/a)138.60 (n/a)15.17 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)241.10 (n/a)194.46 (n/a)194.80 (n/a)163.00 (n/a)30.32 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)238.70 (n/a)179.48 (n/a)166.30 (n/a)148.90 (n/a)35.62 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)182.90 (n/a)160.08 (n/a)166.80 (n/a)125.70 (n/a)21.53 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)249.30 (n/a)207.24 (n/a)211.60 (n/a)159.50 (n/a)39.14 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)210.70 (n/a)172.18 (n/a)159.90 (n/a)152.30 (n/a)24.41 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)205.90 (n/a)187.20 (n/a)182.20 (n/a)164.00 (n/a)18.05 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-2.97%)0.05 (-3.24%)0.05 (-9.86%)0.04 (+3.49%)0.01 (-20.51%)184.00 (-3.36%)164.20 (+2.55%)171.20 (+10.95%)133.00 (+3.02%)19.24 (-23.55%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)190.40 (n/a)160.12 (n/a)154.30 (n/a)129.10 (n/a)25.16 (n/a)

test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-16.64%)0.04 (+2.15%)0.04 (-3.17%)0.03 (+61.45%)0.00 (-62.96%)234.30 (-38.07%)208.76 (-9.60%)217.00 (+3.28%)181.80 (+19.92%)22.00 (-74.46%)
9c70ba8 — 2026-06-29 16:51:590.05 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)378.30 (n/a)230.92 (n/a)210.10 (n/a)151.60 (n/a)86.13 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (+2.96%)0.06 (+4.46%)0.06 (+4.83%)0.04 (-2.82%)0.01 (+14.23%)194.50 (+2.86%)147.66 (-3.52%)146.40 (-4.56%)116.10 (-2.85%)32.41 (+11.64%)
9c70ba8 — 2026-06-29 16:51:590.07 (n/a)0.06 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)189.10 (n/a)153.04 (n/a)153.40 (n/a)119.50 (n/a)29.03 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-30.69%)0.05 (-16.37%)0.05 (-5.58%)0.04 (-1.25%)0.00 (-68.42%)194.90 (+1.30%)171.88 (+15.48%)166.90 (+5.90%)155.00 (+44.32%)15.53 (-52.89%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.06 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)192.40 (n/a)148.84 (n/a)157.60 (n/a)107.40 (n/a)32.97 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-5.09%)0.04 (+2.17%)0.04 (-3.22%)0.04 (+39.84%)0.00 (-61.23%)200.60 (-28.48%)183.08 (-5.75%)188.00 (+3.35%)164.40 (+5.32%)13.99 (-71.95%)
9c70ba8 — 2026-06-29 16:51:590.05 (n/a)0.04 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)280.50 (n/a)194.24 (n/a)181.90 (n/a)156.10 (n/a)49.85 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-23.13%)0.05 (-7.72%)0.05 (+2.76%)0.04 (+8.41%)0.00 (-60.11%)201.70 (-7.77%)179.70 (+4.99%)174.30 (-2.68%)159.50 (+30.10%)18.05 (-51.03%)
9c70ba8 — 2026-06-29 16:51:590.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)218.70 (n/a)171.16 (n/a)179.10 (n/a)122.60 (n/a)36.86 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-5.60%)0.04 (-10.31%)0.04 (-10.56%)0.04 (-17.49%)0.01 (+38.73%)220.40 (+21.23%)187.84 (+12.33%)191.30 (+11.81%)157.80 (+5.91%)23.74 (+78.07%)
9c70ba8 — 2026-06-29 16:51:590.05 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.00 (n/a)181.80 (n/a)167.22 (n/a)171.10 (n/a)149.00 (n/a)13.33 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-8.94%)0.04 (-18.58%)0.04 (-18.28%)0.03 (-21.19%)0.01 (+7.19%)256.90 (+26.86%)204.72 (+24.04%)204.30 (+22.41%)154.70 (+9.87%)36.79 (+49.07%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)202.50 (n/a)165.04 (n/a)166.90 (n/a)140.80 (n/a)24.68 (n/a)
iron/operators/mha

test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.21 (+0.07%)0.21 (+0.12%)0.21 (+0.07%)0.21 (+0.21%)0.00 (-51.55%)40900.90 (-0.21%)40876.74 (-0.12%)40886.10 (-0.07%)40838.70 (-0.07%)24.68 (-51.72%)
9c70ba8 — 2026-06-29 16:51:590.21 (n/a)0.20 (n/a)0.21 (n/a)0.20 (n/a)0.00 (n/a)40988.20 (n/a)40927.74 (n/a)40913.30 (n/a)40866.90 (n/a)51.11 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (+13.17%)0.05 (+7.41%)0.05 (-5.52%)0.04 (+23.18%)0.01 (+11.33%)188.30 (-18.84%)161.54 (-7.25%)174.20 (+5.83%)122.40 (-11.62%)27.55 (-22.31%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)232.00 (n/a)174.16 (n/a)164.60 (n/a)138.50 (n/a)35.45 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (-5.99%)0.06 (-11.34%)0.07 (-10.16%)0.05 (-22.26%)0.01 (+47.63%)251.80 (+28.67%)193.96 (+14.34%)183.30 (+11.29%)167.10 (+6.37%)33.47 (+108.89%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.07 (n/a)0.07 (n/a)0.06 (n/a)0.01 (n/a)195.70 (n/a)169.64 (n/a)164.70 (n/a)157.10 (n/a)16.02 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (+22.31%)0.05 (-1.43%)0.05 (+4.80%)0.04 (-14.29%)0.01 (+196.35%)207.50 (+16.70%)168.26 (+5.51%)155.00 (-4.62%)118.20 (-18.26%)38.49 (+196.79%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.00 (n/a)177.80 (n/a)159.48 (n/a)162.50 (n/a)144.60 (n/a)12.97 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-12.87%)0.06 (+5.35%)0.06 (+15.02%)0.04 (+41.29%)0.01 (-42.81%)268.50 (-29.23%)189.44 (-12.86%)171.00 (-13.02%)159.70 (+14.73%)44.70 (-53.23%)
9c70ba8 — 2026-06-29 16:51:590.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.02 (n/a)379.40 (n/a)217.40 (n/a)196.60 (n/a)139.20 (n/a)95.58 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.08 (+57.84%)0.06 (+31.73%)0.06 (+21.33%)0.04 (+14.95%)0.02 (+116.63%)226.40 (-13.02%)154.40 (-21.08%)148.30 (-17.57%)102.20 (-36.64%)45.68 (+17.79%)
9c70ba8 — 2026-06-29 16:51:590.05 (n/a)0.04 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)260.30 (n/a)195.64 (n/a)179.90 (n/a)161.30 (n/a)38.78 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.07 (-8.22%)0.05 (-13.76%)0.05 (-12.85%)0.03 (-27.94%)0.02 (+28.85%)299.20 (+38.78%)200.76 (+20.72%)192.00 (+14.76%)143.70 (+8.95%)62.44 (+92.84%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.01 (n/a)215.60 (n/a)166.30 (n/a)167.30 (n/a)131.90 (n/a)32.38 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-22.92%)0.05 (-4.10%)0.05 (+5.79%)0.04 (-6.30%)0.01 (-44.13%)212.30 (+6.74%)164.48 (+1.61%)161.90 (-5.49%)134.40 (+29.73%)29.51 (-16.34%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)198.90 (n/a)161.88 (n/a)171.30 (n/a)103.60 (n/a)35.27 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-24.53%)0.04 (-20.32%)0.04 (-20.04%)0.02 (-35.20%)0.01 (-20.59%)376.10 (+54.33%)232.00 (+28.38%)212.60 (+25.06%)151.90 (+32.55%)84.82 (+74.64%)
9c70ba8 — 2026-06-29 16:51:590.08 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.02 (n/a)243.70 (n/a)180.72 (n/a)170.00 (n/a)114.60 (n/a)48.57 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-21.46%)0.05 (+0.23%)0.05 (-1.90%)0.04 (+76.47%)0.01 (-63.24%)208.40 (-43.34%)178.80 (-12.56%)178.10 (+1.95%)143.50 (+27.33%)24.10 (-75.15%)
9c70ba8 — 2026-06-29 16:51:590.07 (n/a)0.05 (n/a)0.05 (n/a)0.02 (n/a)0.02 (n/a)367.80 (n/a)204.48 (n/a)174.70 (n/a)112.70 (n/a)96.97 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.08 (+23.54%)0.05 (-3.36%)0.05 (-9.08%)0.04 (-18.31%)0.01 (+273.70%)213.00 (+22.41%)174.28 (+7.01%)181.60 (+9.99%)122.10 (-19.03%)34.49 (+261.00%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.06 (n/a)0.06 (n/a)0.05 (n/a)0.00 (n/a)174.00 (n/a)162.86 (n/a)165.10 (n/a)150.80 (n/a)9.56 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (-19.48%)0.04 (-3.79%)0.04 (-9.09%)0.04 (+35.51%)0.01 (-58.48%)214.70 (-26.19%)188.88 (-2.27%)198.20 (+9.99%)157.40 (+24.23%)22.62 (-63.08%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)290.90 (n/a)193.26 (n/a)180.20 (n/a)126.70 (n/a)61.27 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.04 (-23.95%)0.04 (-8.04%)0.04 (+5.13%)0.04 (-0.34%)0.00 (-66.03%)230.40 (+0.35%)213.42 (+6.22%)207.00 (-4.87%)197.40 (+31.51%)15.73 (-55.38%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)229.60 (n/a)200.92 (n/a)217.60 (n/a)150.10 (n/a)35.26 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.06 (-4.19%)0.04 (-17.93%)0.04 (-25.88%)0.02 (-30.41%)0.01 (+25.72%)357.60 (+43.67%)216.90 (+29.14%)210.00 (+34.87%)134.60 (+4.42%)85.95 (+83.05%)
9c70ba8 — 2026-06-29 16:51:590.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)248.90 (n/a)167.96 (n/a)155.70 (n/a)128.90 (n/a)46.95 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.05 (+12.89%)0.04 (+2.20%)0.04 (-2.88%)0.03 (-15.26%)0.01 (+133.34%)276.00 (+18.00%)214.74 (-0.07%)216.60 (+3.00%)171.90 (-11.44%)39.64 (+143.31%)
9c70ba8 — 2026-06-29 16:51:590.04 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)233.90 (n/a)214.88 (n/a)210.30 (n/a)194.10 (n/a)16.29 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.04 (-12.69%)0.03 (-16.96%)0.03 (-7.54%)0.02 (-18.57%)0.01 (+17.83%)375.10 (+22.82%)276.28 (+24.98%)234.40 (+8.17%)192.80 (+14.56%)86.24 (+68.38%)
9c70ba8 — 2026-06-29 16:51:590.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)305.40 (n/a)221.06 (n/a)216.70 (n/a)168.30 (n/a)51.22 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.77 (+2.59%)0.61 (+6.30%)0.74 (+54.91%)0.40 (-13.61%)0.20 (+29.47%)247.40 (+15.77%)176.58 (-1.56%)133.20 (-35.47%)128.30 (-2.51%)63.91 (+48.23%)
9c70ba8 — 2026-06-29 16:51:590.75 (n/a)0.58 (n/a)0.48 (n/a)0.46 (n/a)0.15 (n/a)213.70 (n/a)179.38 (n/a)206.40 (n/a)131.60 (n/a)43.11 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.57 (-18.27%)0.52 (+7.53%)0.55 (+31.41%)0.41 (+15.72%)0.06 (-57.81%)237.70 (-13.60%)192.38 (-12.09%)180.10 (-23.88%)172.30 (+22.37%)26.61 (-55.81%)
9c70ba8 — 2026-06-29 16:51:590.70 (n/a)0.48 (n/a)0.42 (n/a)0.36 (n/a)0.15 (n/a)275.10 (n/a)218.84 (n/a)236.60 (n/a)140.80 (n/a)60.23 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.77 (+31.64%)0.57 (+8.21%)0.52 (-6.04%)0.40 (-7.11%)0.15 (+127.04%)245.90 (+7.66%)182.02 (-3.72%)189.50 (+6.40%)128.20 (-24.01%)46.86 (+83.87%)
9c70ba8 — 2026-06-29 16:51:590.58 (n/a)0.53 (n/a)0.55 (n/a)0.43 (n/a)0.07 (n/a)228.40 (n/a)189.06 (n/a)178.10 (n/a)168.70 (n/a)25.49 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.68 (+12.24%)0.56 (+11.33%)0.55 (+6.75%)0.48 (+20.96%)0.08 (-21.77%)203.50 (-17.31%)176.80 (-11.70%)179.80 (-6.35%)144.20 (-10.88%)22.77 (-42.68%)
9c70ba8 — 2026-06-29 16:51:590.61 (n/a)0.51 (n/a)0.51 (n/a)0.40 (n/a)0.10 (n/a)246.10 (n/a)200.22 (n/a)192.00 (n/a)161.80 (n/a)39.73 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.59 (+25.46%)0.40 (-8.20%)0.29 (-32.71%)0.29 (-29.03%)0.15 (+574.11%)250.70 (+40.92%)202.18 (+20.62%)250.50 (+48.58%)124.10 (-20.30%)66.46 (+684.86%)
9c70ba8 — 2026-06-29 16:51:590.47 (n/a)0.44 (n/a)0.44 (n/a)0.41 (n/a)0.02 (n/a)177.90 (n/a)167.62 (n/a)168.60 (n/a)155.70 (n/a)8.47 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.48 (-0.97%)0.42 (-2.21%)0.42 (-1.41%)0.38 (+13.15%)0.04 (-33.88%)192.70 (-11.65%)176.78 (+1.13%)177.60 (+1.43%)152.00 (+0.93%)16.44 (-40.16%)
9c70ba8 — 2026-06-29 16:51:590.49 (n/a)0.43 (n/a)0.42 (n/a)0.34 (n/a)0.06 (n/a)218.10 (n/a)174.80 (n/a)175.10 (n/a)150.60 (n/a)27.47 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.55 (+14.51%)0.42 (+1.74%)0.42 (+3.62%)0.30 (-12.16%)0.10 (+92.51%)246.40 (+13.86%)184.58 (+1.74%)175.60 (-3.52%)135.00 (-12.68%)45.54 (+92.07%)
9c70ba8 — 2026-06-29 16:51:590.48 (n/a)0.41 (n/a)0.41 (n/a)0.34 (n/a)0.05 (n/a)216.40 (n/a)181.42 (n/a)182.00 (n/a)154.60 (n/a)23.71 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.62 (+16.70%)0.35 (-10.39%)0.33 (-10.76%)0.22 (-15.22%)0.16 (+32.92%)339.60 (+17.96%)236.48 (+17.24%)225.00 (+12.05%)119.20 (-14.31%)82.00 (+32.68%)
9c70ba8 — 2026-06-29 16:51:590.53 (n/a)0.39 (n/a)0.37 (n/a)0.26 (n/a)0.12 (n/a)287.90 (n/a)201.70 (n/a)200.80 (n/a)139.10 (n/a)61.81 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:211.17 (+25.85%)0.88 (+23.99%)0.94 (+31.07%)0.55 (-2.21%)0.30 (+116.20%)240.20 (+2.26%)166.14 (-12.66%)139.50 (-23.69%)111.90 (-20.58%)62.59 (+77.49%)
9c70ba8 — 2026-06-29 16:51:590.93 (n/a)0.71 (n/a)0.72 (n/a)0.56 (n/a)0.14 (n/a)234.90 (n/a)190.22 (n/a)182.80 (n/a)140.90 (n/a)35.26 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:211.20 (+53.05%)0.85 (+23.35%)0.82 (+22.16%)0.70 (+16.62%)0.20 (+174.01%)186.10 (-14.24%)159.66 (-16.64%)160.00 (-18.16%)109.40 (-34.65%)30.74 (+51.99%)
9c70ba8 — 2026-06-29 16:51:590.78 (n/a)0.69 (n/a)0.67 (n/a)0.60 (n/a)0.07 (n/a)217.00 (n/a)191.54 (n/a)195.50 (n/a)167.40 (n/a)20.22 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:211.18 (+13.94%)0.88 (+12.55%)0.90 (+25.07%)0.42 (-24.70%)0.28 (+48.51%)313.80 (+32.80%)170.16 (-3.72%)145.20 (-20.04%)111.50 (-12.27%)81.83 (+89.74%)
9c70ba8 — 2026-06-29 16:51:591.03 (n/a)0.78 (n/a)0.72 (n/a)0.55 (n/a)0.19 (n/a)236.30 (n/a)176.74 (n/a)181.60 (n/a)127.10 (n/a)43.13 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.00 (+0.00%)0.00 (+0.00%)0.00 (+0.00%)0.00 (-11.11%)0.00 (+35.40%)4957.38 (+10.76%)4017.51 (+0.59%)3591.75 (-6.02%)3504.29 (-0.38%)676.89 (+64.97%)
9c70ba8 — 2026-06-29 16:51:590.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)4475.59 (n/a)3994.10 (n/a)3821.83 (n/a)3517.56 (n/a)410.30 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.00 (+0.00%)0.00 (+13.40%)0.00 (+22.22%)0.00 (+23.53%)0.00 (-75.46%)3971.54 (-16.78%)3696.35 (-12.75%)3647.88 (-19.13%)3564.79 (+0.88%)158.09 (-73.33%)
9c70ba8 — 2026-06-29 16:51:590.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)4772.10 (n/a)4236.73 (n/a)4510.67 (n/a)3533.78 (n/a)592.76 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:210.28 (+0.15%)0.21 (+17.77%)0.24 (+58.73%)0.15 (+8.81%)0.06 (+7.49%)14150.51 (-8.09%)10503.01 (-14.39%)8739.07 (-37.01%)7606.30 (-0.16%)3271.38 (+3.91%)
9c70ba8 — 2026-06-29 16:51:590.28 (n/a)0.18 (n/a)0.15 (n/a)0.14 (n/a)0.06 (n/a)15396.52 (n/a)12268.40 (n/a)13872.79 (n/a)7618.14 (n/a)3148.14 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:214.09 (+31.88%)3.68 (+35.08%)3.72 (+35.57%)3.36 (+53.94%)0.28 (-26.54%)156.10 (-35.07%)143.16 (-26.89%)141.00 (-26.22%)128.20 (-24.19%)10.85 (-63.28%)
9c70ba8 — 2026-06-29 16:51:593.10 (n/a)2.72 (n/a)2.74 (n/a)2.18 (n/a)0.39 (n/a)240.40 (n/a)195.82 (n/a)191.10 (n/a)169.10 (n/a)29.54 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:216.50 (+17.96%)5.99 (+17.11%)5.82 (+9.47%)5.72 (+33.22%)0.32 (-32.35%)183.40 (-24.93%)175.42 (-15.09%)180.20 (-8.67%)161.30 (-15.24%)9.15 (-57.67%)
9c70ba8 — 2026-06-29 16:51:595.51 (n/a)5.12 (n/a)5.32 (n/a)4.29 (n/a)0.48 (n/a)244.30 (n/a)206.60 (n/a)197.30 (n/a)190.30 (n/a)21.62 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:223.72 (+20.32%)3.12 (+16.02%)3.04 (+6.32%)2.73 (+32.06%)0.43 (+8.15%)192.20 (-24.30%)170.32 (-14.26%)172.50 (-5.94%)141.00 (-16.86%)22.57 (-32.31%)
4d4b803 — 2026-06-22 17:54:573.09 (n/a)2.69 (n/a)2.86 (n/a)2.07 (n/a)0.40 (n/a)253.90 (n/a)198.64 (n/a)183.40 (n/a)169.60 (n/a)33.34 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:28:214.07 (+18.44%)3.70 (+29.22%)3.86 (+35.20%)3.30 (+44.17%)0.33 (-30.43%)158.90 (-30.64%)142.58 (-23.79%)135.90 (-26.02%)129.00 (-15.52%)12.84 (-58.87%)
9c70ba8 — 2026-06-29 16:51:593.43 (n/a)2.86 (n/a)2.85 (n/a)2.29 (n/a)0.47 (n/a)229.10 (n/a)187.10 (n/a)183.70 (n/a)152.70 (n/a)31.23 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:224.36 (+7.95%)3.26 (+13.04%)3.33 (+22.37%)2.38 (+18.97%)0.79 (-14.92%)220.70 (-15.96%)168.40 (-14.68%)157.40 (-18.28%)120.30 (-7.39%)40.78 (-34.38%)
4d4b803 — 2026-06-22 17:54:574.04 (n/a)2.89 (n/a)2.72 (n/a)2.00 (n/a)0.93 (n/a)262.60 (n/a)197.38 (n/a)192.60 (n/a)129.90 (n/a)62.14 (n/a)
Krackan - Examples

IRON

Tested on 2026_06_29_17_24_36 at commit 3ab7ee2.

iron/applications/llama_3.2_1b
TestChecksTTFT (mean)TPS (mean)
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]✅ 5/52.13n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]✅ 5/52.154.34
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]✅ 5/52.08n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]✅ 5/52.084.33

Trends:

IRON Trends

iron/applications/llama_3.2_1b

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
3ab7ee2 — 2026-06-29 17:18:592.14 (-0.37%)2.13 (+0.24%)2.13 (+0.14%)2.12 (+1.00%)0.01 (-60.71%)
9c70ba8 — 2026-06-29 17:01:322.15 (n/a)2.12 (n/a)2.12 (n/a)2.10 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
3ab7ee2 — 2026-06-29 17:18:594.35 (+3.57%)4.34 (+4.00%)4.34 (+3.95%)4.33 (+4.66%)0.01 (-64.41%)2.29 (+1.33%)2.15 (+0.01%)2.12 (-0.56%)2.11 (-0.28%)0.08 (+26.17%)
9c70ba8 — 2026-06-29 17:01:324.20 (n/a)4.18 (n/a)4.17 (n/a)4.14 (n/a)0.02 (n/a)2.26 (n/a)2.15 (n/a)2.13 (n/a)2.11 (n/a)0.06 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
3ab7ee2 — 2026-06-29 17:18:592.09 (-0.29%)2.08 (-0.15%)2.08 (+0.14%)2.06 (-0.15%)0.01 (+2.52%)
9c70ba8 — 2026-06-29 17:01:322.10 (n/a)2.08 (n/a)2.08 (n/a)2.06 (n/a)0.01 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
3ab7ee2 — 2026-06-29 17:18:594.34 (+3.95%)4.33 (+3.96%)4.33 (+3.96%)4.33 (+4.14%)0.01 (-29.80%)2.09 (-0.62%)2.08 (+0.43%)2.09 (+0.72%)2.06 (+0.29%)0.01 (-39.40%)
9c70ba8 — 2026-06-29 17:01:324.17 (n/a)4.17 (n/a)4.17 (n/a)4.15 (n/a)0.01 (n/a)2.10 (n/a)2.07 (n/a)2.07 (n/a)2.06 (n/a)0.02 (n/a)
Phoenix - Small

IRON

Tested on 2026_06_29_17_21_46 at commit 3ab7ee2.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5330.040.04n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5541.480.03n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5742.100.03n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5418.920.01n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5389.320.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5449.100.01n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5350.900.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5785.720.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5550.260.01n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5388.780.04n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5547.240.03n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5403.740.03n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5379.360.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5515.160.02n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5511.580.03n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5413.600.02n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5466.420.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5360.060.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5488.120.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5427.600.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5522.220.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5797.080.3916.66
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5577.680.4016.96
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/583147.140.30206.64
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/524942.661.01688.95
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/53247.082.96775.59
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/55654.940.2312.39
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.090.09
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.673.66
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a6.186.18
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a10.0310.02
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.683.68
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a6.406.39
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a8.798.79
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]✅ 5/5n/an/an/a
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]✅ 5/5n/an/an/a
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]✅ 5/5n/an/an/a
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]✅ 5/5n/an/an/a
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5354.440.03n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5341.880.03n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5389.200.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5378.540.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5680.740.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5492.380.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5359.640.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5411.700.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5393.600.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5555.620.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5363.400.03n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5507.600.02n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5374.320.02n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5711.180.02n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5522.320.02n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5464.420.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5439.340.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5576.100.02n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/5426.860.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5422.840.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/5462.260.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5477.580.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/5368.660.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5532.220.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5419.420.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5508.660.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5441.260.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/5472.160.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5819.160.01n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5358.700.28n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5435.780.24n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5338.240.31n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5463.440.17n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5710.660.18n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5529.460.18n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5335.460.03n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5313.900.03n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5321.720.03n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5480.420.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5319.420.03n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5460.540.02n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5469.180.02n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5289.160.03n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5345.100.03n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5439.100.34n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5385.220.38n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5349.200.40n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/516040.390.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/513460.430.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/522942.670.10n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/51133.220.01n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5407.740.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5394.520.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5452.240.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5526.540.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5369.560.02n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5668.901.33n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/51120.961.72n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5504.301.17n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.05 (+2.77%)0.04 (+8.01%)0.04 (+20.08%)0.02 (-7.24%)0.01 (+22.33%)492.00 (+7.80%)330.04 (-5.41%)293.70 (-16.73%)249.20 (-2.69%)96.87 (+32.38%)
9c70ba8 — 2026-06-29 16:29:160.05 (n/a)0.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)456.40 (n/a)348.90 (n/a)352.70 (n/a)256.10 (n/a)73.17 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.05 (+37.86%)0.03 (+12.59%)0.03 (-6.42%)0.01 (+29.87%)0.01 (+28.63%)1031.30 (-23.00%)541.48 (-14.07%)475.90 (+6.87%)248.00 (-27.44%)291.47 (-29.14%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)1339.30 (n/a)630.14 (n/a)445.30 (n/a)341.80 (n/a)411.31 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (+50.65%)0.03 (+26.63%)0.03 (+18.76%)0.01 (+3.17%)0.01 (+46.78%)2104.90 (-3.07%)742.10 (-11.24%)429.10 (-15.78%)313.80 (-33.62%)764.33 (+2.32%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)2171.60 (n/a)836.08 (n/a)509.50 (n/a)472.70 (n/a)746.98 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (-31.69%)0.01 (-32.67%)0.01 (-27.88%)0.01 (-14.20%)0.00 (-38.82%)575.10 (+16.53%)418.92 (+42.97%)378.20 (+38.64%)271.80 (+46.37%)117.34 (-0.66%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)493.50 (n/a)293.02 (n/a)272.80 (n/a)185.70 (n/a)118.12 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (-23.87%)0.02 (+13.80%)0.02 (+44.33%)0.01 (+93.82%)0.01 (-41.60%)553.20 (-48.41%)389.32 (-30.14%)347.40 (-30.71%)216.80 (+31.31%)149.76 (-54.11%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)0.01 (n/a)1072.20 (n/a)557.28 (n/a)501.40 (n/a)165.10 (n/a)326.36 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-28.69%)0.01 (-46.11%)0.01 (-49.17%)0.01 (-55.17%)0.01 (+1.83%)589.10 (+123.06%)449.10 (+106.48%)462.30 (+96.72%)193.50 (+40.22%)157.62 (+208.04%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)264.10 (n/a)217.50 (n/a)235.00 (n/a)138.00 (n/a)51.17 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (-33.37%)0.02 (-28.96%)0.02 (-25.53%)0.01 (-31.67%)0.01 (-24.93%)489.90 (+46.33%)350.90 (+42.92%)300.90 (+34.27%)246.30 (+50.09%)119.97 (+60.41%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)334.80 (n/a)245.52 (n/a)224.10 (n/a)164.10 (n/a)74.79 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (+27.00%)0.01 (-23.37%)0.01 (-45.97%)0.00 (-68.83%)0.01 (+66.51%)1914.30 (+220.76%)785.72 (+98.85%)566.00 (+85.09%)200.10 (-21.25%)674.85 (+318.18%)
9c70ba8 — 2026-06-29 16:29:160.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)596.80 (n/a)395.14 (n/a)305.80 (n/a)254.10 (n/a)161.38 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.01 (+38.26%)0.01 (+38.56%)0.01 (+12.88%)0.01 (+174.97%)0.00 (-34.19%)674.50 (-63.63%)550.26 (-38.36%)564.80 (-11.40%)445.20 (-27.68%)89.66 (-83.38%)
9c70ba8 — 2026-06-29 16:29:160.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)0.00 (n/a)1854.70 (n/a)892.66 (n/a)637.50 (n/a)615.60 (n/a)539.55 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.06 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.02 (n/a)612.20 (n/a)388.78 (n/a)266.80 (n/a)203.60 (n/a)202.28 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.05 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.02 (n/a)1239.00 (n/a)547.24 (n/a)378.30 (n/a)260.30 (n/a)405.59 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)515.50 (n/a)403.74 (n/a)404.10 (n/a)307.70 (n/a)74.48 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.05 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)560.10 (n/a)379.36 (n/a)346.20 (n/a)228.70 (n/a)140.45 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)636.00 (n/a)515.16 (n/a)528.70 (n/a)357.00 (n/a)100.79 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.07 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.02 (n/a)1035.70 (n/a)511.58 (n/a)456.20 (n/a)172.20 (n/a)326.74 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)530.60 (n/a)413.60 (n/a)416.70 (n/a)327.00 (n/a)81.38 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)575.00 (n/a)466.42 (n/a)462.40 (n/a)318.90 (n/a)96.25 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)525.40 (n/a)360.06 (n/a)312.10 (n/a)198.10 (n/a)131.32 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)630.10 (n/a)488.12 (n/a)502.30 (n/a)273.90 (n/a)131.74 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)628.40 (n/a)427.60 (n/a)435.00 (n/a)227.90 (n/a)163.66 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)663.90 (n/a)522.22 (n/a)536.50 (n/a)351.90 (n/a)135.92 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:450.62 (-4.16%)0.39 (-9.35%)0.46 (+21.85%)0.12 (-65.15%)0.20 (+64.59%)1812.70 (+186.96%)797.08 (+47.72%)482.80 (-17.92%)357.10 (+4.35%)604.82 (+422.47%)26.43 (-4.16%)16.66 (-9.35%)19.55 (+21.85%)5.21 (-65.15%)8.62 (+64.59%)
9c70ba8 — 2026-06-29 16:29:160.65 (n/a)0.43 (n/a)0.38 (n/a)0.35 (n/a)0.12 (n/a)631.70 (n/a)539.58 (n/a)588.20 (n/a)342.20 (n/a)115.76 (n/a)27.58 (n/a)18.38 (n/a)16.04 (n/a)14.94 (n/a)5.23 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:450.54 (-0.96%)0.40 (-10.61%)0.36 (-16.16%)0.32 (-10.08%)0.09 (+10.75%)690.20 (+11.21%)577.68 (+13.04%)616.90 (+19.28%)410.50 (+0.96%)117.26 (+26.01%)22.99 (-0.96%)16.96 (-10.61%)15.30 (-16.16%)13.67 (-10.08%)3.89 (+10.75%)
9c70ba8 — 2026-06-29 16:29:160.54 (n/a)0.44 (n/a)0.43 (n/a)0.36 (n/a)0.08 (n/a)620.60 (n/a)511.02 (n/a)517.20 (n/a)406.60 (n/a)93.06 (n/a)23.21 (n/a)18.98 (n/a)18.25 (n/a)15.21 (n/a)3.51 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:450.31 (-0.86%)0.30 (-1.17%)0.30 (-2.06%)0.30 (+0.90%)0.00 (-38.73%)83928.20 (-0.90%)83147.14 (+1.16%)83386.50 (+2.10%)81679.40 (+0.86%)927.64 (-38.80%)210.33 (-0.86%)206.64 (-1.17%)206.03 (-2.06%)204.70 (+0.90%)2.32 (-38.73%)
9c70ba8 — 2026-06-29 16:29:160.31 (n/a)0.31 (n/a)0.31 (n/a)0.30 (n/a)0.01 (n/a)84687.50 (n/a)82190.72 (n/a)81670.10 (n/a)80980.40 (n/a)1515.67 (n/a)212.15 (n/a)209.08 (n/a)210.36 (n/a)202.86 (n/a)3.79 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:451.03 (+0.93%)1.01 (+2.70%)1.01 (+0.17%)0.98 (+8.25%)0.02 (-62.41%)25584.30 (-7.62%)24942.66 (-2.79%)24885.50 (-0.17%)24431.70 (-0.92%)445.36 (-65.50%)703.18 (+0.93%)688.95 (+2.70%)690.36 (+0.17%)671.50 (+8.25%)12.23 (-62.41%)
9c70ba8 — 2026-06-29 16:29:161.02 (n/a)0.98 (n/a)1.01 (n/a)0.91 (n/a)0.05 (n/a)27693.90 (n/a)25659.76 (n/a)24928.50 (n/a)24658.50 (n/a)1291.06 (n/a)696.71 (n/a)670.84 (n/a)689.16 (n/a)620.35 (n/a)32.54 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:454.25 (+87.13%)2.96 (+70.40%)3.43 (+113.16%)1.47 (+10.12%)1.23 (+201.45%)5495.10 (-9.19%)3247.08 (-33.01%)2349.70 (-53.09%)1896.70 (-46.56%)1597.25 (+47.14%)1114.54 (+87.13%)775.59 (+70.40%)899.68 (+113.16%)384.69 (+10.12%)322.70 (+201.45%)
9c70ba8 — 2026-06-29 16:29:162.27 (n/a)1.74 (n/a)1.61 (n/a)1.33 (n/a)0.41 (n/a)6051.30 (n/a)4847.08 (n/a)5008.50 (n/a)3549.20 (n/a)1085.54 (n/a)595.60 (n/a)455.16 (n/a)422.07 (n/a)349.34 (n/a)107.05 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:450.33 (+13.20%)0.23 (+4.28%)0.20 (-2.41%)0.18 (+3.22%)0.06 (+35.63%)6768.70 (-3.12%)5654.94 (-2.61%)6171.20 (+2.46%)3800.30 (-11.66%)1181.67 (+16.03%)17.66 (+13.20%)12.39 (+4.28%)10.87 (-2.41%)9.91 (+3.22%)3.14 (+35.63%)
9c70ba8 — 2026-06-29 16:29:160.29 (n/a)0.22 (n/a)0.21 (n/a)0.18 (n/a)0.04 (n/a)6986.90 (n/a)5806.54 (n/a)6022.80 (n/a)4302.00 (n/a)1018.41 (n/a)15.60 (n/a)11.88 (n/a)11.14 (n/a)9.60 (n/a)2.32 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:450.12 (-8.24%)0.09 (+18.35%)0.08 (+6.47%)0.07 (+64.66%)0.03 (-24.60%)0.12 (-8.24%)0.09 (+18.35%)0.07 (+6.47%)0.06 (+64.66%)0.03 (-24.60%)
9c70ba8 — 2026-06-29 16:29:160.13 (n/a)0.07 (n/a)0.07 (n/a)0.04 (n/a)0.03 (n/a)0.13 (n/a)0.07 (n/a)0.07 (n/a)0.04 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:453.82 (+1.73%)3.67 (+1.99%)3.78 (+2.05%)3.38 (+0.11%)0.19 (+7.68%)3.82 (+1.73%)3.66 (+1.99%)3.77 (+2.05%)3.38 (+0.11%)0.19 (+7.68%)
9c70ba8 — 2026-06-29 16:29:163.76 (n/a)3.59 (n/a)3.70 (n/a)3.38 (n/a)0.18 (n/a)3.75 (n/a)3.59 (n/a)3.70 (n/a)3.38 (n/a)0.18 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:457.54 (+2.59%)6.18 (-0.88%)5.85 (-0.35%)4.78 (-14.49%)1.15 (+45.24%)7.54 (+2.59%)6.18 (-0.88%)5.84 (-0.35%)4.78 (-14.49%)1.15 (+45.24%)
9c70ba8 — 2026-06-29 16:29:167.35 (n/a)6.24 (n/a)5.87 (n/a)5.59 (n/a)0.79 (n/a)7.35 (n/a)6.23 (n/a)5.86 (n/a)5.59 (n/a)0.79 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:4513.43 (+5.18%)10.03 (+6.16%)9.20 (+9.98%)7.86 (-2.38%)2.39 (+19.01%)13.42 (+5.18%)10.02 (+6.16%)9.19 (+9.98%)7.86 (-2.38%)2.39 (+19.01%)
9c70ba8 — 2026-06-29 16:29:1612.77 (n/a)9.45 (n/a)8.36 (n/a)8.06 (n/a)2.01 (n/a)12.76 (n/a)9.44 (n/a)8.36 (n/a)8.05 (n/a)2.01 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:453.91 (+2.18%)3.68 (-1.14%)3.72 (-2.34%)3.40 (-0.99%)0.21 (+25.03%)3.90 (+2.18%)3.68 (-1.14%)3.72 (-2.34%)3.39 (-0.99%)0.21 (+25.03%)
9c70ba8 — 2026-06-29 16:29:163.82 (n/a)3.72 (n/a)3.81 (n/a)3.43 (n/a)0.17 (n/a)3.82 (n/a)3.72 (n/a)3.81 (n/a)3.43 (n/a)0.17 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:457.11 (+5.46%)6.40 (+1.43%)6.55 (-2.38%)5.32 (-5.42%)0.75 (+30.89%)7.10 (+5.46%)6.39 (+1.43%)6.55 (-2.38%)5.31 (-5.42%)0.74 (+30.89%)
9c70ba8 — 2026-06-29 16:29:166.74 (n/a)6.31 (n/a)6.71 (n/a)5.62 (n/a)0.57 (n/a)6.73 (n/a)6.30 (n/a)6.71 (n/a)5.62 (n/a)0.57 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
3ab7ee2 — 2026-06-29 17:18:4511.23 (-19.99%)8.79 (-16.32%)8.28 (-2.81%)7.60 (-6.31%)1.42 (-52.05%)11.23 (-19.99%)8.79 (-16.32%)8.27 (-2.81%)7.59 (-6.31%)1.41 (-52.05%)
9c70ba8 — 2026-06-29 16:29:1614.04 (n/a)10.51 (n/a)8.52 (n/a)8.11 (n/a)2.95 (n/a)14.03 (n/a)10.50 (n/a)8.51 (n/a)8.10 (n/a)2.95 (n/a)

test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]

No metrics available.

test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]

No metrics available.

test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]

No metrics available.

test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]

No metrics available.

iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)583.20 (n/a)354.44 (n/a)285.90 (n/a)223.40 (n/a)152.76 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)523.50 (n/a)341.88 (n/a)299.00 (n/a)213.00 (n/a)134.23 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)487.90 (n/a)389.20 (n/a)389.20 (n/a)269.80 (n/a)91.15 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)570.50 (n/a)378.54 (n/a)301.80 (n/a)250.30 (n/a)145.36 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.01 (n/a)1934.10 (n/a)680.74 (n/a)432.10 (n/a)265.90 (n/a)705.97 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)765.00 (n/a)492.38 (n/a)478.70 (n/a)278.20 (n/a)175.72 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (+23.21%)0.02 (+8.58%)0.02 (-3.00%)0.01 (+21.49%)0.01 (+4.20%)546.50 (-17.68%)359.64 (-10.05%)332.10 (+3.10%)239.10 (-18.81%)114.18 (-27.14%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)663.90 (n/a)399.80 (n/a)322.10 (n/a)294.50 (n/a)156.72 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (+3.07%)0.02 (-0.53%)0.02 (-21.00%)0.01 (+217.73%)0.01 (-21.02%)591.90 (-68.52%)411.70 (-34.31%)372.60 (+26.61%)229.10 (-2.96%)169.64 (-75.97%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.03 (n/a)0.00 (n/a)0.01 (n/a)1880.50 (n/a)626.76 (n/a)294.30 (n/a)236.10 (n/a)705.97 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-11.00%)0.02 (+20.66%)0.02 (+42.69%)0.02 (+296.35%)0.01 (-55.68%)491.20 (-74.77%)393.60 (-48.04%)406.80 (-29.92%)299.30 (+12.39%)90.54 (-86.93%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)0.01 (n/a)1946.90 (n/a)757.54 (n/a)580.50 (n/a)266.30 (n/a)692.98 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-5.84%)0.02 (-15.96%)0.01 (-17.63%)0.01 (-32.68%)0.01 (+22.00%)775.60 (+48.55%)555.62 (+27.17%)577.90 (+21.41%)291.00 (+6.20%)190.98 (+97.80%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)522.10 (n/a)436.90 (n/a)476.00 (n/a)274.00 (n/a)96.55 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.06 (+59.00%)0.03 (+45.96%)0.03 (+73.40%)0.01 (-2.86%)0.02 (+91.72%)643.00 (+2.93%)363.40 (-20.76%)273.90 (-42.32%)146.90 (-37.09%)201.26 (+32.31%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)624.70 (n/a)458.58 (n/a)474.90 (n/a)233.50 (n/a)152.11 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (-15.49%)0.02 (-2.36%)0.02 (+4.44%)0.01 (+0.16%)0.00 (-34.02%)635.40 (-0.16%)507.60 (-0.81%)537.00 (-4.24%)366.20 (+18.32%)102.33 (-21.87%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)636.40 (n/a)511.74 (n/a)560.80 (n/a)309.50 (n/a)130.97 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-27.21%)0.02 (-18.19%)0.02 (-38.93%)0.01 (-11.19%)0.01 (-22.03%)551.10 (+12.61%)426.86 (+22.26%)530.10 (+63.76%)242.60 (+37.37%)155.36 (+22.62%)
9c70ba8 — 2026-06-29 16:29:160.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)489.40 (n/a)349.14 (n/a)323.70 (n/a)176.60 (n/a)126.69 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.05 (+44.69%)0.03 (+20.50%)0.03 (+24.73%)0.02 (-16.53%)0.01 (+261.07%)614.10 (+19.80%)422.84 (-6.54%)358.30 (-19.81%)262.10 (-30.90%)172.65 (+205.67%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)512.60 (n/a)452.44 (n/a)446.80 (n/a)379.30 (n/a)56.48 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (+8.97%)0.02 (-6.08%)0.02 (-11.58%)0.01 (-9.22%)0.01 (+11.85%)661.60 (+10.16%)462.26 (+7.52%)459.80 (+13.08%)217.00 (-8.25%)163.19 (-1.20%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)600.60 (n/a)429.92 (n/a)406.60 (n/a)236.50 (n/a)165.18 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (-2.37%)0.02 (-15.10%)0.02 (-17.65%)0.02 (+3.93%)0.01 (-15.70%)609.90 (-3.79%)477.58 (+14.76%)504.80 (+21.43%)281.70 (+2.40%)129.76 (-13.33%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)633.90 (n/a)416.14 (n/a)415.70 (n/a)275.10 (n/a)149.72 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-3.90%)0.02 (+6.83%)0.02 (+22.06%)0.02 (+5.88%)0.01 (-18.56%)534.70 (-5.55%)368.66 (-10.34%)363.10 (-18.07%)235.00 (+4.03%)115.73 (-21.22%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)566.10 (n/a)411.16 (n/a)443.20 (n/a)225.90 (n/a)146.90 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.04 (-13.04%)0.02 (-8.85%)0.02 (+0.85%)0.01 (-27.21%)0.01 (-13.35%)848.90 (+37.36%)532.22 (+11.67%)511.80 (-0.83%)244.60 (+15.00%)220.18 (+39.11%)
9c70ba8 — 2026-06-29 16:29:160.05 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)618.00 (n/a)476.62 (n/a)516.10 (n/a)212.70 (n/a)158.28 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-24.74%)0.02 (-7.92%)0.02 (-10.53%)0.02 (+42.64%)0.01 (-42.91%)533.10 (-29.90%)419.42 (-4.74%)463.00 (+11.75%)286.40 (+32.90%)116.22 (-46.73%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)760.50 (n/a)440.30 (n/a)414.30 (n/a)215.50 (n/a)218.15 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (-31.05%)0.02 (-8.76%)0.02 (+4.02%)0.02 (+15.53%)0.00 (-57.07%)588.30 (-13.45%)508.66 (-0.09%)529.80 (-3.86%)352.80 (+45.07%)91.42 (-43.18%)
9c70ba8 — 2026-06-29 16:29:160.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)679.70 (n/a)509.14 (n/a)551.10 (n/a)243.20 (n/a)160.89 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (+17.61%)0.02 (-3.75%)0.02 (-23.61%)0.01 (-9.90%)0.01 (+30.06%)619.60 (+10.98%)441.26 (+9.36%)507.20 (+30.92%)234.30 (-14.95%)166.16 (+25.50%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)558.30 (n/a)403.50 (n/a)387.40 (n/a)275.50 (n/a)132.40 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.03 (+17.83%)0.02 (-6.66%)0.02 (-25.29%)0.02 (+6.11%)0.01 (+28.17%)592.50 (-5.76%)472.16 (+8.84%)503.30 (+33.86%)290.80 (-15.12%)124.02 (+3.14%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)628.70 (n/a)433.80 (n/a)376.00 (n/a)342.60 (n/a)120.25 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.02 (-21.92%)0.01 (-24.47%)0.01 (-17.01%)0.00 (-70.48%)0.01 (+12.35%)1937.90 (+238.73%)819.16 (+69.37%)624.40 (+20.49%)374.10 (+28.07%)634.45 (+477.99%)
9c70ba8 — 2026-06-29 16:29:160.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)572.10 (n/a)483.64 (n/a)518.20 (n/a)292.10 (n/a)109.77 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.33 (-14.17%)0.28 (+7.56%)0.31 (+0.19%)0.22 (+347.65%)0.05 (-62.08%)456.40 (-77.66%)358.70 (-47.03%)313.50 (-0.19%)299.10 (+16.47%)74.44 (-90.34%)
9c70ba8 — 2026-06-29 16:29:160.38 (n/a)0.26 (n/a)0.31 (n/a)0.05 (n/a)0.14 (n/a)2043.20 (n/a)677.16 (n/a)314.10 (n/a)256.80 (n/a)770.83 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.34 (-33.63%)0.24 (-34.51%)0.20 (-48.53%)0.16 (-16.06%)0.08 (-32.25%)599.10 (+19.15%)435.78 (+48.93%)480.70 (+94.30%)292.40 (+50.64%)130.56 (+7.69%)
9c70ba8 — 2026-06-29 16:29:160.51 (n/a)0.37 (n/a)0.40 (n/a)0.20 (n/a)0.11 (n/a)502.80 (n/a)292.60 (n/a)247.40 (n/a)194.10 (n/a)121.24 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.41 (-8.53%)0.31 (+2.60%)0.33 (+1.59%)0.22 (+30.98%)0.08 (-33.91%)452.00 (-23.65%)338.24 (-11.46%)297.60 (-1.55%)242.30 (+9.34%)93.69 (-45.29%)
9c70ba8 — 2026-06-29 16:29:160.44 (n/a)0.30 (n/a)0.33 (n/a)0.17 (n/a)0.12 (n/a)592.00 (n/a)382.00 (n/a)302.30 (n/a)221.60 (n/a)171.27 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.24 (-16.27%)0.17 (-20.17%)0.15 (-38.12%)0.11 (-19.35%)0.06 (-18.51%)653.60 (+24.00%)463.44 (+24.01%)494.90 (+61.57%)311.20 (+19.46%)149.72 (+10.71%)
9c70ba8 — 2026-06-29 16:29:160.28 (n/a)0.22 (n/a)0.24 (n/a)0.14 (n/a)0.07 (n/a)527.10 (n/a)373.70 (n/a)306.30 (n/a)260.50 (n/a)135.24 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.29 (-10.38%)0.18 (-25.26%)0.15 (-49.11%)0.04 (-71.23%)0.10 (+9.01%)1972.50 (+247.52%)710.66 (+91.35%)494.50 (+96.46%)256.90 (+11.55%)717.75 (+308.19%)
9c70ba8 — 2026-06-29 16:29:160.32 (n/a)0.23 (n/a)0.29 (n/a)0.13 (n/a)0.10 (n/a)567.60 (n/a)371.40 (n/a)251.70 (n/a)230.30 (n/a)175.84 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.25 (-15.92%)0.18 (-11.41%)0.20 (-4.01%)0.07 (-45.26%)0.08 (+5.23%)1093.20 (+82.69%)529.46 (+27.42%)369.10 (+4.18%)293.80 (+18.95%)338.38 (+106.02%)
9c70ba8 — 2026-06-29 16:29:160.30 (n/a)0.20 (n/a)0.21 (n/a)0.12 (n/a)0.08 (n/a)598.40 (n/a)415.52 (n/a)354.30 (n/a)247.00 (n/a)164.24 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.56 (+1.79%)0.34 (-7.01%)0.30 (-7.63%)0.20 (-9.33%)0.14 (+4.96%)643.00 (+10.29%)439.10 (+9.63%)442.40 (+8.27%)232.70 (-1.77%)163.98 (+14.77%)
9c70ba8 — 2026-06-29 16:29:160.55 (n/a)0.37 (n/a)0.32 (n/a)0.22 (n/a)0.14 (n/a)583.00 (n/a)400.52 (n/a)408.60 (n/a)236.90 (n/a)142.88 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.58 (+16.92%)0.38 (+9.23%)0.33 (+15.64%)0.24 (+3.10%)0.15 (+20.29%)553.50 (-3.01%)385.22 (-6.42%)397.00 (-13.53%)227.90 (-14.48%)140.64 (+4.25%)
9c70ba8 — 2026-06-29 16:29:160.49 (n/a)0.35 (n/a)0.29 (n/a)0.23 (n/a)0.12 (n/a)570.70 (n/a)411.66 (n/a)459.10 (n/a)266.50 (n/a)134.91 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.55 (+60.39%)0.40 (+37.53%)0.38 (+38.60%)0.27 (+9.94%)0.12 (+184.07%)489.60 (-9.03%)349.20 (-23.16%)342.70 (-27.85%)237.20 (-37.66%)103.97 (+61.62%)
9c70ba8 — 2026-06-29 16:29:160.34 (n/a)0.29 (n/a)0.28 (n/a)0.24 (n/a)0.04 (n/a)538.20 (n/a)454.48 (n/a)475.00 (n/a)380.50 (n/a)64.33 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.00 (-14.29%)0.00 (-21.05%)0.00 (+0.00%)0.00 (+0.00%)0.00 (-30.44%)21107.06 (-5.48%)16040.39 (+3.63%)17412.21 (-16.82%)7433.15 (+30.00%)5165.20 (-37.81%)
9c70ba8 — 2026-06-29 16:29:160.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)22329.93 (n/a)15477.94 (n/a)20933.44 (n/a)5717.79 (n/a)8304.94 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.00 (+0.00%)0.00 (-8.33%)0.00 (+0.00%)0.00 (+25.00%)0.00 (-19.41%)16880.81 (-20.37%)13460.43 (-1.06%)14650.22 (+3.14%)7181.40 (-4.68%)3712.21 (-37.50%)
9c70ba8 — 2026-06-29 16:29:160.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)21199.65 (n/a)13604.77 (n/a)14204.05 (n/a)7534.07 (n/a)5939.55 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:450.13 (-1.90%)0.10 (-16.08%)0.08 (-30.45%)0.08 (-9.25%)0.02 (-6.07%)27424.53 (+10.24%)22942.67 (+18.96%)24885.82 (+43.87%)15591.46 (+1.95%)4562.03 (+1.92%)
9c70ba8 — 2026-06-29 16:29:160.14 (n/a)0.11 (n/a)0.12 (n/a)0.08 (n/a)0.02 (n/a)24876.33 (n/a)19286.40 (n/a)17297.26 (n/a)15293.44 (n/a)4476.13 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:451.84 (+27.55%)1.33 (+33.35%)1.54 (+72.86%)0.26 (-68.86%)0.62 (+142.94%)2024.00 (+221.07%)668.90 (+22.43%)341.20 (-42.15%)284.50 (-21.60%)758.14 (+595.49%)
9c70ba8 — 2026-06-29 16:29:161.44 (n/a)1.00 (n/a)0.89 (n/a)0.83 (n/a)0.26 (n/a)630.40 (n/a)546.34 (n/a)589.80 (n/a)362.90 (n/a)109.01 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:452.29 (-6.31%)1.72 (+30.08%)2.20 (+41.89%)0.30 (+1.10%)0.85 (-14.01%)3527.80 (-1.09%)1120.96 (-35.60%)477.00 (-29.52%)458.20 (+6.76%)1348.44 (-17.94%)
9c70ba8 — 2026-06-29 16:29:162.44 (n/a)1.32 (n/a)1.55 (n/a)0.29 (n/a)0.99 (n/a)3566.50 (n/a)1740.52 (n/a)676.80 (n/a)429.20 (n/a)1643.15 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
3ab7ee2 — 2026-06-29 17:18:451.69 (+31.49%)1.17 (+12.97%)1.02 (-4.52%)0.67 (-11.39%)0.44 (+129.70%)778.60 (+12.86%)504.30 (-3.32%)515.00 (+4.74%)310.40 (-23.94%)192.53 (+83.06%)
9c70ba8 — 2026-06-29 16:29:161.28 (n/a)1.04 (n/a)1.07 (n/a)0.76 (n/a)0.19 (n/a)689.90 (n/a)521.62 (n/a)491.70 (n/a)408.10 (n/a)105.17 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)
Phoenix - Examples

IRON

Tested on 2026_06_29_17_25_24 at commit 3ab7ee2.

Trends:

IRON Trends

@andrej

andrej commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Oh, one more thing, could you please add the latency/gflops metrics to the batched tests as well? I know it's not technically part of this PR but it would be nice to keep track of the performance numbers in the CI results:

    print(f"Throughput: {gflops:.6e} GFLOP/s")
    print(f"Effective Bandwidth: {bandwidth_gbps:.6e} GB/s\n")
image image

atassis added a commit to atassis/IRON that referenced this pull request Jun 30, 2026
@atassis
gemv: address review (explain non-linear TAP, fix backpressure commen…
3e84c40 
…t, dedup branches)

Responds to andrej's review on amd#127:
- Explain the coalesced TAP is NOT a single linear transfer: contiguous within a batch
  but strided by the full matrix across batches (M*K != run for cols>1), so the batch
  iteration dim and the TAP are required. The inner run-split is only to fit the 1023
  wrap cap; note it is removable once IRON's mlir_aie pin moves past mlir-aie #3036.
- Add FIXME to source the shim BD bounds from the MLIR-AIE target model (AIEXDialect.cpp).
- Reword the depth>=2 rationale: ObjectFifo backpressure blocks (never overruns/corrupts);
  depth>=2 is a performance/overlap guard, not correctness.
- Dedup the coalesced/fallback branches into one loop over num_waits (1 vs num_batches).
@atassis

atassis commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Pushed all of the above. I also added @pytest.mark.metrics to test_gemv_batched - it was correctness-only before, so the coalescing speedup wasn't showing up in CI.

@atassis atassis requested a review from andrej June 30, 2026 12:56
@andrej

andrej commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Thanks for looking into those suggestions. I did just update the pinned MLIR-AIE. If you can look into whether that helps you simplify any of the linearizing code, that would be great. After that, or if it doesn't end up helping you, I'll run the workflows and merge :)

atassis added 3 commits July 10, 2026 21:38
@atassis
gemv: coalesce batched DMA into one iterated BD per column (default)
5cf9474 
The batched GEMV unrolled num_batches host DMA descriptors per column (one fill +
one drain + a task-group wait per batch). Express the batch as a single iterated BD
instead, the same B-unroll -> BD-iteration idiom GEMM already uses for tiling: place
num_batches in the size-uncapped descriptor dim and split the contiguous run across
the two wrap dims (each <= 1023), per the AIE shim BD limits (AIEXDialect.cpp
verifyStridesWraps). The run split keeps the inner size and strides 4-byte-aligned
(granularity-aware), and the per-batch drain wait is dropped in favour of ObjectFifo
backpressure (fifo depth >= 2 asserted; the A-fill and C-drain run on separate shim
channels, paced by the fifo locks).

num_batches == 1 and any config that cannot be coalesced (run with no aligned split
under the wrap cap, or a batch stride that is too large or unaligned) fall back to
the existing per-batch path and are byte-identical. The coalesced descriptor accesses
the exact same DRAM elements in the same order as the unroll (access-equivalent), so
this is a descriptor-count / build-time and correctness change, not a runtime change.

(cherry picked from commit d4bde97)
@atassis
gemv: add num_batches > 1 golden tests
72940a2 
The test suite had no num_batches > 1 GEMV coverage. Add a batched golden reference
(num_batches independent matrix-vector products, stacked contiguously) and a
parametrized device test covering: the coalesced path with large num_batches (the
size-uncapped dim) and a multi-dimension run split; a run that requires an aligned
(even) inner split; and the per-batch fallback (batch stride over the limit).

(cherry picked from commit 6d9a845)
@atassis
gemv: address review (explain non-linear TAP, fix backpressure commen…
95d1c67 
…t, dedup branches)

Responds to andrej's review on amd#127:
- Explain the coalesced TAP is NOT a single linear transfer: contiguous within a batch
  but strided by the full matrix across batches (M*K != run for cols>1), so the batch
  iteration dim and the TAP are required. The inner run-split is only to fit the 1023
  wrap cap; note it is removable once IRON's mlir_aie pin moves past mlir-aie #3036.
- Add FIXME to source the shim BD bounds from the MLIR-AIE target model (AIEXDialect.cpp).
- Reword the depth>=2 rationale: ObjectFifo backpressure blocks (never overruns/corrupts);
  depth>=2 is a performance/overlap guard, not correctness.
- Dedup the coalesced/fallback branches into one loop over num_waits (1 vs num_batches).
@atassis
gemv: record that the run split is required for cols>1 even on v1.3.4
cffa6a9 
Verified on the v1.3.4 pin (mlir-aie #2924 + #3036): the contiguous-run
linearization only bypasses the 1023 wrap cap when the whole transfer is
contiguous (cols==1, bstride==run). For a real cols>1 gather the inner run stays
a wrap dim -- [1, num_batches, 4096] fails aiecc 'Size 4096 exceeds [0:1023]'.
Correcting the earlier speculative FUTURE note; the split_run stays.
@atassis

atassis commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@andrej Hi there
I have looked into it on v1.3.4. The contiguous-run linearization (#2924/#3036) only drops the split when the whole transfer is contiguous, i.e. cols==1 (bstride==run) - there [1, num_batches, run] lowers to a single buffer-length transfer. For a real cols>1 gather the batch stride leaves a gap, so the inner run stays a wrap dim: [1, num_batches, 4096] fails aiecc with "Size 4096 exceeds [0:1023]". So the split stays for cols>1. I rebased onto devel and migrated to the new placer; the full gemv suite (80 tests, batched + non-batched) passes on NPU2 with the v1.3.4 wheel. Ready for your workflows whenever

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jgmelber jgmelber Awaiting requested review from jgmelber jgmelber is a code owner
@hunhoffe hunhoffe Awaiting requested review from hunhoffe hunhoffe is a code owner
@andrej andrej Awaiting requested review from andrej andrej is a code owner

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@atassis @andrej