Issues

• BUG: nemotron-bolt-24gb export fails during kernel tuning on SM120: b12x

  NVIDIA/hyperflare-inference#278 · Filter by author roobyz opened on Nov 12, 2026

• GLM-5.3 Flash across two hosts: compiled-forward binding fails on a rank-local guard

  NVIDIA/hyperflare-inference#276 · Filter by author jetnet opened on Nov 12, 2026

• Multi-host RoCE cores exported at 0572fd12 do not start: pinned FlashInfer 7b35510b records proxy ABI 14, the runtime provides 11

  NVIDIA/hyperflare-inference#271 · Filter by author javieratomic opened on Nov 12, 2026

• GB10 TP3: DeepSeek V4.1 Flash export at 0572fd12 fails kernel tuning with a 0-byte scratch limit; 7.1-7.9 GiB available, below the 8 GiB floor

  NVIDIA/hyperflare-inference#270 · Filter by author javieratomic opened on Nov 12, 2026
  1

• Engram row cache: four-way sets evict or refuse re-read rows while a third of slots are free (87 % vs 99.8 % re-read hit rate at 1 vs 4 GiB)

  NVIDIA/hyperflare-inference#269 · Filter by author javieratomic opened on Nov 12, 2026

• BUG: qwen3.8-27b export fails warming compiled forwards on SM120

  NVIDIA/hyperflare-inference#268 · Filter by author roobyz opened on Nov 12, 2026

• Can we get c2606aa merged in?

  NVIDIA/hyperflare-inference#267 · Filter by author officiallymarky opened on Nov 11, 2026
  2

• First captured graph keeps a transient cuBLAS split-K workspace: Xid 31 on DeepSeek V4.1 TP4 with --prefill-chunk-size 8192 at 7f534bb6

  NVIDIA/hyperflare-inference#266 · Filter by author calvarado2004 opened on Nov 11, 2026
  3

• GB10 TP1 (Qwen3.8 Flash Next): alpha package at --max-concurrency 16 gives +47% KV and faster C1/C4; one-box DFlash2+A16 export does not beat MTP

  NVIDIA/hyperflare-inference#263 · Filter by author CaptKitCarson opened on Nov 11, 2026

• Host prefix cache: ~30 % of hits dropped as capacity_rejected and fully recomputed

  NVIDIA/hyperflare-inference#255 · Filter by author alejandroed opened on Nov 10, 2026

• Retrospective DCO confirmation by Martin Vit for 131 commits

  NVIDIA/hyperflare-inference#254 · Filter by author voipmonitor opened on Nov 10, 2026

• DeepSeek-V4-Flash-Vision-Exp TP2/TP4: b12x tiny-decode MoE kernel skips swiglu_limit (fix in local-inference-lab/flashinfer#83)

  NVIDIA/hyperflare-inference#249 · Filter by author AzallazA opened on Nov 10, 2026
  1

• Multi-host export compiles for 22 minutes, then fails, when one rank has no memory for a forward compiler

  NVIDIA/hyperflare-inference#242 · Filter by author javieratomic opened on Nov 9, 2026
  1

• break out engram wait time

  area:cachingarea:engramenhancementperformancetriage:design-review
  NVIDIA/hyperflare-inference#230 · Filter by author officiallymarky opened on Nov 8, 2026
  1

• WARM prefix-cache regression under KV pressure on DGX Spark (Qwen3.8 Flash Next, TP1)

  area:cachingarea:schedulingarea:selenebugmodel:qwen3.8-flash-nextperformancepriority:P2triage:review
  NVIDIA/hyperflare-inference#225 · Filter by author shadownetX opened on Nov 7, 2026
  1

• Qwen3.8 Flash Next DFlash2: mixed prefill rounds, YaRN serving, and served measurements from a fork

  model:qwen3.8-flash-nexttriage:design-review
  NVIDIA/hyperflare-inference#210 · Filter by author Raydius opened on Nov 5, 2026

• Payload inspection omits the selected deployment rebuild manifest

  area:packagingbugpriority:P2triage:review
  NVIDIA/hyperflare-inference#195 · Filter by author 0z5a opened on Nov 4, 2026

• DeepSeek V4.1 Flash gb202 TP4: next-token logits intermittently corrupt mid-reply (stray "##", "-", "<ds_safety>" classifier text in answers)

  bugmodel:deepseek-v4.1priority:P1triage:investigate
  NVIDIA/hyperflare-inference#193 · Filter by author mmssix opened on Nov 4, 2026
  9

  

• Multi-host prefix cache: startup row thinning of a recovered disk store overflows the 64-frame record backlog and stops the server

  area:cachingarea:selenebugmodel:deepseek-v4.1priority:P1triage:investigate
  NVIDIA/hyperflare-inference#189 · Filter by author javieratomic opened on Nov 3, 2026
  1

• DeepSeek 4.1 TP3 on GB10: 0.46 GiB/GPU less prefill scratch gave +7.6 % KV tokens (measurement only)

  area:memoryarea:preparationmodel:deepseek-v4.1performancetriage:design-review
  NVIDIA/hyperflare-inference#186 · Filter by author javieratomic opened on Nov 3, 2026
  2

• Roadmap question: prefill/decode disaggregation roles, now that the prefix cache pool exists?

  area:cachingarea:selenetriage:design-review
  NVIDIA/hyperflare-inference#160 · Filter by author JCTaz9 opened on Oct 31, 2026

• Multi-host export: when rank zero rejects a peer's checkpoint completion, the reason is never reported

  area:preparationbugmodel:deepseek-v4.1priority:P2triage:review
  NVIDIA/hyperflare-inference#156 · Filter by author javieratomic opened on Oct 30, 2026

• DeepSeek V4.1 Flash TP3 on GB10: a speculative round costs 1.4-2x a width-one round, a fixed draft cost plus about 4 ms per verified row

  area:kernelsarea:schedulingmodel:deepseek-v4.1performancetriage:design-review
  NVIDIA/hyperflare-inference#153 · Filter by author javieratomic opened on Oct 30, 2026
  1

• DeepSeek V4.1 Flash TP3 on GB10: with four requests, the balanced policy verifies most rounds at width one; throughput is 8% faster on prose

  area:schedulingmodel:deepseek-v4.1performancetriage:design-review
  NVIDIA/hyperflare-inference#152 · Filter by author javieratomic opened on Oct 30, 2026
  1

• GB10 TP2 serving: first request times out on RoCE collective (HCA 0, sequence 1) when both CX-7 rails are active; single rail works

  area:rocenantebugmodel:glm-5.3priority:P2triage:investigate
  NVIDIA/hyperflare-inference#151 · Filter by author evehour opened on Oct 30, 2026
  2