Skip to content

From NVIDIA Megatron-LM for visibility#18

Open
RaymondLi0 wants to merge 7316 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7316 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0 RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0 RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
ko3n1g and others added 27 commits June 1, 2026 17:52
@ko3n1g
ci: prune old artifacts on cluster lustre during weekly/release runs (#…
595f697 
…5084)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g @claude
ci(test): isolate ckpt-resume tensorboard per phase (#5074)
46f1af7 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lhb8125
test: unmark EP A2A activation offload test flaky (#5009)
33da12c
@Phlip79
Change ownership groups (#5021)
e656940
@wujingyue @claude
test: skip mfsdp_fully_shard cases when world_size < mesh size (#4487)
496e1ff 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@liding-nv
fix mimo optimizer checkpoint metadata restore (#4791)
80cf756 
Signed-off-by: Li Ding <liding@nvidia.com>
@liding-nv
[mimo] Support bridge fan-out for variable modality tokens (#5062)
de030fc 
Signed-off-by: Li Ding <liding@nvidia.com>
@yfw @claude
Add separate mtp_grad_scale_func for MTP loss scaling (#3459)
b8dcaab 
Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@maanug-nv
[training migration] Migrate GPT builder (#4741)
5f88438 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@wujingyue
Make Mamba conv params direct mixer params (#4899)
35992ba 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
907ebcd
@Phlip79
Update oncall reviewer assignment (#5093)
f5da5ea
@yashaswikarnati
Pass explicit process groups to hybrid logging (#4781)
bb192ff
@Phlip79
Clean up top-level repository files (#5097)
7d9f259
@yuzhongw-nvidia
[main] fix(moe): Fix several bugs for DSA rope and spec. (#3026)
b0eb914 
Co-authored-by: Yuzhong Wang <yuzhongw@computelab-frontend-3.nvidia.com>
@yashaswikarnati
Move MIMO unit tests into models/mimo (#5063)
c69697d
@wujingyue
test: update DeepSeek FSDP2 GB200 memory golden (#5094)
54e1996
@janEbert
Remove DeepEP hardware limit check (#4846)
a1ac2f6
@balasaajay
Update transformer-engine dependency to revision 4220403 (#5112)
591605e
@ko3n1g @claude
ci: make CI resilient to pip/uv network timeouts (#5118)
44f254a 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ko3n1g @claude
ci: treat docker container-removal conflict as flaky (#5120)
d94604e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@conver334
Fix GDN DTensor splitting for FSDP checkpointing (#4843)
655a064 
Signed-off-by: conver334 <conver334@gmail.com>
@deepakn94 @claude
Fix MoE aux_loss / z_loss gradient scaling with TP > 1 (#5047)
9ef8a2a 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
540ff5d
@balasaajay
Update Claude copy workflow to enforce user restrictions and improve …
eac5ca7 
…error messages (#5117)
@yashaswikarnati
Add advisory process group guidance to Claude reviews (#5111)
a823ea0
@cuichenx
build: cap pydantic<2.14 in transformer-engine dependency metadata (#…
3b6bc31 
…5125)

Signed-off-by: Chen Cui <chcui@nvidia.com>
github-actions Bot and others added 30 commits July 6, 2026 12:13
@github-actions
chore(beep boop 🤖): Bump (main) (2026-07-06)
6856424
@santhnm2
Pre-size the all-gather buffer for inference to max capacity (#5546)
959f3f1 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@kevalmorabia97 @claude
Scatter embeddings for sequence parallelism in standalone LM forwards (…
7ee524e 
…#5628)

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@deepakn94 @claude
Fix inter-document masking crash and NaNs with TP > 1 and micro_batch…
bf32f44 
…_size > 1 (#5635)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@dimapihtar
add safe version of numpy.load (#5500)
1bb5ff5 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@sraman-rgb
Fuse shared expert MLP with grouped GEMM (#5604)
115ce7f 
Signed-off-by: Siddhartha Raman Sundara Raman <sraman@nvidia.com>
@mathemakitten
MoE routing analysis and metrics capture (#5220)
c1560d9 
Signed-off-by: Helen Ngo <helenn@nvidia.com>
@Phlip79
Add cspades to oncall rotation (#5695)
c797a5e 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@wujingyue
Add microbatch context helper (#5652)
5dbea46 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@balasaajay
ci: Update test configurations to unify legacy scope names (#5316)
68f4c62 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@factnn @cspades
Fix Torch FSDP2 crash: add force_all_reduce kwarg to base finish_grad…
6ee356a 
…_sync (#4953)

Signed-off-by: factnn <166481866+factnn@users.noreply.github.com>
Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>
@wujingyue
Separate mFSDP v2 unit tests (#5640)
72a78d6 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@github-actions
chore: rotate oncall schedule
465264b
@ZhiyuLi-Nvidia @claude
test(determinism): add determinism tests (#5041)
3ab71ee 
Signed-off-by: Zhiyu Li <zhiyul@NVIDIA.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dimapihtar
deprecate common strategy (#5160)
40b1fd3 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@ko3n1g @claude
ci: revert unify legacy scope names (#5316) (#5709)
509efe6 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Phlip79
Normalize CRLF in Claude fix commands (#5712)
65d6c23 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@shurkat-nvidia @deepakn94
Remove some barriers in save_checkpoint_and_time (#5557)
2a21b8e 
Signed-off-by: Skand Hurkat <shurkat@nvidia.com>
Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>
@wujingyue
Add FSDP NVTX annotations (#5704)
91fcdfe 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@YangFei1990
NCCL EP support (#5129)
2d7060f 
Signed-off-by: YangFei1990 <feiw@nvidia.com>
@balasaajay
Update base image to nvcr.io/nvidia/pytorch:26.06-py3 (#5632)
64bee49 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@lauradang @claude
Refactor RL rollout pipeline (#5491)
027fa4a 
Signed-off-by: Laura Dang <laurad@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@HollowMan6
[2/2] Wiring cuDNN fused DSA kernels support with THD, CP and IndexSh…
d1cab16 
…are (GLM5.2) (#5099)

Signed-off-by: Hollow Man <hollowman@opensuse.org>
@sidsingh-nvidia
Triton kernels - avoid recompilation and autotuning in prod (#5608)
f29c747 
Signed-off-by: Siddharth Singh <sidsingh@nvidia.com>
@yqwangustc @claude
Add NeMo Transformer audio encoder model (#5565)
cf2f07d 
Signed-off-by: Yongqiang Wang <yongqiang.seagull@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dimapihtar
remove deprecated modules from core/dist_checkpointing (#5134)
a2496aa 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@fallintoplace
fix(fsdp): import os in safe_get_rank fallback (#4959)
328a77f 
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
@wujingyue
Add forward all-gather overlap (#5513)
ce8865c 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@deepakn94 @claude
Fix seq_load_balancing loss with inter-document masking and MBS > 1 (#…
e86c262 
…5696)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@guihong-nv
Avoid X11 master port default (#5299)
779c5b7 
Signed-off-by: guihong-nv <guihongl@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.