Skip to content

feat: add qwen3 support #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
May 2, 2025
Merged

feat: add qwen3 support #289

merged 6 commits into from
May 2, 2025

Conversation

gshennvm
Copy link
Contributor

@gshennvm gshennvm commented Apr 29, 2025 •
edited
Loading

What does this PR do ?

FSDP1 vs FSDP2
FSDP1 vs FSDP2

Different FSDP2 configs

Different FSDP2 Configs

@gshennvm gshennvm changed the title qwen3 sp feat: add qwen3 support Apr 29, 2025
@gshennvm gshennvm added Run CICD CI:L1 Run doctests, unit tests, and functional tests CI:L2 Run doctests, unit tests, functional tests, and convergence tests and removed Run CICD labels Apr 29, 2025
@gshennvm gshennvm marked this pull request as ready for review April 29, 2025 17:36
parthchadha
parthchadha reviewed Apr 29, 2025
View reviewed changes
parthchadha
parthchadha previously approved these changes Apr 29, 2025
View reviewed changes
@terrykong terrykong added CI:docs Run doctest and removed Run CICD CI:L1 Run doctests, unit tests, and functional tests CI:L2 Run doctests, unit tests, functional tests, and convergence tests labels Apr 29, 2025
@gshennvm gshennvm changed the title feat: add qwen3 support feat:add qwen3 support Apr 30, 2025
@gshennvm gshennvm added Run CICD CI:L0 Run doctests and unit tests CI:L1 Run doctests, unit tests, and functional tests CI:L2 Run doctests, unit tests, functional tests, and convergence tests labels Apr 30, 2025
@gshennvm gshennvm changed the title feat:add qwen3 support feat: add qwen3 support Apr 30, 2025
@gshennvm @terrykong
qwen3 sp
ad7ad4c 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
gshennvm and others added 5 commits May 1, 2025 15:53
@gshennvm @terrykong
add tests
7bb1163 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
@gshennvm @terrykong
disable tied word embeddings when testing qwen3
7ce9b4d 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
@gshennvm @terrykong
update transformers version
bd603f9 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
@gshennvm @terrykong
fix lint
a5413a6 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
@terrykong
chore: bump uv.lock (#296)
9743c45 
Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong terrykong added this pull request to the merge queue May 1, 2025
Merged via the queue into main with commit 1db1d75 May 2, 2025
12 checks passed
@terrykong terrykong deleted the geshen/qwen3 branch May 2, 2025 00:47
YzjiaoNvd pushed a commit to YzjiaoNvd/NeMo-RL that referenced this pull request Jun 10, 2025
@gshennvm @terrykong
feat: add qwen3 support (NVIDIA-NeMo#289)
3204abd 
Signed-off-by: Gerald Shen <geshen@nvidia.com>
Signed-off-by: Terry Kong <terryk@nvidia.com>
Co-authored-by: Terry Kong <terrycurtiskong@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@terrykong terrykong terrykong approved these changes

@parthchadha parthchadha parthchadha left review comments

Assignees
No one assigned
Labels
CI:docs Run doctest CI:L0 Run doctests and unit tests CI:L1 Run doctests, unit tests, and functional tests CI:L2 Run doctests, unit tests, functional tests, and convergence tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gshennvm @terrykong @parthchadha