Skip to content

Conversation

qeternity
Copy link
Contributor

Motivation

Xgrammar compiler now supports regex compilation directly and the latest contains fixes (e.g. lazy groups) which bring it closer to parity with Outlines.

Modifications

Upgrade deps and migrate to regex compiler.

Checklist

@zhyncs
Copy link
Member

zhyncs commented Feb 16, 2025

There are some issues with 0.1.13, please wait for @Ubospica to release a new version.

@zhyncs zhyncs marked this pull request as draft February 16, 2025 15:06
@qeternity
Copy link
Contributor Author

@zhyncs understood, thanks for the review and info.

I presume the issues you reference are build related? Installing from wheel passed our internal smoke tests fwiw.

@Ubospica
Copy link
Collaborator

Ubospica commented Feb 17, 2025

@qeternity Yes the CI fails because xgrammar's new version contains a kernel not compatible with CUDA<12.4. It should work properly for CUDA>=12.4. We have fixed this issue (mlc-ai/xgrammar#206) and will release a new version later.

@qeternity
Copy link
Contributor Author

@Ubospica gotcha. We are running 12.4 so I guess that explains things :)

@zhyncs
Copy link
Member

zhyncs commented Feb 18, 2025

Ah Currently SGLang is using CUDA 12.1 for the CIs :)
ref https://github.com/sgl-project/sglang/blob/main/docs/developer/setup_github_runner.md#step-1-start-a-docker-container
Maybe we should upgrade CUDA 12.4 @merrymercy @Ying1123

@WhoisZihan
Copy link

xgrammar just releases 0.1.14, which supports lower version of glibc mlc-ai/xgrammar#113 . Is it possible to support latest version xgrammar to ease the installation on older os?

@qeternity qeternity changed the title xgrammar 0.1.13 xgrammar 0.1.14 Feb 27, 2025
@qeternity
Copy link
Contributor Author

I have updated this PR to 0.1.14

@qeternity qeternity marked this pull request as ready for review February 27, 2025 16:04
@zhyncs
Copy link
Member

zhyncs commented Feb 27, 2025

Why did this PR skip the test? @shuaills @zhaochenyang20

Note: python/sglang/srt/constrained/xgrammar_backend.py has changed.

@zhaochenyang20
Copy link
Collaborator

@zhyncs I think this should be fine

@zhaochenyang20 zhaochenyang20 merged commit d281587 into sgl-project:main Feb 27, 2025
15 checks passed
@zhyncs
Copy link
Member

zhyncs commented Feb 27, 2025

@zhyncs I think this should be fine

@zhaochenyang20 I don't think so. When we update python/pyproject.toml or python files, we must trigger the pr test.

@zhaochenyang20
Copy link
Collaborator

Sure. We are fixing this today:

#3566

If failed, we can revert it.

aoshen524 pushed a commit to aoshen524/sglang that referenced this pull request Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants