[Feature] integrate flash-attention #4385
Description
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
similar with #4384
Since SGLang now supports page sizes greater than 1 #4356, we should integrate flash-attention https://github.com/Dao-AILab/flash-attention
Related resources
No response