Contents

Native support for verl on Ascend NPU has attracted the attention of some developers. This roadmap shows the progress of native support, welcome to join in the discussion.

Quick Start

document: ascend_quick_start.rst

Plan

Dependencies (Q1 done)

• Native support transformers
• Native support ray
• Native support FSDP worker
  • Review PR: [NPU] feat: Support FSDP worker and vLLM Ascend #332
• Support vLLM-ascend v0.7.3 (Some features have been temporarily circumvented and marked in the Q2 Plan)

Q2 Plan

• vLLM-ascend sleep mode: [Feature]: Support sleep mode in vllm-ascend vllm-project/vllm-ascend#320
• --use_remove_padding
• LLM FSDP GRPO
• LLM FSDP DAPO
• VLM FSDP GRPO
• Support profiling function
• Support megatron/mindspeed worker (for npu, megatron≈mindspeed)
• Performance Optimization

Release Accuracy Comparison Results

Modify the default config as little as possible to keep the accuracy.

Easy of use

flash-attn is not supported on Ascend NPU. So we need to use torch_npu.npu_fusion_attention to replace.

• [Temporary solution] NPU support SDPA: NPU support SDPA huggingface/transformers#35165
• Using flash_attn on NPU @FightingZhen: [Feature] Support using FlashAttention2 on Ascend NPU huggingface/transformers#36696

Long-term Planning

• torch.compile