Support InternVL2.5 #4433
Support InternVL2.5 #4433
Conversation
|
@hehesangsj thansk, please fix the conflicts and we can merge this ASAP. |
|
@mickqian @zhaochenyang20 Could you review the code~ |
hey, is this #4456 (comment) provided by your team? Otherwise, please provide the benchmark score at #4456 Also, @yizhang2077 could you help to review? thanks so much! |
| has_flash_attn = False | ||
|
|
||
|
|
||
| class FlashAttention(nn.Module): |
There was a problem hiding this comment.
We provide some attention code in vision.py, could you try to use it instead of writing attention again?
There was a problem hiding this comment.
I think it's a little bit hard because the num_heads of InternViT is 25. Using VisionAttention can easily leads to bugs when deploying using --tp 8
|
@zhaochenyang20 @yizhang2077 #3351 These are duplicated |
I let them decouple and I will credit you! thanks ! @mickqian |
|
@hehesangsj fix the lint and solve the conflicts? |
|
@hehesangsj do you sovle all the reviews above? if so, please ping us and we can continue to review. thanks a lot |
okay, I will solve the reviews ASAP. |
reviews are fixed and latest main branch is merged. check the code plz~ |
|
@yizhang2077 yi and @mickqian could you help to review? |
|
This pull request has been automatically closed due to inactivity. Please feel free to reopen it if needed. |
Motivation
Support InternVL2.5
Modifications
Based on PR #3351.
Support both InternLM2ForCausalLM & Qwen2ForCausalLM as language model.
Fix bugs in multi-nodes deployment
Checklist