Skip to content

SFT w/ FSDP2 #197

New issue
New issue
Closed
#206
Closed
SFT w/ FSDP2#197
#206
Assignees
yfw
@yfw

Description

@yfw
yfw
opened on Apr 15, 2025
  • Compare FSDP2 and FSDP1
  • w/ TP > 1
  • sequence parallel
  • activation checkpointing
  • cpu offload
  • target context length: 32k, llama3-8b

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions