Skip to content

feat: add OpenAI format dataset for SFT #485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jul 2, 2025
Merged

feat: add OpenAI format dataset for SFT #485

merged 3 commits into from
Jul 2, 2025

Conversation

AtsunoriFujita
Copy link
Contributor

@AtsunoriFujita AtsunoriFujita commented Jun 5, 2025 •
edited
Loading

What does this PR do ?

This PR enables using the OpenAI format dataset from a json/jsonl when running SFT.

Issues

List issues that this PR closes (syntax):

Usage

Modify examples/configs/sft.yaml

 data:
  max_input_seq_length: ${policy.max_total_sequence_length}
  dataset_name: "openai_format"
  add_bos: true
  add_eos: true
  add_generation_prompt: false
  train_data_path: "datasets/train.json"
  val_data_path: "datasets/valid.json"
  chat_key: "messages"  # set chat_key like "messages" and "conversations"
  system_key: null  # (optional) set system_key (system_prompt key name) if dataset has
  system_prompt: null  # (optional) set system_prompt, e.g., "You are a helpful assistant." If system_key is not null, it will be prioritized.

Run SFT job

uv run examples/run_sft.py --config examples/configs/sft.yaml

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

@AtsunoriFujita
Add OpenAI format dataset for SFT
236bbc7 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
@terrykong terrykong requested a review from ashors1 June 5, 2025 16:28
ashors1
ashors1 reviewed Jun 17, 2025
View reviewed changes
Copy link
Contributor

@ashors1 ashors1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Could we add some documentation on this class and how it differs from prompt_response_dataset.py? Do you think it would make sense to consider merging this class with PromptResponseDataset?

@AtsunoriFujita
Add class docstrings and unit test
b5343be 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
@AtsunoriFujita
Copy link
Contributor Author

Hi @ashors1, added docstrings and unit tests.

Could we add some documentation on this class and how it differs from prompt_response_dataset.py?

The openai_format deals with a single key (like messages or conversations) and single or multiple turns (user, assistant), similar to HuggingFaceTB/smoltalk.

ashors1
ashors1 previously approved these changes Jun 18, 2025
View reviewed changes
Copy link
Contributor

@ashors1 ashors1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you for the contribution!

@terrykong terrykong changed the title Add OpenAI format dataset for SFT feat: add OpenAI format dataset for SFT Jun 18, 2025
@terrykong
Copy link
Contributor

@AtsunoriFujita could you run pre-commit on your change? It fails our linter job

@AtsunoriFujita
Apply pre-commit
8a966bc 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
@AtsunoriFujita
Copy link
Contributor Author

Hi @terrykong, applied pre-commit.

@terrykong
Copy link
Contributor

@AtsunoriFujita do you mind putting an example run command in the description so that users finding this PR can learn how to use this?

@AtsunoriFujita
Copy link
Contributor Author

@terrykong, thank you. I added it.

@parthchadha parthchadha added this pull request to the merge queue Jul 2, 2025
Merged via the queue into NVIDIA-NeMo:main with commit d3c58f0 Jul 2, 2025
14 of 15 checks passed
xxman-google pushed a commit to xxman-google/NeMo-RL that referenced this pull request Jul 2, 2025
@AtsunoriFujita @xxman-google
feat: add OpenAI format dataset for SFT (NVIDIA-NeMo#485)
ddac07c 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
@AtsunoriFujita AtsunoriFujita deleted the afujita/add_openai_format_sft branch July 3, 2025 00:10
therealnaveenkamal pushed a commit to therealnaveenkamal/RL that referenced this pull request Jul 7, 2025
@AtsunoriFujita @therealnaveenkamal
feat: add OpenAI format dataset for SFT (NVIDIA-NeMo#485)
ac2e4e6 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
YzjiaoNvd pushed a commit to YzjiaoNvd/NeMo-RL that referenced this pull request Jul 14, 2025
@AtsunoriFujita
feat: add OpenAI format dataset for SFT (NVIDIA-NeMo#485)
e810c24 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
KiddoZhu pushed a commit that referenced this pull request Jul 28, 2025
@AtsunoriFujita @KiddoZhu
feat: add OpenAI format dataset for SFT (#485)
100a7f5 
Signed-off-by: Atsunori Fujita <afujita@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@terrykong terrykong terrykong approved these changes

@parthchadha parthchadha parthchadha approved these changes

@ashors1 ashors1 ashors1 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@AtsunoriFujita @terrykong @parthchadha @ashors1