Skip to content

🍟 [SFT] Handles the dataset if it has been preprocessed #2863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Feb 18, 2025
Merged

🍟 [SFT] Handles the dataset if it has been preprocessed #2863

merged 16 commits into from
Feb 18, 2025
+58 −6

Conversation

BenasdTW
Copy link
Contributor

What does this PR do?

Fixes #2861

Adding back the code in SFTTrainer that properly handles the dataset if it has been preprocessed.
This part of the code was removed in #2405, but it should not have been removed.

        # If the dataset is already preprocessed (tokenized), return as-is. Only works if dataset is
        # a datasets.Dataset or datasets.IterableDataset -- not for torch Dataset
        column_names = (
            dataset.column_names if isinstance(dataset, (Dataset, IterableDataset)) else None
        )
        if column_names and "input_ids" in column_names:
            if formatting_func is not None:
                warnings.warn(
                    "You passed a dataset that is already processed (contains an `input_ids` field) together with a "
                    "valid formatting function. Therefore `formatting_func` will be ignored. Either remove the "
                    "`formatting_func` or pass a dataset that is not already processed.",
                    UserWarning,
                )
            if not packing:
                return dataset

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@BenasdTW BenasdTW mentioned this pull request Feb 14, 2025
Closed
5 tasks
qgallouedec
qgallouedec reviewed Feb 14, 2025
View reviewed changes
qgallouedec
qgallouedec reviewed Feb 14, 2025
View reviewed changes
qgallouedec
qgallouedec reviewed Feb 14, 2025
View reviewed changes
@qgallouedec
Copy link
Member

Thanks for fixing it!
I made a few comments. Can you also add a test?

@BenasdTW
Copy link
Contributor Author

Added test: test_sft_trainer_directly_with_pretokenized_data

root@813ecfcb235b:/workspaces/LLMTrain/trl# python -m pytest tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_directly_with_pretokenized_data
=================================================== test session starts ====================================================
platform linux -- Python 3.11.10, pytest-8.3.4, pluggy-1.5.0
rootdir: /workspaces/LLMTrain/trl
configfile: pyproject.toml
plugins: hypothesis-6.115.5, rerunfailures-15.0, cov-6.0.0, xdist-3.6.1, anyio-4.8.0
collected 1 item                                                                                                           

tests/test_sft_trainer.py .                                                                                          [100%]

==================================================== 1 passed in 18.36s ====================================================

qgallouedec
qgallouedec reviewed Feb 15, 2025
View reviewed changes
@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

qgallouedec
qgallouedec reviewed Feb 17, 2025
View reviewed changes
qgallouedec
qgallouedec approved these changes Feb 17, 2025
View reviewed changes
Copy link
Member

@qgallouedec qgallouedec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @BenasdTW ! can be merged as soon as the CI is green

qgallouedec
qgallouedec reviewed Feb 17, 2025
View reviewed changes
@qgallouedec qgallouedec changed the title [SFT] Handles the dataset if it has been preprocessed 🍟 [SFT] Handles the dataset if it has been preprocessed Feb 17, 2025
kashif
kashif reviewed Feb 17, 2025
View reviewed changes
@qgallouedec qgallouedec merged commit aafd8cb into huggingface:main Feb 18, 2025
13 checks passed
qgallouedec added a commit that referenced this pull request Feb 18, 2025
@BenasdTW @kashif @qgallouedec
🍟 [SFT] Handles the dataset if it has been preprocessed (#2863)
2ccca1d 
* return dataset if it's preprocessed

* add is_processed flag variable

* add test

* move test_sft_trainer_directly_with_pretokenized_data to Tester2

* Update sft_trainer.py

* no need for padding and truncation

* minor reorganization

* Update trl/trainer/sft_trainer.py

* let the collator pad

* style

* fix tests

---------

Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
@hanyin88
Copy link

Thanks for the kind contribution. Wondering why

def tokenize(ex):
                    tokenized = processing_class(ex[args.dataset_text_field])
                    return {"input_ids": tokenized["input_ids"], "attention_mask": tokenized["attention_mask"]}

dataset = dataset.map(tokenize, **map_kwargs)

would generate new dataset fingerprint now, and not be able to utilize prior cache? All prior steps could utilize the cache with same fingerprint without problem. Many thanks.

yxliu-TAMU pushed a commit to mincheolseong/ECEN743-GRPO-Project-Proposal that referenced this pull request Apr 20, 2025
@BenasdTW @kashif @qgallouedec
🍟 [SFT] Handles the dataset if it has been preprocessed (huggingface#…
d47b4cd 
…2863)

* return dataset if it's preprocessed

* add is_processed flag variable

* add test

* move test_sft_trainer_directly_with_pretokenized_data to Tester2

* Update sft_trainer.py

* no need for padding and truncation

* minor reorganization

* Update trl/trainer/sft_trainer.py

* let the collator pad

* style

* fix tests

---------

Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kashif kashif kashif left review comments

@qgallouedec qgallouedec qgallouedec approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Cannot run SFTTrainer with tokenized data after updating TRL.
5 participants
@BenasdTW @qgallouedec @HuggingFaceDocBuilderDev @hanyin88 @kashif