🏊 [SFT] Compatibility with padding free and iterable dataset #3053
Conversation
| @@ -386,8 +386,9 @@ def _func(example): | |||
| dataset = dataset.map(_func, batched=batched, **map_kwargs) | |||
|
|
|||
| # If the dataset is prompt-completion, convert it to language modeling type | |||
| if "prompt" in dataset.column_names and "completion" in dataset.column_names: | |||
| key = "messages" if is_conversational(dataset[0]) else "text" | |||
| first_example = next(iter(dataset)) | |||
There was a problem hiding this comment.
Iterable dataset doesn't have column_names. So we get the keys of the first example
There was a problem hiding this comment.
so here instead of the conditioned loop, it just explicitly fetches the first example of the dataset? Nice!
There was a problem hiding this comment.
You still have the condition, but when the dataset is iterable, dataset.column_name is None so you need find another way to get it
|
@loricxy0707 can you confirm that this fixes your issue? |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…m/huggingface/trl into fix-sft-iterable-and-padding-free
…m/huggingface/trl into fix-sft-iterable-and-padding-free
| result = torch.all(encoded_instance["labels"] == -100) | ||
| self.assertTrue(result, "Not all values in the tensor are -100.") | ||
|
|
||
| def test_padding_free(self): | ||
| tokenizer = AutoTokenizer.from_pretrained("trl-internal-testing/tiny-Qwen2ForCausalLM-2.5") | ||
| if tokenizer.pad_token_id is None: | ||
| tokenizer.pad_token = tokenizer.eos_token |
There was a problem hiding this comment.
tokenizer.pad_token_id = tokenizer.eos_token_id does that
| inst1 = "### System: You are a helpful assistant.\n\n### User: How much is 2+2?\n\n### Assistant: 2+2 equals 4" | ||
| inst2 = "### System: You are a honest and helpful assistant.\n\n### User: What is the answer of 22x22?\n\n### Assistant: 22x22 equals 484" | ||
|
|
||
| response_template = "\n\n### Assistant:" |
There was a problem hiding this comment.
the tokenizer has a token for ?\n\n, so the template won't be found. This change fixes it.
| @@ -147,7 +147,7 @@ jobs: | |||
| run: | | |||
| python -m pip install --upgrade pip | |||
| python -m pip install accelerate==0.34.0 | |||
| python -m pip install datasets==2.21.0 | |||
There was a problem hiding this comment.
This test would fail with 2.21. So we bump to the next version: 3.0.0:
FAILED tests/test_sft_trainer.py::SFTTrainerTester2::test_train_with_iterable_dataset - ValueError: Batch does not contain any data (`None`). At the end of all iterable data available before expected stop iteration.
There was a problem hiding this comment.
Locally yes, let's wait and see if the CI passes now
| @@ -147,7 +147,7 @@ jobs: | |||
| run: | | |||
| python -m pip install --upgrade pip | |||
| python -m pip install accelerate==0.34.0 | |||
| python -m pip install datasets==2.21.0 | |||
There was a problem hiding this comment.
This test would fail with 2.21. So we bump to the next version: 3.0.0:
FAILED tests/test_sft_trainer.py::SFTTrainerTester2::test_train_with_iterable_dataset - ValueError: Batch does not contain any data (`None`). At the end of all iterable data available before expected stop iteration.
| @@ -147,7 +147,7 @@ jobs: | |||
| run: | | |||
| python -m pip install --upgrade pip | |||
| python -m pip install accelerate==0.34.0 | |||
| python -m pip install datasets==2.21.0 | |||
| @@ -386,8 +386,9 @@ def _func(example): | |||
| dataset = dataset.map(_func, batched=batched, **map_kwargs) | |||
|
|
|||
| # If the dataset is prompt-completion, convert it to language modeling type | |||
| if "prompt" in dataset.column_names and "completion" in dataset.column_names: | |||
| key = "messages" if is_conversational(dataset[0]) else "text" | |||
| first_example = next(iter(dataset)) | |||
There was a problem hiding this comment.
so here instead of the conditioned loop, it just explicitly fetches the first example of the dataset? Nice!
…face#3053) * Compatibilitywith padding free and iterable dataset * Fix collator test * add a test for streaming * some cleaning * improve and fix tests * tiny revert * bump datasets to 3.0.0
What does this PR do?
Fixes #3030
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.