For code here: https://github.com/huggingface/trl/blob/main/trl/trainer/utils.py#L538C4-L550 Does it always padding to right?

I get error when run dpo for mixtral 7b.

I am still seeing this error after set tokenizer.padding_side = 'left'
This is my demo notebook: https://colab.research.google.com/drive/1sVqbYEOqjJYl7CzNzXzviEBB6A984cMq?usp=sharing

Tokenizer already set with left padding

Train

Still have: ValueError: You are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to call tokenizer.padding_side = 'left' before tokenizing the input.

Seems https://github.com/huggingface/trl/blob/main/trl/trainer/utils.py#L538C4-L550 or other part of code might lead to bug?

transformers 4.36.2
trl 0.7.7
peft 0.6.0