i think we may need to generalize that bos check. i found another place where it fails:

in the deepscaler base when you try to eval, it’ll add the double-bos, but the chat-template doesn’t have any obvious indicators b/c it’s a more complicated jinja template

https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/blob/main/tokenizer_config.json#L34

i think we may need some kind of apply_safe_chat_template() or something that we re-use throughout the repo that remembers the decision to handle this bos-token. do you think you can look into a general fix?