Motivation

#2876 enforces a strict check that disallows cases where the sum of the number of input tokens and max_new_tokens exceeds the context length. However, as @merrymercy pointed out and several reports ([1] [2]) indicate, this check may be too restrictive for users who want to set a very large fixed max_new_tokens value when the input length is not easy to determine or not important.

Modifications

This PR introduces an optional auto-truncation feature by integrating the existing allow_auto_truncate option from server_args into TokenizerManager.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.