Points: 2-3 days

Description: Unify Logprobs handling and UsageInfo in v1/chat/completions and v1/completions. Reduce the repetitive code, increase code reusability and structure

Deliverables:

• Complete Task 1 and Task 2
• Add UTs

Task 1: Token Logprobs Handling

Current logic in adapter.py#L1327-L1368:

New logic in serving_chat.py:

Non streaming logprobs first calls _process_response_logprobs, then calls _process_logprobs_tokens.

Unify Logprobs

• Logic is fine, but it has quite some convoluated with streaming logprobs, and completions endpoint.
  Inconsistent Entry Points:
• Chat has 2 different methods (_process_response_logprobs vs _process_streaming_logprobs ) for similar work
• Duplicated Logic: Both chat and completions call to_openai_style_logprobs
• Mixed Responsibilities: Some methods do conversion + processing, others just processing
• Hard to Test: Complex call chains make unit testing difficult

Design

• Approach: Create a unified LogProbsProcessor using factory pattern to eliminate code duplication and inconsistent APIs.

• New File: sglang/python/sglang/srt/entrypoints/openai/logprobs_processor.py

• High Level Design:

  • serving_chat.py: Replace _process_streaming_logprobs and _process_response_logprobs with factory calls, remove _process_logprobs_tokens
  • serving_completions.py: Replace inline to_openai_style_logprobs calls with factory methods
  • utils.py: Deprecate or remove to_openai_style_logprobs function

Task 2: UsageInfo

Current Problem

• Code Duplication: aggregate_token_usage (utils.py) vs _calculate_streaming_usage_base (serving_base.py)
• Different Data Formats: Non-streaming uses response lists, streaming uses token dictionaries
• Similar Logic: Both calculate total tokens with n_choices handling and cache reporting

Design Recommendation

• Approach: Create unified UsageProcessor following same factory pattern as LogProbs.

• New File: sglang/python/sglang/srt/entrypoints/openai/usage_processor.py

• Files to Update:

  • serving_chat.py: Replace aggregate_token_usage calls with factory methods
  • serving_completions.py: Replace aggregate_token_usage calls with factory methods
  • serving_base.py: Replace _calculate_streaming_usage_base with factory calls
  • utils.py: Deprecate aggregate_token_usage function

• Functions to Consolidate:

  • aggregate_token_usage (from utils.py) → UsageProcessor.calculate_response_usage
  • _calculate_streaming_usage_base (from serving_base.py) → UsageProcessor.calculate_streaming_usage