Motivation

The current router system passes the HTTP client as a parameter through every RouterTrait method, leading to:

• Verbose method signatures that are difficult to maintain
• PDRouter creating a redundant HTTP client that goes unused
• Poor scalability when adding new dependencies (would require changing all method signatures)
• Unnecessary parameter passing overhead throughout the codebase

This PR refactors the router system to use proper dependency injection through an AppContext pattern, making the code cleaner, more maintainable, and easier to extend.

Modifications

1. Introduced AppContext Pattern

• Created AppContext struct to serve as a dependency container holding the HTTP client, router configuration, and concurrency limiter
• Modified AppState to contain both the router and the shared context
• Provides a single place to manage all cross-cutting concerns

2. Updated RouterFactory

• Changed create_router signature to accept &Arc<AppContext> instead of &RouterConfig
• Factory now extracts dependencies from context and passes them to router constructors
• Maintains clean separation between dependency management and router creation

3. Improved Router Constructors

• Router: Added client: reqwest::Client parameter and field to store the injected client
• PDRouter: Added client: reqwest::Client parameter, renamed internal http_client to client for consistency
• Removed redundant client creation in PDRouter (saving ~4 lines of code and a potential failure point)

4. Simplified RouterTrait Interface

• Removed client: &Client parameter from all 10 RouterTrait methods
• Methods now have cleaner signatures focused on their actual responsibilities
• Improved API usability for router implementations

5. Updated Implementations

• Modified all router methods to use self.client instead of receiving it as a parameter
• Updated background load monitoring tasks to use the injected client
• Ensured consistent client usage across all router types

6. Cleaned Up Server Route Handlers

• Updated handlers to access client through state.context.client
• Removed client parameter passing from all router method calls
• Resulted in cleaner, more readable route handler code

7. Fixed All Tests

• Updated test helpers to create AppContext for tests
• Modified all test files to use the new factory signature
• All tests pass successfully with the new architecture

Accuracy Test

This PR only affects the router infrastructure and dependency injection pattern. It does not modify any model-side code, kernels, or model architecture. No accuracy testing is required.

Benchmark & Profiling

This refactoring is performance-neutral as it:

• Uses the same HTTP client configuration with identical connection pooling settings
• Maintains the same request routing logic and load balancing algorithms
• Reduces parameter passing overhead (minor performance improvement)
• Does not introduce any new allocations or computational overhead

The change is purely architectural and should not impact throughput or latency.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
  • All existing tests updated and passing
  • Test coverage maintained at the same level
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
  • Updated all relevant code comments
  • Method signatures in trait documentation updated
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
  • Not applicable - architectural refactoring only
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.