Skip to content

Conversation

gty111
Copy link
Contributor

@gty111 gty111 commented Jan 10, 2025

Motivation

When serving with large port number

python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct \
--port 65531 --host 0.0.0.0

it will trigger following error

Traceback (most recent call last):
  File "/miniconda3/envs/sglang/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/miniconda3/envs/sglang/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "sglang/python/sglang/launch_server.py", line 14, in <module>
    launch_server(server_args)
  File "sglang/python/sglang/srt/server.py", line 534, in launch_server
    launch_engine(server_args=server_args)
  File "sglang/python/sglang/srt/server.py", line 433, in launch_engine
    port_args = PortArgs.init_new(server_args)
  File "sglang/python/sglang/srt/server_args.py", line 929, in init_new
    if is_port_available(port):
  File "sglang/python/sglang/srt/utils.py", line 333, in is_port_available
    s.bind(("", port))
OverflowError: bind(): port must be 0-65535.

I check the code and find sglang will use another port for inter-process communications. However, the logic for allocating port number only add random number to the specific port number which will cause overflow of port number.

Modifications

Avoid port number overflow.

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@merrymercy merrymercy merged commit b170646 into sgl-project:main Jan 10, 2025
1 of 14 checks passed
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants