📚 The doc issue

On page https://docs.vllm.ai/en/latest/serving/engine_args.html#engine-args

Regarding the flag --override-pooler-config. The documentation provides the following example:

Override or set the pooling method for pooling models. e.g. {“pooling_type”: “mean”, “normalize”: false}.’

However this example does not work if copy-pasted into a UTF-8 aware text editor as it is not a valid JSON document. (The quotation marks are not ascii quotation marks, they are left-quote and right-quote.) This is an insidious error as it is nearly invisible to the naked eye.

In addition to --override-pooler-config, this issue affects --override-neuron-config, --rope-scaling, and --mm-processor-kwargs.

Suggest a potential alternative/fix

Change

Override or set the pooling method for pooling models. e.g. {“pooling_type”: “mean”, “normalize”: false}.’

to

Override or set the pooling method for pooling models. e.g. {"pooling_type": "mean", "normalize": false}.’

(Replace non-ascii quotes with ascii quotes and surround with a code block to ensure verbatim inclusion.)

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.