You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been playing with the vLLM (VLLMModel) support and generated a sample.
model=VLLMModel(model_id=model_name)
The support is super nice but it could benefit from some additional flexibility.
Other model clients like HfApiModel and OpenAIServerModel already provide client_kwargs where you can add the params for the client.
It could be interesting to extend this functionality to VLLMModel. For example, it would enable configurations like: