Skip to content

[BUG] When model API provider doesn't support tool_choice="required", OpenAIServerModel breaks at requesting response #1391

@duguyue100

Description

@duguyue100

Describe the bug

OpenAIServerModel can use any 3rd party model provider with an OpenAI-compatible endpoint.
In the Model._prepare_completion_kwargs method, tool_choice is hard coded as required.
However, some model providers doesn't support this choice, it's either 'auto', tool name or 'none'.

The tool_choice can be potentially modified through **kwargs, however, there is no way to pass tool_choice as part of the argument.

I see this as a bug rather than a feature request because this is effectively blocking the use of 3rd party provider's model completely.

For now, what I could do is that I can extend OpenAIServerModel and write a special _prepare_completion_kwargs myself.
I hope that the agent can somehow expose completion keyward arguments and we can manage them more precisely.

Code to reproduce the error
The simplest code snippet that produces your bug.

I'm using Novita as the API provider, they do not support 'required' option for 'tool_choice'

Error logs (if any)
Provide error logs if there are any.

Part of the error log is here:

, \'ctx\': {\'error\': ValueError(\'`tool_choice` must either be a named tool, "auto", or "none".\')}}]', 'type': 'BadRequestError'}

Packages version:
Run pip freeze | grep smolagents and paste it here.

Name: smolagents
Version: 1.16.1

Additional context
Add any other context about the problem here.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions