-
Notifications
You must be signed in to change notification settings - Fork 2k
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Unable to pass additional parameters to tokenizer through the MLXModel module. For example, QWEN3 has enable_thinking flag which you can pass as a boolean value. However MLXModel does not have a provision for that to be passed.
Above should be allowed. Other examples are Cohere RAG mode in the chat template. There is currently no way to pass those parameters that I was able to find.
Example below is where the value is passed but it is ignored as it is a tokenizer parameter rather than a model one.
from smolagents import MLXModel
model = MLXModel(model_id="mlx-community/Qwen3-8B-4bit", max_tokens=128)
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "Explain quantum mechanics in simple terms."}
]
}
]
response = model(messages, {"enable_thinking":False})
print(response)
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request