Skip to content

BUG FIX: AmazonBedrockServerModel crashes when thinking mode is enabled #1632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

benoriol
Copy link
Contributor

Problem
AmazonBedrockServerModel is trying to access the wrong response field when Claude thinking mode is enabled. I believe this might be the case with other bedrock models when thinking mode is enabled too.

The problem is the AmazonBedrockServerModel assumes the response is always the 'text' field of the first element [0] of response["output"]["message"]["content"] but that is not the case when thinking mode is enabled.

Steps to reproduce

  1. pass {"thinking": {"type": "enabled", "budget_tokens": 4096}} as additionalModelRequestFields in the constructor of AmazonBedrockServerModel.
  2. Run the agent.
  3. When internally trying to access the text answer from API response, it crashes.

Reproduce

from boto3 import client
from smolagents import CodeAgent, AmazonBedrockServerModel


prompt = "Find positive integer values a, b, c, d such that a + b*9 + c*9**2 + d*9**3 = 1000. a,b,c,d != 0"

# Default model
model_id = "us.anthropic.claude-3-7-sonnet-20250219-v1:0"
boto_client = client(service_name='bedrock-runtime')
model = AmazonBedrockServerModel(model_id=model_id, client=boto_client, additionalModelRequestFields = {"thinking": {"type": "enabled", "budget_tokens": 4096}})
agent = CodeAgent(
                tools=[],
                model=model, 
                stream_outputs=False,
            )
result = agent.run(prompt)
----- Step 1 ------
Error in generating model output:
'text'
[Step 1: Duration 50.57 seconds]
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/code/agents/smolagents/src/smolagents/agents.py:1637, in CodeAgent._step_stream(self, memory_step)
   1636 else:
-> 1637     chat_message: ChatMessage = self.model.generate(
   1638         input_messages,
   1639         stop_sequences=stop_sequences,
   1640         **additional_args,
   1641     )
   1642     memory_step.model_output_message = chat_message

File ~/code/agents/smolagents/src/smolagents/models.py:1902, in AmazonBedrockServerModel.generate(self, messages, stop_sequences, response_format, tools_to_call_from, **kwargs)
   1901 # Get first message
-> 1902 response["output"]["message"]["content"] = response["output"]["message"]["content"][0]["text"]
   1904 self._last_input_token_count = response["usage"]["inputTokens"]

KeyError: 'text'

The above exception was the direct cause of the following exception:

AgentGenerationError                      Traceback (most recent call last)
Cell In[1], line 19
     10 model = AmazonBedrockServerModel(model_id=model_id, client=boto_client, additionalModelRequestFields = {"thinking": {"type": "enabled", "budget_tokens": 4096}})
     11 agent = CodeAgent(
     12                 tools=[],
     13                 #additional_authorized_imports=["fractions", "sympy", "numpy", "functools", "numpy.random", "*"],
   (...)     17                 #logger=logger
     18             )
---> 19 result = agent.run(prompt)

File ~/code/agents/smolagents/src/smolagents/agents.py:468, in MultiStepAgent.run(self, task, stream, reset, images, additional_args, max_steps)
    465 run_start_time = time.time()
    466 # Outputs are returned only at the end. We only look at the last step.
--> 468 steps = list(self._run_stream(task=self.task, max_steps=max_steps, images=images))
    469 assert isinstance(steps[-1], FinalAnswerStep)
    470 output = steps[-1].output

File ~/code/agents/smolagents/src/smolagents/agents.py:562, in MultiStepAgent._run_stream(self, task, max_steps, images)
    558             action_step.is_final_answer = True
    560 except AgentGenerationError as e:
    561     # Agent generation errors are not caused by a Model error but an implementation error: so we should raise them and exit.
--> 562     raise e
    563 except AgentError as e:
    564     # Other AgentError types are caused by the Model, so we should log them and iterate.
    565     action_step.error = e

File ~/code/agents/smolagents/src/smolagents/agents.py:544, in MultiStepAgent._run_stream(self, task, max_steps, images)
    542 self.logger.log_rule(f"Step {self.step_number}", level=LogLevel.INFO)
    543 try:
--> 544     for output in self._step_stream(action_step):
    545         # Yield all
    546         yield output
    548         if isinstance(output, ActionOutput) and output.is_final_answer:

File ~/code/agents/smolagents/src/smolagents/agents.py:1659, in CodeAgent._step_stream(self, memory_step)
   1657     memory_step.model_output = output_text
   1658 except Exception as e:
-> 1659     raise AgentGenerationError(f"Error in generating model output:\n{e}", self.logger) from e
   1661 ### Parse output ###
   1662 try:

AgentGenerationError: Error in generating model output:
'text'

Expected behavior
Dont crash and provide the code and solution

Environment:
Commit: 84e5e8c

pip show smolagents
Name: smolagents
Version: 1.20.0.dev0
Summary: 🤗 smolagents: a barebones library for agents. Agents write python code to call tools or orchestrate other agents.
Home-page:
Author:
Author-email: Aymeric Roucher <aymeric@hf.co>
...

Other context is irrelevant


Checklist

  • I have searched the existing issues and have not found a similar bug report.
  • I have provided a minimal, reproducible example.
  • I have provided the full traceback of the error.
  • I have provided my environment details.
  • I am willing to work on this issue and submit a pull request.

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moltes gràcies, @benoriol 😉

Thanks a lot for reporting and addressing the bug with AmazonBedrockServerModel when Claude thinking mode is enabled: very helpful! I really appreciate the clear and thorough PR description: it makes understanding and reviewing the fix much easier!

One question though: instead of introducing a loop to iterate through all messages, did you consider simply accessing the last message with [-1] rather than [0]? That might be a more concise and efficient way to get the final answer in this case, assuming the format always places the completion last.

Curious to hear your thoughts!

@benoriol
Copy link
Contributor Author

I was iterating the list to verify that "reasoningContent" is a key in the non-last elements and hence the output format I am assuming is correct and there are no silent bugs or unexpected behavior. This approach could also be more easily adapted to access the reasoning trace, if necessary.

But you are right, just accessing the last element would be expected to have the same effect as the current code.

Gràcies a tu :)

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense: thanks for the clarification!

For now, I'd suggest we keep things simple and just access the last message with [-1], since it should reliably contain the final output and avoids extra logic. If there's a concrete use case where users want to access or inspect the reasoning trace explicitly, we can always extend the implementation to iterate through the full message list at that point.

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for spotting this and proposing a fix! 🤗

@albertvillanova albertvillanova merged commit d651a81 into huggingface:main Jul 30, 2025
3 checks passed
@benoriol
Copy link
Contributor Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants