Skip to content

Conversation

keetrap
Copy link
Contributor

@keetrap keetrap commented Apr 7, 2025

Fixes #1143

@aymeric-roucher
Copy link
Collaborator

aymeric-roucher commented Apr 8, 2025

Nice, thank you! Could you also test the ToolCallingAgent's replay function to make sure that the issue #1143 does not reappear?

@keetrap
Copy link
Contributor Author

keetrap commented Apr 8, 2025

I tried to add the test like shown below

  agent = ToolCallingAgent(tools=[PythonInterpreterTool()], model=FakeToolCallModel(), verbosity_level=0)
  agent.run("What is 2 multiplied by 3.6452?")
  with agent.logger.console.capture() as capture:
      agent.replay()
  
  output = capture.get().replace("\n", "")
  assert "Called Tool" in output

but captured output includes formatting codes.

assert 'Called Tool' in "\x1b[2;36m[16:57:38]\x1b[0m\x1b[2;36m \x1b[0mReplaying the agent's steps:                                                                                   
                                                      \x1b]8;id=669843;file://C:\\Users\\parte\\Desktop\\Open_Source\\smolagents\\src\\smolagents\\memory.py\x1b\\\x1b[2mmemory.py\x1b[0m\x1b]8;;\x1b\\\x1b[2m:\x1b[0m\x1b]8;id=627247;file://C:\\Users\\parte\\Desktop\\Open_Source\\smolagents\\src\\smolagents\\memory.py#214\x1b\\\x1b[2m214\x1b[0m\x1b]8;;\x1b\\\x1b[38;2;212;183;2m╭─\x1b[0m\x1b[38;2;212;183;2m────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;212;183;2m \x1b[0m\x1b[1;38;2;212;183;2mNew run\x1b[0m\x1b[38;2;212;183;2m \x1b[0m\x1b[38;2;212;183;2m────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;212;183;2m─╮\x1b[0m\x1b[38;2;212;183;2m│\x1b[0m                                                                                                                                                                                    
       \x1b[38;2;212;183;2m│\x1b[0m\x1b[38;2;212;183;2m│\x1b[0m \x1b[1mWhat is 2 multiplied by 3...;2;230;237;243;48;2;13;17;23m'2*3.6452'}\x1b[0m\x1b[48;2;13;17;23m                        
                                                                                               \x1b[0m\x1b[1;3mAgent output:\x1b[0m \x1b[38;2;212;183;2m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mCalled\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mTool:\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m'final_answer'\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mwith\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23marguments:\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m{'answer':\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m'7.2904'}\x1b[0m\x1b[48;2;13;17;23m                                                                                                                             \x1b[0m"

@aymeric-roucher
Copy link
Collaborator

@keetrap I see string Called\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mTool: in output: maybe assert 'Called' in output and 'Tool:' in output?

@@ -160,7 +160,7 @@ def test_streaming_agent_image_output(self):
)
)

self.assertEqual(len(outputs), 5)
self.assertEqual(len(outputs), 6)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have one extra ChatMessage in outputs here.
In this case it is ChatMessage(content="Called Tool: 'final_answer' with arguments: {'answer': 'image'}", role='assistant', metadata={}, options=[])
``

@sysradium
Copy link
Contributor

sysradium commented Apr 8, 2025

You can either remove ansi sequences before asserting, something like:

def remove_ansi(text):
    ansi_escape = re.compile(r'\x1B\[[0-?]*[ -/]*[@-~]')
    return ansi_escape.sub('', text)

or just disable them altogether by changing AgentLogger:

logger = AgentLogger(2)
logger.console = Console(force_terminal=False)
# inject or monkeypatch the logger

Actually I have no idea why Console isn't DI-able: https://github.com/huggingface/smolagents/blob/main/src/smolagents/monitoring.py#L89
Maybe the time has come to change that.

@aymeric-roucher
Copy link
Collaborator

@sysradium agree that being able to pass a console arg upon logger initialization would be better! If you have some time, it'd be great to make a PR for this!

Copy link
Collaborator

@aymeric-roucher aymeric-roucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @keetrap !

@aymeric-roucher aymeric-roucher merged commit 3a25900 into huggingface:main Apr 9, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] agent.replay errors
3 participants