Fix structured outputs #1642
Merged
Fix structured outputs #1642
+25
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
Fixes a bug where CodeAgent would append closing code block tags to structured output JSON, making it unparseable. The issue occurred when use_structured_outputs_internally was enabled but the agent still added code block delimiters to the JSON response.
- Conditionally append closing code block tags only when structured outputs are disabled
- Add test coverage for the structured outputs functionality to prevent regression
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/smolagents/agents.py | Wraps code block tag appending logic with structured outputs check |
| tests/test_agents.py | Adds test to verify structured outputs don't get corrupted with code block tags |
Comments suppressed due to low confidence (2)
tests/test_agents.py:2133
- The test only verifies that the tool call arguments match the expected code, but doesn't verify that the JSON parsing works correctly. Consider adding an assertion that validates the JSON structure remains valid (e.g.,
json.loads(model.generate.return_value.content)succeeds).
tool_call: ToolCall = next(agent._step_stream(ActionStep(step_number=1, timing="mock_timing", model_output="")))
tests/test_agents.py:2133
- The test doesn't verify the core bug being fixed - that closing code block tags are not appended to structured outputs. Consider adding an assertion that checks the model output message content remains unchanged (i.e., still valid JSON without appended tags).
tool_call: ToolCall = next(agent._step_stream(ActionStep(step_number=1, timing="mock_timing", model_output="")))
5776f95 to
67d528b
Compare
|
Ran ruff and force-pushed changes to keep clean commit history |
aymeric-roucher
approved these changes
Aug 5, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, CodeAgent will append the closing code block tag to the model output if it's not present, even when
self.use_structured_outputs_internallyis set totrue. This means that the structured outputs outputs (a JSON object) can't then be parsed as JSON. For example, you end up with something like:'{"observation": " ... ", "code":" ... "}</code>'which isn't valid JSON.
Fixed by only appending the the closing code block if
self.use_structured_outputs_internallyis set tofalse.