Skip to content

Fix when response.usage is None for some OpenRouter models #1402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

sentienthouseplant
Copy link
Contributor

@sentienthouseplant sentienthouseplant commented May 31, 2025

The openrouter API has response.usage as optional, while the OpenAI API does not.

Fixes #1401

Checks.

All commands from the CONTRIBUTING.md pass.

Tests

689 passed

@sentienthouseplant
Copy link
Contributor Author

Similar bug fix here: microsoft/autogen#1599

@sentienthouseplant sentienthouseplant changed the title Default to 0 for prompt_tokens when response.usage is None. Fix when response.usage is None for OpenAI model. May 31, 2025
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code lines are not the source of the reported issue: just in the preceding line, this check is already in place:

if event.usage:

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that we support streaming with our stream_outputs parameter:

  • if True, we call generate_stream and call the model with stream=True
  • if False, we call generate, and the model is not called with stream=True

And according to OpenRouter docs, response.usage can only be None when passing stream=True, that is, when stream_outputs is True, and therefore when we call generate_stream. We don't call generate when streaming outputs.

Could you please provide an example raising the error in the reported issue:

@sentienthouseplant
Copy link
Contributor Author

Added traceback to the issue :)

@albertvillanova albertvillanova changed the title Fix when response.usage is None for OpenAI model. Fix when response.usage is None for some OpenRouter models Jun 25, 2025
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Just did a minor refactoring.

@albertvillanova albertvillanova merged commit fc73322 into huggingface:main Jun 25, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] OpenRouter usage has different schema to OpenAI's usage, causing attribute error.
2 participants