-
Notifications
You must be signed in to change notification settings - Fork 2k
Fix when response.usage
is None
for some OpenRouter models
#1402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix when response.usage
is None
for some OpenRouter models
#1402
Conversation
Similar bug fix here: microsoft/autogen#1599 |
0
for prompt_tokens
when response.usage
is None.response.usage
is None
for OpenAI model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this code lines are not the source of the reported issue: just in the preceding line, this check is already in place:
if event.usage:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please note that we support streaming with our stream_outputs
parameter:
- if True, we call
generate_stream
and call the model withstream=True
- if False, we call
generate
, and the model is not called withstream=True
And according to OpenRouter docs, response.usage
can only be None when passing stream=True
, that is, when stream_outputs
is True, and therefore when we call generate_stream
. We don't call generate
when streaming outputs.
Could you please provide an example raising the error in the reported issue:
Added traceback to the issue :) |
response.usage
is None
for OpenAI model.response.usage
is None
for some OpenRouter models
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Just did a minor refactoring.
The openrouter API has
response.usage
as optional, while the OpenAI API does not.Fixes #1401
Checks.
All commands from the
CONTRIBUTING.md
pass.Tests
689 passed