Skip to content

PyGithub should provide means to throttle API requests #1989

@EnricoMi

Description

@EnricoMi

Sending requests to the GitHub API too quickly will result in abuse rate limit errors. This has been discussed in several issues here (#1235, #2113, #2005, #1319, #1115, #810). I'd like to propose to wrap the Requester.__requestRaw method in a throttling method that ensures subsequent calls are at least a configured number of (float) seconds apart. This way the user of the client library can rely on calls not being too frequent.

Throttling in user code adds complexity on top of the client library and duplicates code across projects that use this client library. A good example is clockfort/GitHub-Backup#44. Especially when iterating over paginated results, throttling becomes difficult in user code.

I have wrapped the Requester.__requestRaw method with the following code, which allows me to configure seconds between read (GET) and write (all verbs other than GET) requests. All requests will then be deferred by the client library if called too quickly:

https://github.com/EnricoMi/publish-unit-test-result-action/blob/4b6277f1917325e96cadc94a1116031a0ae906bb/python/publish_unit_test_results.py#L99-L122

I am delaying retrys by a urllib Retry instance with a backoff factor identical to the configured delays above, which is out of scope of this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions