Skip to content

Add record_ecosystem_meta API support #407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 7, 2025
Merged

Add record_ecosystem_meta API support #407

merged 3 commits into from
May 7, 2025

Conversation

rhyskoedijk
Copy link
Contributor

@rhyskoedijk rhyskoedijk commented Mar 9, 2025 •
edited
Loading

In dependabot/dependabot-core#10905, a new record_ecosystem_meta API was added. This API is not implemented by dependabot-cli.

When performing updates for an ecosystem that invokes the API (e.g. pip), the API server will throw 501 errors until updater retry limit is reached:

cli | 2025/03/09 21:40:17 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:17 [024] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:17 [024] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:22 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:22 [025] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:22 [025] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:27 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:27 [026] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:27 [026] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:31 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:31 [027] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:31 [027] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
updater | 2025/03/09 21:40:31 ERROR <job_update_0_pip_all> Failed to record ecosystem meta after 3 retries

This API is called multiple times per dependency update and is compounded by the 3 retries per call. The volume of these errors causes the update process to take significantly longer than normal.

After this change, the API server returns 200 and the updater doesn't report error:

proxy | 2025/03/09 22:15:29 [128] POST http://host.docker.internal:35303/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 22:15:29 [128] 200 http://host.docker.internal:35303/update_jobs/update_0_pip_all/record_ecosystem_meta

The output scenario file now contains the record_ecosystem_meta data:

image

@rhyskoedijk rhyskoedijk requested a review from a team as a code owner March 9, 2025 22:31
@rhyskoedijk rhyskoedijk mentioned this pull request Mar 9, 2025
Merged
@rhyskoedijk
Copy link
Contributor Author

@kbukum1, @amazimbe, @abdulapopoola tagging for visibility as you were involved with the original change dependabot/dependabot-core#10905.

@rhyskoedijk rhyskoedijk mentioned this pull request Mar 11, 2025
Merged
@robaiken robaiken force-pushed the feature/enable-record-ecosystem-meta branch from c0e79f5 to f6788c3 Compare March 17, 2025 14:17
@jakecoffman jakecoffman force-pushed the feature/enable-record-ecosystem-meta branch from f6788c3 to 8c7b98e Compare May 7, 2025 15:36
jakecoffman
jakecoffman approved these changes May 7, 2025
View reviewed changes
Copy link
Member

@jakecoffman jakecoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@jakecoffman jakecoffman added this pull request to the merge queue May 7, 2025
Merged via the queue into dependabot:main with commit 65e5e09 May 7, 2025
76 checks passed
@rhyskoedijk rhyskoedijk deleted the feature/enable-record-ecosystem-meta branch May 7, 2025 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jakecoffman jakecoffman jakecoffman approved these changes

@JamieMagee JamieMagee JamieMagee approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rhyskoedijk @jakecoffman @JamieMagee