Combine different release pipelines by the use of reusable workflows #6277
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6277 +/- ##
==========================================
+ Coverage 56.95% 57.19% +0.24%
==========================================
Files 506 507 +1
Lines 30467 31257 +790
Branches 4592 4663 +71
==========================================
+ Hits 17353 17879 +526
- Misses 12285 12531 +246
- Partials 829 847 +18 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
justinchuby
reviewed
Aug 22, 2024
justinchuby
approved these changes
Aug 22, 2024
justinchuby
requested changes
Aug 22, 2024
There was a problem hiding this comment.
@andife For the create release workflow, is it possible to skip the publish step in pull requests? Should we only allow it when the trigger is schedule?
justinchuby
reviewed
Aug 22, 2024
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com> Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Both variants would be ok for me. I have added conditions in this regard. Could they be simplified to really only execute it on schedule? On the other hand, you might still have the download step separate? I at schedule we are creating a preview build. I am convinced that it also makes sense to be able to trigger a preview build manually and that the packages are then also pushed to https://pypi.org/project/onnx-weekly/ with the current date. But this I will/would not include into this PR |
Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
justinchuby
approved these changes
Aug 23, 2024
|
Note for clarification: Trusted Publishing does not work for forks (as wanted) |
justinchuby
reviewed
Aug 23, 2024
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com> Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 26, 2024
### Description After the release-pipeline upgrade (#6277) the publishing the weekly does not work any more as the secrets are not available at the resuable workflows ( https://github.com/onnx/onnx/actions/runs/10551283959/job/29228497156 ) Link for the option I used for the secrets: https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows ### Motivation and Context In my opinion, we no longer need these secrets at the end. In the future, the publishing of the Onnx weekly builds will be solved via “trusted publishing” outside the individual OS release pipelines. This is already included in create_release.yml. Currently, the previous weeklys are published at pypi.org, in the test via trusted publishing at test.pypi.org After a successful schedule test (next Monday), we could adjust the publishing. a) e.g. throw out the upload steps with twine and adjust the target of separate steps... b) or leave it as it is for now... Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
linshokaku
pushed a commit
to linshokaku/onnx
that referenced
this pull request
Oct 2, 2024
…nnx#6277) ### Description <!-- - Describe your changes. --> This pull request should be the start of a pipeline (current status of the considerations under onnx#6246) that automatically generates a whls after tagging, signs it and publishes it directly to Pypi to improve supply chain security. As this is all too much at once, the changes will be introduced and tested gradually. Changes related to this PR: Reusable Workflows are used to merge the individual OS release builds together (we need one process as we want to have all artifacts in one pipeline, so there is not need to download them manuelly). The workflows are currently not really "reusable", as we more or less use pypi creditionals in the workflow, among other things. I reused our testpypi weekly repo as I didn't want to change our behavoir for pypi and testpypi. Maybe it would make sense to adapt the process right away so that you can trigger our "Weekly" manually via the Github interface for testing and don't have to wait a week ;-) #### Key points for the review: * How do we want to configure concurrency * What concurrency-groups do we need? * Where do we need "cancel-in-progress"? #### Not part of the pull request: * Extract publish whl to pypi/testpyp * Introduce trusted publishing * ... ### Motivation and Context * see onnx#6246 --------- Signed-off-by: Andreas Fehlner <fehlner@arcor.de> Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com> Signed-off-by: Linsho Kaku <linsho@preferred.jp>
linshokaku
pushed a commit
to linshokaku/onnx
that referenced
this pull request
Oct 2, 2024
### Description After the release-pipeline upgrade (onnx#6277) the publishing the weekly does not work any more as the secrets are not available at the resuable workflows ( https://github.com/onnx/onnx/actions/runs/10551283959/job/29228497156 ) Link for the option I used for the secrets: https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows ### Motivation and Context In my opinion, we no longer need these secrets at the end. In the future, the publishing of the Onnx weekly builds will be solved via “trusted publishing” outside the individual OS release pipelines. This is already included in create_release.yml. Currently, the previous weeklys are published at pypi.org, in the test via trusted publishing at test.pypi.org After a successful schedule test (next Monday), we could adjust the publishing. a) e.g. throw out the upload steps with twine and adjust the target of separate steps... b) or leave it as it is for now... Signed-off-by: Andreas Fehlner <fehlner@arcor.de> Signed-off-by: Linsho Kaku <linsho@preferred.jp>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This pull request should be the start of a pipeline (current status of the considerations under #6246) that automatically generates a whls after tagging, signs it and publishes it directly to Pypi to improve supply chain security. As this is all too much at once, the changes will be introduced and tested gradually.
Changes related to this PR:
Reusable Workflows are used to merge the individual OS release builds together (we need one process as we want to have all artifacts in one pipeline, so there is not need to download them manuelly). The workflows are currently not really "reusable", as we more or less use pypi creditionals in the workflow, among other things. I reused our testpypi weekly repo as I didn't want to change our behavoir for pypi and testpypi.
Maybe it would make sense to adapt the process right away so that you can trigger our "Weekly" manually via the Github interface for testing and don't have to wait a week ;-)
Key points for the review:
Not part of the pull request:
Motivation and Context