CI: silent merge check #33145
CI: silent merge check #33145
Conversation
Add "Restore" and "Save" caching actions. These actions reduce boilerplate in the main ci.yml configuration file. These actions are implemented so that caches will be saved on `push` only. When a pull request is opened it will cache hit on the caches from the lastest push, or in the case of depends will hit on any matching depends hash, falling back to partial matches. Depends caches are hashed using `$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)` and this hash is passed in as an input to the actions. This means we direct cache hit in cases where depends would not be re-built, otherwise falling back to a partial match. The cirruslabs cache action will fallback transparently to GitHub's cache in the case that the job is not being run on a Cirrus Runner, making these compatible with running on forks (on free GH hardware).
Another action to reduce boilerplate in the main ci.yml file. This action will set up a docker builder compatible with caching build layers to a container registry using the `gha` build driver. It will then configure the docker build cache args.
This workflow will be triggered when the label 'PeriodicMergeCICheck' is added to a PR. This label could be added manually or periodically by Drahtbot. It will check that since the PR was opened that there isn't a silent merge conflict that prevents compilation or tests passing. Co-authored-by: will <will@256k1.dev>
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33145. ReviewsSee the guideline for information on the review process. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
| previous-releases: | ||
| name: 'Previous releases, depends DEBUG' | ||
| if: contains(github.event.label.name, 'PeriodicMergeCICheck') |
There was a problem hiding this comment.
Seems fine, but currently a vector of task names is accepted, see --task in https://github.com/maflcko/DrahtBot/blob/2038d4d541b89bf2601614b5656d7d30d73e17be/rerun_ci/src/main.rs#L7-L23
It would be good to allow the same somehow?
There was a problem hiding this comment.
Can you elaborate a little more on what this would look like? Would this be different labels for different jobs? As in a label that runs the Previous Releases job and a different label that would run a different job?
There was a problem hiding this comment.
I don't know. Not sure if we want to see pull requests spammed with "added and removed" label events (testing-cirrus-runners#19 (comment)).
GitHub doesn't make this easy and I wonder if we may want to just spin our own CI to detect silent merge conflicts.
There was a problem hiding this comment.
To give some more context: We are now trying to add 200+ lines of code to this repo which have to be maintained, but they cause label spam, and likely aren't sufficient to achieve the goal, because the ci-failed notification will go to the one adding the label and triggering the workflow?
If so, we'd have to create a comment (or other notification) anyway. So if additional glue code is needed, we might as well handle all code externally.
There was a problem hiding this comment.
It's a fair comment. I'll see what the ci-failed notification could look like but you could be right that this approach is less than ideal. @0xB10C has suggested perhaps merging this into the main CI file but might come with a verbose predicate before each job.
There was a problem hiding this comment.
Yeah, merging this with the main CI file is certainly better, if possible. However, it still leaves the issue of label spam. Also, it needs to be confirmed to be working correctly (to send the failed-notification to the right person)
|
Nice work on this! The label-based trigger for PeriodicMergeCICheck is a clean approach. One thought though since this is driven by label events, do we need to consider cases where multiple labels are applied at once? Would the condition still behave as expected? Also agree with the suggestion from @maflcko having a vector of task names could as well improve flexibility |
|
Other stuff to check would be that this works at all and doesn't just re-run an ancient commit, like #32989 (comment) |
Just ran a test to specifically test for this by updating the default (master) branch and adding and removing the label and it correctly picked an updated merge commit based on the PR merged into the updated master. |
There was a problem hiding this comment.
Have you considered doing something like the following in .github/workflows/ci.yml to avoid having to maintain separate files?
name: CI
on:
push:
pull_request:
types: [opened, reopened, synchronize, labeled]
jobs:
conditional-job:
if: >
github.event_name == 'push' ||
(
github.event_name == 'pull_request' &&
(
github.event.action == 'opened' ||
github.event.action == 'reopened' ||
github.event.action == 'synchronize' ||
(
github.event.action == 'labeled' &&
contains(github.event.label.name, 'PeriodicMergeCICheck')
)
)
)
runs-on: ubuntu-latest
steps:
- run: echo "Running CI job"
Could be a good shout, seems a bit of a verbose check if that has to be included in each job. I'll have a think about it. This is exactly the type of feedback that's good to consider. |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
|
concept ack, but I don't have the slightest idea if this is even possible (or how) with GHA |
|
Currently working on an alternative approach of one job that loops through PRs, might close this PR in favour of the other one based on feedback. |
With the upcoming potential migration to cirrus runners (#32989) we need a way to periodically check for a silent merge conflict. Cirrus CI currently does this every week.
This is part of a proposal that will look for a label on a PR as a trigger to run the lint and get_previous_releases test. This label could be added manually or better, Drahtbot could periodically add (and remove) this label.
An example of this running on a test org: https://github.com/testing-cirrus-runners/bitcoin/actions/runs/16780213204/job/47516540034