Skip to content

ci: Run self-hosted ci #21619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Apr 11, 2021
Merged

ci: Run self-hosted ci #21619

merged 4 commits into from
Apr 11, 2021

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Apr 6, 2021

Due to our heavy use of the Cirrus CI community cluster, some tasks may take a long time to get scheduled. While it is possible to use "Compute Credits" to get immediately scheduled on the cluster, I couldn't find a sponsor that'd be willing to cover the total cost, if all tasks were paid for with credits.

However, it is also possible to bring our own runners to Cirrus CI.

For testing purposes, a single task will be transformed to run on the DrahtBot infrastructure. If all goes well, the other tasks can be moved, too.

@DrahtBot DrahtBot added the Tests label Apr 6, 2021
@maflcko maflcko force-pushed the 2103-ciSelfHosted branch from fa1291b to fa38c0c Compare April 6, 2021 20:42
@jarolrod
Copy link
Member

jarolrod commented Apr 6, 2021

Concept ACK

DrahtBot strong 🦾

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 6, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot mentioned this pull request Apr 7, 2021
Merged
@maflcko maflcko force-pushed the 2103-ciSelfHosted branch from fa38c0c to fa4d30c Compare April 9, 2021 06:01
hebasto
hebasto reviewed Apr 9, 2021
View reviewed changes
Copy link
Member

@hebasto hebasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

Commit "ci: Restart docker before run". Why we need to restart docker? Isn't it a CI's responsibility to create a new container for each task?

MarcoFalke added 4 commits April 9, 2021 15:56
ci: [refactor] Create setting for ephemeral config in .cirrus.yml
fad4f48 
This allows easier switching between self-hosted runners and the
community cluster. Also, named variables can be documented better.
ci: Restart docker before run
fa27841 
Also, add setting for persistent worker in .cirrus.yml
ci: Make cirrus cache folders relative to cirrus base dir
fa52a40
ci: Run self-hosted ci
fa41a91
@maflcko maflcko force-pushed the 2103-ciSelfHosted branch from fa4d30c to fa41a91 Compare April 9, 2021 14:06
@maflcko
Copy link
Member Author

maflcko commented Apr 9, 2021

Why we need to restart docker? Isn't it a CI's responsibility to create a new container for each task?

This can simplify the local workflow when ci tasks are executed repeatedly. Also, it is needed for persistent runners, which are not ephemeral. See also https://cirrus-ci.org/guide/persistent-workers/#what-is-a-persistent-worker

hebasto
hebasto approved these changes Apr 10, 2021
View reviewed changes
Copy link
Member

@hebasto hebasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fa41a91, I have reviewed the code and it looks OK.

CI log looks OK.

@hebasto
Copy link
Member

hebasto commented Apr 10, 2021

Unrelated to this PR, but why is depends_release cache populated after CI run?

Screenshot from 2021-04-10 16-04-35

@maflcko maflcko merged commit 1e3db68 into bitcoin:master Apr 11, 2021
@maflcko maflcko deleted the 2103-ciSelfHosted branch April 11, 2021 11:15
This was referenced Apr 11, 2021
Merged
Merged
@maflcko
Copy link
Member Author

maflcko commented Apr 11, 2021

Unrelated ...

Fixed in #21653

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Apr 11, 2021
Merge bitcoin#21619: ci: Run self-hosted ci
cec8ee9
RonSherfey
RonSherfey reviewed Apr 11, 2021
View reviewed changes
Copy link

@RonSherfey RonSherfey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed changes

78051301012
78051301012 reviewed Apr 11, 2021
View reviewed changes
Copy link

@78051301012 78051301012 left a comment •
edited by fanquake
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@hebasto hebasto hebasto approved these changes

+2 more reviewers

@78051301012 78051301012 78051301012 left review comments

@RonSherfey RonSherfey RonSherfey left review comments

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
Tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@maflcko @jarolrod @DrahtBot @hebasto @78051301012 @RonSherfey