Skip to content

Conversation

fanquake
Copy link
Member

@fanquake fanquake commented Feb 6, 2020

Addresses one part of #16664, by making it easier to identify CI containers that are running locally. By default Docker will generate random names, like peaceful_rubin, with this change, we explicitly set names for all containers.

@fanquake fanquake added the Tests label Feb 6, 2020
@fanquake fanquake requested a review from maflcko February 6, 2020 03:59
@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 6, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #18077 (net: Add NAT-PMP port forwarding support by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, and LGTM.

@hebasto
Copy link
Member

hebasto commented Feb 6, 2020

What are the actual benefits of exporting DOCKER_NAME variable? Do we use it somehow?

@fanquake fanquake force-pushed the name_ci_docker_containers branch from db4df60 to d87e029 Compare February 6, 2020 12:40
@theStack
Copy link
Contributor

theStack commented Feb 6, 2020

Concept ACK.

What are the actual benefits of exporting DOCKER_NAME variable? Do we use it somehow?

It is used as argument for Docker (--name $DOCKER_NAME) to give the containers a concrete name instead of generating a random one, see #16623 (review).

@hebasto
Copy link
Member

hebasto commented Feb 6, 2020

It is used as argument for Docker (--name $DOCKER_NAME) to give the containers a concrete name instead of generating a random one, see #16623 (review).

I know it ;)
What are container concrete names used for? What is wrong with random ones? (All questions are in context of CI)

@theStack
Copy link
Contributor

theStack commented Feb 6, 2020

What are container concrete names used for? What is wrong with random ones? (All questions are in context of CI)

From my perspective it would be a convenience feature right now: whenever as developer you make non-trivial changes in the CI scripts you very likely need to reproduce the travis build locally (as described in e.g. https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine) to dig deeper and inspect the container in some way (show logs, kill it, restart it, execute a command inside it etc. etc....) . For that it is way more comfortable to have deterministic names instead of the need to always find out the container hash or random name via docker ps first.

@hebasto
Copy link
Member

hebasto commented Feb 6, 2020

From my perspective it would be a convenience feature right now: whenever as developer you make non-trivial changes in the CI scripts you very likely need to reproduce the travis build locally (as described in e.g. https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine) to dig deeper and inspect the container in some way (show logs, kill it, restart it, execute a command inside it etc. etc....) . For that it is way more comfortable to have deterministic names instead of the need to always find out the container or random name via docker ps first.

In case a developer keeps travis docker containers locally, it is up to her to patch scripts locally.

IMHO, no need for these changes to the repository (if the only use case is local running).

@maflcko
Copy link
Member

maflcko commented Feb 6, 2020

Just a hint that how to run locally is described here: https://github.com/bitcoin/bitcoin/tree/master/ci#running-a-stage-locally

Travis only provides the compute to use, we don't rely on any of its software

@theStack
Copy link
Contributor

theStack commented Feb 6, 2020

In case a developer keeps travis docker containers locally, it is up to her to patch scripts locally.

IMHO, no need for these changes to the repository (if the only use case is local running).

While I agree that the changes are not strictly needed right now, I'd always prefer a reproducible behaviour to random behaviour in CI/testing, even if it concerns only something trivial as just container names.

Just a hint that how to run locally is described here: https://github.com/bitcoin/bitcoin/tree/master/ci#running-a-stage-locally

I'm aware of that, but just because all CI stages pass on a local (highly individual) development environment doesn't necessarily mean it also does on the Travis environment. While working on #17900 locally everything was passing fine, but on Travis it failed. I didn't see any alternative as to recreate the Travis environment in a Docker container locally to reproduce exactly what's going an and what is different to my local development machine (that's why I wrote "non-trivial" changes in my reasoning).

Travis only provides the compute to use, we don't rely on any of its software

Quick counter-example: from where would the testing scripts get the variable $TRAVIS_OS_NAME from without Travis (set in .travis.yml)? I highly doubt that the CI setup as it is right now is as decoupled from Travis than most people think (primarily because it is almost never run locally by anyone, only from people making changes).

@maflcko
Copy link
Member

maflcko commented Feb 6, 2020

Quick counter-example: from where would the testing scripts get the variable $TRAVIS_OS_NAME from without Travis (set in .travis.yml)? I highly doubt that the CI setup as it is right now is as decoupled from Travis than most people think (primarily because it is almost never run locally by anyone, only from people making changes).

I do test all linux builds locally on an arm machine continuously. TRAVIS_OS_NAME is only used to distinguish osx from linux, and should work fine unless you want to run the osx build locally.

@fanquake
Copy link
Member Author

fanquake commented Feb 7, 2020

I don't really understand the push back here. This is straightforward, and makes it easier to identify containers when you're running tests locally. I can't see why that's something you should have to "patch scripts locally." to do. However if this is so controversial it can just be closed.

@hebasto
Copy link
Member

hebasto commented Feb 7, 2020

@fanquake

However if this is so controversial it can just be closed.

Please don't close.

This is straightforward, and makes it easier to identify containers when you're running tests locally.

It was not clear for me from the OP. Now I'm convinced that these changes are useful.

Concept ACK.
Could the PR description be more descriptive with adding "makes it easier to identify containers when you're running tests locally"?

Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d87e029
(The occured travis fail is unrelated, has to do with low disk space for s390x build, see #18016)

@fanquake
Copy link
Member Author

fanquake commented Feb 7, 2020

Could the PR description be more descriptive

I've updated the PR description.

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK, though I'd prefer to use the ci_ prefix for the name to avoid name collisions with identically named local containers.

Error: error creating container storage: the container name "centos_7" is already in use by "b61e5703bd1b6d0021dfc8d7b0b21cf2843721a4081bbdd7027c7c511e63a9b4". You have to remove that container to be able to reuse that name.: that name is already in use

@fanquake fanquake force-pushed the name_ci_docker_containers branch from d87e029 to 9e111db Compare February 10, 2020 12:04
@fanquake
Copy link
Member Author

though I'd prefer to use the ci_ prefix for the name

Done, and addressed other comments.

@maflcko
Copy link
Member

maflcko commented Feb 10, 2020

ACK 9e111db

laanwj added a commit that referenced this pull request Feb 10, 2020
9e111db test: set a name for CI Docker containers (fanquake)

Pull request description:

  Addresses one part of #16664, by making it easier to identify CI containers that are running locally. By default Docker will generate random names, like `peaceful_rubin`, with this change, we explicitly set names for all containers.

ACKs for top commit:
  MarcoFalke:
    ACK 9e111db

Tree-SHA512: 0a29ada0d8cf6b0e9ae7a35f4f6df7a3dcc448523ceaed01371124360d6e3d1bf351172104a5fb629488eeaa57994ba04134dcb83c261eb1dfd2f0d73edf5f60
@laanwj laanwj merged commit 9e111db into bitcoin:master Feb 10, 2020
@fanquake fanquake deleted the name_ci_docker_containers branch February 12, 2020 00:36
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants