Skip to content

Conversation

jenniferplusplus
Copy link
Contributor

@jenniferplusplus jenniferplusplus commented Jan 29, 2024

The k8s Vault Agent can automatically handle authentication to Vault, and inject a client token into a shared volume. The Vault agent manages the token lifecycle and refreshes the file contents as necessary. This change enables Concourse to read that token file, and take advantage of the agent's managed token.

Changes proposed by this PR

  • Read vault client token from file
  • Update config?
  • Tests?

Notes to reviewer

  • I think this is a very straightforward change, but the vault API client doesn't appear to have much test coverage, and I'm not sure how I would go about adding it.
  • Does anything else have to be done to add the appropriate CLI args and env vars? It didn't seem like it, but I'm not entirely sure.

Release Note

  • This allows Concourse to use the client tokens provided by the K8s Vault-Agent sidecar by setting CONCOURSE_VAULT_CLIENT_TOKEN_PATH

@jenniferplusplus jenniferplusplus requested a review from a team as a code owner January 29, 2024 21:56
@jenniferplusplus
Copy link
Contributor Author

I see that some checks failed, but I don't have access to view them, so I don't know how to respond to that.

@xtremerui
Copy link
Contributor

@jenniferplusplus Hi, there is a CI issue we have to make the concourse-ci/integration check failed. We are fixing it in this PR. Once that is merge the check here should pass.

@xtremerui
Copy link
Contributor

If you could rebase master I think those checks will pass now.

@jenniferplusplus
Copy link
Contributor Author

@xtremerui Thanks for the followup. I'm traveling for work this week, but I'll try to get to that soon.

The k8s Vault Agent can automatically handle authentication to Vault, and inject a client token into a shared volume. The Vault agent manages the token lifecycle and refreshes the file contents as necessary. This change enables Concourse to read that token file, and take advantage of the agent's managed token.

Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
@xtremerui
Copy link
Contributor

xtremerui commented Feb 8, 2024

Thank you @jenniferplusplus . There is indeed concourse-bosh-release and concourse-chart that needs to updated with the new env var. You can find an example like this PR where at the bottom there are PRs for adding env var to those 2 repos.

Also, if possible, could you add an integration test for this feature. So far, we have a general creds test that utilizing vault here. I suppose you can add a local client token file and config the vault to pick it up. But I am not sure how you would simulate an content update of that file like k8s vault agent.

@jenniferplusplus
Copy link
Contributor Author

@xtremerui just following up, I opened PRs to the bosh and chart repos to add the appropriate config value. I'm not in the habit of maintaining either kind of project, so please let me know if I got any of that wrong.

I'll try to figure out a way to do integration tests soon.

Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
@jenniferplusplus
Copy link
Contributor Author

jenniferplusplus commented Feb 20, 2024

@xtremerui I've had no luck getting any integration tests to run locally. I'm doing this work on an M1 mac, and that really complicates working with Concourse's docker setup. Still, I think these integration tests will work. I'm hoping CI can answer that question for me.

edit: also, for what it's worth, I've had my own build from the original commits on this PR running continuously for 3 weeks, with no issues. I know that's not the same as explicit integration tests, but it's also not nothing.

@xtremerui
Copy link
Contributor

xtremerui commented Feb 23, 2024

@jenniferplusplus could you confirm whether you can see the build logs here for the failed integration test? You will need to accept the invitation to join Concourse org to join the team to see the build logs, if local dev doesn't work for you.

@jenniferplusplus
Copy link
Contributor Author

@xtremerui I accepted the invite and I can see the build logs now. I wasn't aware I had gotten an invite until you mentioned it.

Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
@jenniferplusplus
Copy link
Contributor Author

So, it looks like something gave an exit code of 1, but I can't tell what it is or why. I really don't know what else I can do at this point.

BTW, I'd be happy to discuss in discord, but I still don't have access to the contributors channel there, either.

Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
Signed-off-by: Jennifer Moore <contact@jenniferplusplus.com>
@jenniferplusplus
Copy link
Contributor Author

@xtremerui
Yes!

Copy link
Contributor

@xtremerui xtremerui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@xtremerui xtremerui added this to the v7.12.0 milestone Feb 29, 2024
@xtremerui xtremerui merged commit cc7a76e into concourse:master Feb 29, 2024
@jenniferplusplus jenniferplusplus deleted the vault-token-path branch February 29, 2024 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants