Skip to content

Conversation

cezarsa
Copy link
Contributor

@cezarsa cezarsa commented Oct 13, 2022

Occasionally AWS returns 401 unauthorized errors on GetRoleCredentials with OIDC tokens that shouldn't have expired yet according to the expiration date returned by AWS. The error message in these situations looks like this:

aws-vault: error: exec: Failed to get credentials for <redacted>: operation error SSO: GetRoleCredentials, https response error StatusCode: 401, RequestID: 10a9bfff-90f9-48f4-8ab7-<redacted>, UnauthorizedException: Session token not found or invalid

When this happened the only solution was to run aws-vault clear.

This error has occurred in the past when AWS returned the wrong expiration date for some time (#854 and #722) and it still happens occasionally to some users, I guess that's due to AWS invalidating tokens on some situations for security reasons.

This PR adds the ability for aws-vault to detect when a 401 error is returned, clear the cached OIDC token and retry the operation once with a new OIDC token.

@@ -66,6 +69,23 @@ func (p *SSORoleCredentialsProvider) getRoleCredentials(ctx context.Context) (*s
RoleName: aws.String(p.RoleName),
})
if err != nil {
if cached && p.OIDCTokenCache != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a possibility that a token is invalidated between the time that it's created, so I wonder if checking cached is necessary at all

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking cached ensures getRoleCredentials won't be stuck in an infinite recursion loop if some AWS bug was to cause CreateToken to return invalid or expired tokens. In these rare situations, I thought it would be safer to return an error immediately instead of possibly retrying forever.

@mtibben mtibben merged commit 8ea0b72 into 99designs:master Dec 19, 2022
@mtibben
Copy link
Member

mtibben commented Dec 19, 2022

Thanks @cezarsa

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants