Skip to content

Conversation

nilsgstrabo
Copy link
Contributor

@nilsgstrabo nilsgstrabo commented Nov 21, 2023

Description

Added a new flag skip-claims-from-profile-url that controls if ProviderData will provide ClaimExtractor with the ProfileURL (resolved from OIDC metadata or manually set). If set to true, ProfileURL is not provided, and ClaimExtractor will not call the profile URL endpoint to resolve claims missing in the id_token.

The default value is false, and should therefore not break default behavior.

Motivation and Context

When using login.microsoftonline.com as OIDC issuer (e.g. https://login.microsoftonline.com/common/v2.0) and you define a scope (e.g. openid profile offline_access 6dae42f8-4368-4678-94ff-3960e28e3630/user.read) with a custom API resource, the access_token received from the token endpoint cannot be used as a bearer token with the userinfo (profile URL) endpoint; The aud claim is wrong. When ClaimExtractor calls the userinfo endpoint a 401 is returned, which in turn cause oauth2-proxy to return 500 to the client.

The new flag skip-claims-from-profile-url instructs the ProviderData to not pass the ProfileURL to ClaimExtractor, thereby preventing a request to the ProfileUR and a subsequent 401 error. The flag is only required when a custom API resource is defined in the scope.

The change proposed in this PR should solve #1666 and #2314

How Has This Been Tested?

As mentioned, when using oauth2-proxy without defining a custom resources in scope, everything works fine, for example:

    image: quay.io/oauth2-proxy/oauth2-proxy:v7.5.1
    environment:
      - OAUTH2_PROXY_SCOPE=openid profile offline_access email
      - OAUTH2_PROXY_OIDC_ISSUER_URL=https://login.microsoftonline.com/3aa4a235-b6e2-48d5-9195-7fcf05b459b0/v2.0
      - OAUTH2_PROXY_UPSTREAMS=http://proxy:8080

But if I add a custom resource to the scope it fails:

  auth:
    image: quay.io/oauth2-proxy/oauth2-proxy:v7.5.1
    environment:
      - OAUTH2_PROXY_SCOPE=openid profile offline_access email 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
      - OAUTH2_PROXY_OIDC_ISSUER_URL=https://login.microsoftonline.com/3aa4a235-b6e2-48d5-9195-7fcf05b459b0/v2.0
      - OAUTH2_PROXY_UPSTREAMS=http://proxy:8080

with the following log entries from the oauth2-proxy

radix-auth_container        | [2023/11/21 09:20:03] [oauthproxy.go:834] Error redeeming code during OAuth2 callback: could not get claim "groups": failed to fetch claims from profile URL: error making request to profile URL: unexpected status "401": {"error":{"code":"InvalidAuthenticationToken","message":"Access token validation failure. Invalid audience.","innerError":{"date":"2023-11-21T09:20:03","request-id":"1a60f131-bd91-4337-a8d9-59efc4456cfe","client-request-id":"1a60f131-bd91-4337-a8d9-59efc4456cfe"}}}
radix-auth_container        | 172.24.0.1:57348 - ef160e37-444e-454a-be7e-909f2a3abe93 - - [2023/11/21 09:20:02] localhost:8000 GET - "/oauth2/callback?code=<deprectaed>&state=Z8DwjH0hu4HMdVH6p2rPkIeqi1vctezHO28oVBTpDWA%3a%2f&session_state=6c6765ae-b9bc-41bc-a31e-c5e13adb2be8" HTTP/1.1 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" 500 2771 0.450

because the access_token has a aud claim not intended for the userinfo endpoint.

If I use my own image built with the new option, everything works as intended (at least for me)

  auth:
    image: nilsgustavstrabo/oauth2-proxy:fix
    environment:
      - OAUTH2_PROXY_SCOPE=openid profile offline_access email 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
      - OAUTH2_PROXY_OIDC_ISSUER_URL=https://login.microsoftonline.com/3aa4a235-b6e2-48d5-9195-7fcf05b459b0/v2.0
      - OAUTH2_PROXY_UPSTREAMS=http://proxy:8080
      - OAUTH2_PROXY_SKIP_CLAIMS_FROM_PROFILE_URL=true

If I set OAUTH2_PROXY_SKIP_CLAIMS_FROM_PROFILE_URL=false, or omit it, it falls back do default behavior and fails the same way as v7.5.1 does.

Checklist:

Not sure what tests to update. Existing tests in options package seems very limited, at least when I look for tests for other options.

I could add tests to ClaimExtractor if I pass the SkipClaimsFromProfileURL as an argument, and let ClaimExtractor decided if it should call the provided profileURL or not. The way is works now is that ProviderData passes nil as profileUrl if skip-claims-from-profile-url is true.

  • My change requires a change to the documentation or CHANGELOG.
  • I have updated the documentation/CHANGELOG accordingly.
  • I have created a feature (non-master) branch for my PR.
  • I have written tests for my code changes.

@nilsgstrabo nilsgstrabo marked this pull request as ready for review November 21, 2023 10:05
@nilsgstrabo nilsgstrabo requested a review from a team as a code owner November 21, 2023 10:05
@nilsgstrabo nilsgstrabo changed the title Feature/add skip claims from profile url flag Feature/add option to skip loading claims from profile url Nov 21, 2023
@nilsgstrabo
Copy link
Contributor Author

Any input/feedback from anyone?

@nilsgstrabo
Copy link
Contributor Author

@tuunit @JoelSpeed Any input/thoughts on this?

@kvanzuijlen kvanzuijlen self-requested a review December 7, 2023 00:02
Copy link
Member

@kvanzuijlen kvanzuijlen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code seems quite simple (which is good). It feels a bit hackish though. Wouldn't it be better to be able to configure (a list of) claims that should be ignored and/or that should be ignored if not available from the ID token? Defaults can then eventually be set from the code for i.e. the Azure provider (if indeed necessary).

@tuunit Do you agree or am I overthinking/overcomplicating?

@nilsgstrabo
Copy link
Contributor Author

nilsgstrabo commented Dec 7, 2023

When we define the scope to be for a resource/api other than MS Graph, e.g. openid profile offline_access email 6dae42f8-4368-4678-94ff-3960e28e3630/user.read, then the userinfo endpoint will always return a 401 Unauthorized. With a claims ignore-list, we would need to provide every claim used internally by oauth-proxy which is not present in the ID token. If new versions of oauth2-proxy introduces new claims for ProviderData to get from ClaimsExtractor, and these claims are not present in ID Token, it would break again, and we would have to read through the code to figure out what claim we must add to the ignore list.

One option is to build the ignore-list internally by inspecting other flags like --authenticated-emails-file and --email-domain (if none are set, add email_verified claim to ignore list) and --allowed-group (add GroupsClaim to ignore list). This is more error prone if developers introduce new claims to extract based on some condition (e.g. flags), but forget to add it to the ignore-list if the condition is not met.

@nilsgstrabo
Copy link
Contributor Author

Any update @kvanzuijlen?
It's ok if you want to reject the PR. No hard feelings :)
In that case we'll just build from our fork since we are stuck on an old version.

@kvanzuijlen
Copy link
Member

Hi @nilsgstrabo,

Sorry, I completely overlooked your response. Thanks for the detailed explanation. I'd be curious what other maintainers think as well. @tuunit @JoelSpeed any comments?

@tuunit
Copy link
Member

tuunit commented Dec 20, 2023

Hi, will check it this evening!

Copy link
Member

@tuunit tuunit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, a really well written PR. Thanks a lot for all the effort you put into your description of the issue.

Comment on lines 287 to 292
var profileURL *url.URL
if !p.SkipClaimsFromProfileURL {
profileURL = p.ProfileURL
}

extractor, err := util.NewClaimExtractor(context.TODO(), rawIDToken, profileURL, p.getAuthorizationHeader(accessToken))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I was a bit concerned that this change could cause error messages or other issues because of the uninitialized url.URL when initializing the ClaimExtractor. But after checking the implementation of loadProfileClaims() in providers->util->claim_extractor.go I say we are good here. As we already handle the case for an unconfigured / nil url.URL correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this use case already seems to be well tested in claim_extractor_test.go as well.

@@ -283,7 +284,12 @@ func (p *ProviderData) buildSessionFromClaims(rawIDToken, accessToken string) (*
}

func (p *ProviderData) getClaimExtractor(rawIDToken, accessToken string) (util.ClaimExtractor, error) {
Copy link
Member

@tuunit tuunit Dec 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding testing. Could you take some time for extending the test case TestProviderData_buildSessionFromClaims inside provider_data_test.go

We should be able to set the new skip option inside a new test case entry and get a valid response:

provider.AllowUnverifiedEmail = tc.AllowUnverified
provider.UserClaim = tc.UserClaim
provider.EmailClaim = tc.EmailClaim
provider.GroupsClaim = tc.GroupsClaim
rawIDToken, err := newSignedTestIDToken(tc.IDToken)
g.Expect(err).ToNot(HaveOccurred())
ss, err := provider.buildSessionFromClaims(rawIDToken, "")
if err != nil {
g.Expect(err).To(Equal(tc.ExpectedError))
}
if ss != nil {
g.Expect(ss).To(Equal(tc.ExpectedSession))
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will take a look.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing SkipClaimsFromProfileURL turned out to be a bit trickier than expected. The only feasible way I can see, is to use httptest.Server, set ProfileURL (and AccessToken + getAuthorizationHeaderFunc), and test if the server receives a request based on the value for SkipClaimsFromProfileURL. Ref commit 7dc8c14

Please let me know if you have any better ideas or suggestions.

@github-actions github-actions bot added dependencies Pull requests that update a dependency file tests labels Dec 22, 2023
Copy link
Member

@tuunit tuunit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nilsgstrabo
Whats your opinion?

profileURLTracker mock.Mock
)
if tc.SetProfileURL {
profileURLSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with setting up a http test server here. We do something quite similar in claim_extractor_test.go for the profileURL as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But do we really need to introduce a new go dependency for this "simple" test?

Couldn't we just introduce a simple custom handler like in claim_extractor_test.go#L556:

func requiresAuthProfileHandler(rw http.ResponseWriter, req *http.Request) {
if !hasAuthorizedHeader(req.Header) {
rw.WriteHeader(403)
rw.Write([]byte("Unauthorized"))
return
}
rw.Write([]byte(basicProfileURLPayload))
}

This way we don't need to introduce a new dependency to objx and mock.Mock and we will not need to the assertion in line 485

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. The mock.Mock package is completely overkill here. Only need to test that a request to profileURL is sent or not.
Fixed in ceccfb2

@github-actions github-actions bot removed the dependencies Pull requests that update a dependency file label Dec 27, 2023
Copy link
Member

@tuunit tuunit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tuunit tuunit added the LGTM PR is ready to merge label Dec 27, 2023
@nilsgstrabo
Copy link
Contributor Author

@tuunit Is there anything else I need to do with this PR? Will it be included in the next release, and do you know approximately when the next release is planned?

JoelSpeed
JoelSpeed previously approved these changes Jan 20, 2024
@JoelSpeed
Copy link
Member

Code generation needs to be updated here

@nilsgstrabo
Copy link
Contributor Author

nilsgstrabo commented Jan 20, 2024

Code generation needs to be updated here

Something I need to do?

@JoelSpeed
Copy link
Member

Please run make generate and then commit the changes

@nilsgstrabo
Copy link
Contributor Author

Done and pushed

@JoelSpeed JoelSpeed merged commit 4c2bf5a into oauth2-proxy:master Jan 20, 2024
tuunit added a commit to tuunit/oauth2-proxy that referenced this pull request Jan 21, 2024
* enhancement: Change base image from alpine to distroless (oauth2-proxy#2295)

* Changed base image from alpine to distroless

* chore: updated Makefile

* fix: removed arm/v6 and ppc64le for distroless variant

* Update Dockerfile

* Update Makefile

* docs: Add README-section, CHANGELOG-entry and --pull to prevent caching

---------

Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* Add possibility to encode the state param as UrlEncodedBase64 (oauth2-proxy#2312)

* Add possibility to encode the state param as UrlEncodedBase64

* Update CHANGELOG.md

* Update oauthproxy.go

Co-authored-by: Jan Larwig <jan@larwig.com>

---------

Co-authored-by: Jan Larwig <jan@larwig.com>

* NGINX return 403 for sign_in (oauth2-proxy#2322) (oauth2-proxy#2323)

Co-authored-by: Sven Ertel <sven.ertel@bayernwerk.de>

* chore: Create sha256sum for tar instead of binary (oauth2-proxy#2343)

* Create sha256sum for tar instead of binary

* chore: Add checksum for binary

* chore: Updated changelog

---------

Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* Log error details when failed loading CSRF cookie (oauth2-proxy#2345)

* Log error details when failed loading CSRF cookie

* Add a record about this PR to CHANGELOG.md

---------

Co-authored-by: Ondrej Charvat <ondrej.charvat@yunextraffic.com>
Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* Feature - Add env variable support for alpha struct (oauth2-proxy#2375)

* added envsubstring package and added simple test cases.imple tests.

* added documentation

* added changelog entry

* added documentation to wrong file


.

* changed tests to ginkgo format

* update project to use better maintained library

* use defer to clear test variable after tests finished

* updated docs for the new package documentation and fixed bad english

* refactored function to "reduce" complexity.

* updated changelog for new version

updated readme

* minor formatting

---------

Co-authored-by: Haydn Evans <h.evans@douglas.de>
Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* remove nsswitch workaround (oauth2-proxy#2371)

Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* feat: Added renovate configuration (oauth2-proxy#2377)

* Feature/add option to skip loading claims from profile url (oauth2-proxy#2329)

* add new flag skip-claims-from-profile-url

* skip passing profile URL if SkipClaimsFromProfileURL

* docs for --skip-claims-from-profile-url flag

* update flag comment

* update docs

* update CHANGELOG.md

* Update providers/provider_data.go

Co-authored-by: Jan Larwig <jan@larwig.com>

* Add tests for SkipClaimsFromProfileURL

* simplify tests for SkipClaimsFromProfileURL

* generate alpha_config.md

---------

Co-authored-by: Jan Larwig <jan@larwig.com>

* Add ability to configure username for Redis cluster connections (oauth2-proxy#2381)

* Initial attempt.

* Add CHANGELOG entry.

* Drop commented-out Sentinel test.

---------

Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>

* chore(deps): update module golang.org/x/crypto to v0.17.0 [security] (oauth2-proxy#2400)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update github.com/ghodss/yaml digest to d8423dc (oauth2-proxy#2401)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Improved dev environment (oauth2-proxy#2211)

* Improved dev env setup

* Cleanup duplicate checks

* Applied PR feedback

* Updated go.mod/go.sum

* go mod tidy

* Update .devcontainer/devcontainer.json

* Update pkg/http/server_test.go

Co-authored-by: Jan Larwig <jan@larwig.com>

* Create launch.json

* Update .devcontainer/Dockerfile

* Apply suggestions from code review

---------

Co-authored-by: Jan Larwig <jan@larwig.com>

* feature: add release automation workflows (oauth2-proxy#2224)

* feature: add release automation workflows

* deactivate provenancee because of behaviour change with buildx v0.10.0

* add changelog section extraction for github release notes

* fix registry path; fix EOF

* use correct version of golangci-lint; add additional workflow step for fetching all dependencies

* chore(deps): update module github.com/bsm/redislock to v0.9.4 (oauth2-proxy#2406)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* initial commit for docusaurus 3 upgrade

* fix mdx errors

* fix mdx issues

* fix routing issues

* update docs generation workflow

* fix version

* fix permissions

* move slack to header

* remove background color and minify

* Update docs/docs/configuration/providers/openid_connect.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/openid_connect.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/openid_connect.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/gitlab.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/gitlab.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/github.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/github.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/github.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

* Update docs/docs/configuration/providers/github.md

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>

---------

Co-authored-by: Koen van Zuijlen <8818390+kvanzuijlen@users.noreply.github.com>
Co-authored-by: Joel Speed <Joel.speed@hotmail.co.uk>
Co-authored-by: Jan Brezina <brezinajn@users.noreply.github.com>
Co-authored-by: WhiteRabbit-Code <sven@ertel-net.de>
Co-authored-by: Sven Ertel <sven.ertel@bayernwerk.de>
Co-authored-by: charvadzo <120425386+charvadzo@users.noreply.github.com>
Co-authored-by: Ondrej Charvat <ondrej.charvat@yunextraffic.com>
Co-authored-by: Haydn Evans <h.evans@douglas.de>
Co-authored-by: Nils Gustav Stråbø <65334626+nilsgstrabo@users.noreply.github.com>
Co-authored-by: Ross Golder <ross@golder.org>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants