Skip to content

Conversation

anandf
Copy link
Member

@anandf anandf commented Feb 7, 2025

Fixes #18968

This PR addresses adds improvements for #20993

PR Summary:

This PR addresses an issue with the argocd.argoproj.io/manifest-generate-paths Manifest Paths Annotation in Argo CD when used with Bitbucket repositories.

Key Changes:

Prevents spurious refreshes in cases where multiple Applications are using the same Git monorepo stored in Bitbucket.
Ensures that the manifest-generate-paths annotation is handled correctly, refreshing the app only when the files changed match with the paths in manifest-generate-paths .

Steps to Verify this feature:

  1. Disable the periodic polling of the git repositories by the following command to ensure that all refreshes happen only via webhook notifications.
kubectl patch cm argocd-cm -n argocd -p '{"data": {"timeout.reconciliation": "0"}}'
  1. Set the debug logs in Argo CD Server component using the following command to verify changed files are being calculated correctly.
kubectl patch cm argocd-cmd-params-cm -n argocd -p '{"data": {"server.log.level": "debug"}}'
  1. Restart the argocd-server deployment to ensure that the log level configuration change is picked up by the application.
kubectl scale deploy argocd-server -n argocd --replicas=0
kubectl scale deploy argocd-server -n argocd --replicas=1
  1. Create a webhook for the private repository using the steps provided in https://support.atlassian.com/bitbucket-cloud/docs/manage-webhooks/

  2. Create a Repository webhook token in BitBucket for the private repository and add the repository secret as follows
    Detailed steps provided in this link : https://support.atlassian.com/bitbucket-cloud/docs/create-a-repository-access-token/

argocd login <server_hostname> --username admin --password $(kubectl get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d) --insecure
export WEBHOOK_TOKEN=<token_from_bb_console>
argocd repo add https://bitbucket.org/anandjoseph/argocd-sample-pvt.git --username x-token-auth --password $(echo $WEBHOOK_TOKEN)
  1. Create a webhook in BitBucket, update the webhook UUID in secret
export WEBHOOK_UUID=<uuid_from_bb_console>
kubectl patch secret argocd-secret -n argocd -p '{"stringData": {"webhook.bitbucket.uuid": "$WEBHOOK_UUID"}}'
  1. Check the logs and see if you find some debug logs as below
{"level":"debug","msg":"https://bitbucket.org/anandjoseph/argocd-sample-pvt.git has credentials","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"quay.io/rh-ee-anjoseph has credentials","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"found a matching repository for URL https://bitbucket.org/anandjoseph/argocd-sample-pvt","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"fetched user/password for repository URL 'https://bitbucket.org/anandjoseph/argocd-sample-pvt.git', initializing basic auth client","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"created bitbucket client with base URL 'https://api.bitbucket.org/2.0'","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"invoking diffstat call with parameters: [Owner:anandjoseph, RepoSlug:argocd-sample-pvt, Spec:..397e8a657cc47a32b8c4e150cb731a0f9d3cb751]","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"changed files for spec ..397e8a657cc47a32b8c4e150cb731a0f9d3cb751: [guestbook/guestbook-ui-deployment.yaml]","time":"2025-05-18T07:53:50Z"}
{"level":"info","msg":"Received push event repo: https://bitbucket.org/anandjoseph/argocd-sample-pvt, revision: change_version_28, touchedHead: false","time":"2025-05-18T07:53:51Z"}

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

Copy link

bunnyshell bot commented Feb 7, 2025

❌ Preview Environment deleted from Bunnyshell

Available commands (reply to this comment):

  • 🚀 /bns:deploy to deploy the environment

@crenshaw-dev
Copy link
Member

Good iterations on the previous PR. I think we can only enable this feature if webook auth is enabled. Otherwise it's an easy SSRF

@anandf
Copy link
Member Author

anandf commented Feb 7, 2025

Good iterations on the previous PR. I think we can only enable this feature if webook auth is enabled. Otherwise it's an easy SSRF

Would mandating that the request from webhook handler would be made only if the repo creds are present in the system. If there is no secret present for the given URL, then we will not do the call to get the diffs.

In that case, even for a public repository, users may need to add the repocreds.

@anandf anandf changed the title Add support bb monorepo feat(webhook): Fixed manifest-generate-paths annotation support for monorepos in BitBucket Feb 7, 2025
@anandf
Copy link
Member Author

anandf commented Feb 7, 2025

If there is no valid repo creds for a given URL, then no external call is made and the changeset remains empty. Would this check be sufficient to avoid SSRF.

if repoCreds == nil {
		return []string{}, fmt.Errorf("unable to retrive repository credentials for bitbucket workspace %s and reposlug %s", workspace, repoSlug)
	}

Copy link

codecov bot commented Feb 7, 2025

Codecov Report

Attention: Patch coverage is 78.89908% with 23 lines in your changes missing coverage. Please review.

Project coverage is 55.91%. Comparing base (fe347e3) to head (c72b169).
Report is 451 commits behind head on master.

Files with missing lines Patch % Lines
util/webhook/webhook.go 78.89% 17 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #21811      +/-   ##
==========================================
+ Coverage   55.88%   55.91%   +0.02%     
==========================================
  Files         343      343              
  Lines       57333    57437     +104     
==========================================
+ Hits        32042    32117      +75     
- Misses      22644    22671      +27     
- Partials     2647     2649       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@crenshaw-dev
Copy link
Member

I don't think that's sufficient. Allowing a completely unauthenticated user to cause a credentialed API call would allow anyone with network access to use Argo to launch a DoS attack or to DoS Argo itself.

@anandf anandf force-pushed the add_support_bb_monorepo branch 2 times, most recently from 5575227 to 10a6506 Compare February 10, 2025 17:23
@anandf
Copy link
Member Author

anandf commented Feb 10, 2025

@crenshaw-dev Could you please check my recent change, where I am relying on settings.WebhookBitbucketServerSecret to check if its an authenticated webhook or not. If this is set, then all the webhook calls are checked for signature validity. Please advise if there is a better way to do the same.

@anandf anandf force-pushed the add_support_bb_monorepo branch from e365aad to 52378ce Compare February 11, 2025 03:56
@anandf anandf force-pushed the add_support_bb_monorepo branch 3 times, most recently from 17b5f17 to 93791bb Compare February 25, 2025 06:12
@anandf anandf marked this pull request as ready for review February 26, 2025 13:41
@anandf anandf requested a review from a team as a code owner February 26, 2025 13:41
@anandf anandf force-pushed the add_support_bb_monorepo branch from fc44260 to 5220bff Compare March 1, 2025 06:22
@anandf anandf force-pushed the add_support_bb_monorepo branch from bcddb77 to cbbbe00 Compare March 8, 2025 08:55
@anandf anandf force-pushed the add_support_bb_monorepo branch 2 times, most recently from 07ce521 to abd990f Compare March 16, 2025 17:47
@anandf
Copy link
Member Author

anandf commented Mar 16, 2025

@crenshaw-dev I have addressed your review comments and the PR is ready for review. Can it be considered for release-3.0 release milestone ?

@anandf anandf force-pushed the add_support_bb_monorepo branch 4 times, most recently from 8bb575a to 2525c69 Compare March 19, 2025 13:57
Copy link
Member

@jannfis jannfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anandf anandf force-pushed the add_support_bb_monorepo branch 5 times, most recently from 0436f23 to 61a0ffb Compare March 23, 2025 15:10
@anandf anandf force-pushed the add_support_bb_monorepo branch from 61a0ffb to aae1b38 Compare March 24, 2025 16:05
@anandf anandf requested a review from a team as a code owner March 24, 2025 16:05
Signed-off-by: anandf <anjoseph@redhat.com>
@jannfis jannfis merged commit 2fbace3 into argoproj:master Mar 26, 2025
27 checks passed
Hapshanko pushed a commit to Hapshanko/argo-cd that referenced this pull request Apr 29, 2025
…onorepos in BitBucket (argoproj#21811)

Signed-off-by: anandf <anjoseph@redhat.com>
Signed-off-by: Hapshanko <adamgaming391100@gmail.com>
@anandf anandf deleted the add_support_bb_monorepo branch May 18, 2025 03:47
@anandf
Copy link
Member Author

anandf commented May 19, 2025

Steps to verify this feature:

  1. Disable the periodic polling of the git repositories by the following command to ensure that all refreshes happen only via webhook notifications.
kubectl patch cm argocd-cm -n argocd -p '{"data": {"timeout.reconciliation": "0"}}'
  1. Set the debug logs in Argo CD Server component using the following command to verify changed files are being calculated correctly.
kubectl patch cm argocd-cmd-params-cm -n argocd -p '{"data": {"server.log.level": "debug"}}'
  1. Restart the argocd-server deployment to ensure that the log level configuration change is picked up by the application.
kubectl scale deploy argocd-server -n argocd --replicas=0
kubectl scale deploy argocd-server -n argocd --replicas=1
  1. Create a webhook for the private repository using the steps provided in https://support.atlassian.com/bitbucket-cloud/docs/manage-webhooks/

  2. Create a Repository webhook token in BitBucket for the private repository and add the repository secret as follows
    Detailed steps provided in this link : https://support.atlassian.com/bitbucket-cloud/docs/create-a-repository-access-token/

argocd login <server_hostname> --username admin --password $(kubectl get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d) --insecure
export WEBHOOK_TOKEN=<token_from_bb_console>
argocd repo add https://bitbucket.org/anandjoseph/argocd-sample-pvt.git --username x-token-auth --password $(echo $WEBHOOK_TOKEN)
  1. Create a webhook in BitBucket, update the webhook UUID in secret
export WEBHOOK_UUID=<uuid_from_bb_console>
kubectl patch secret argocd-secret -n argocd -p '{"stringData": {"webhook.bitbucket.uuid": "$WEBHOOK_UUID"}}'
  1. Check the logs and see if you find some debug logs as below
{"level":"debug","msg":"https://bitbucket.org/anandjoseph/argocd-sample-pvt.git has credentials","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"quay.io/rh-ee-anjoseph has credentials","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"found a matching repository for URL https://bitbucket.org/anandjoseph/argocd-sample-pvt","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"fetched user/password for repository URL 'https://bitbucket.org/anandjoseph/argocd-sample-pvt.git', initializing basic auth client","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"created bitbucket client with base URL 'https://api.bitbucket.org/2.0'","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"invoking diffstat call with parameters: [Owner:anandjoseph, RepoSlug:argocd-sample-pvt, Spec:..397e8a657cc47a32b8c4e150cb731a0f9d3cb751]","time":"2025-05-18T07:53:49Z"}
{"level":"debug","msg":"changed files for spec ..397e8a657cc47a32b8c4e150cb731a0f9d3cb751: [guestbook/guestbook-ui-deployment.yaml]","time":"2025-05-18T07:53:50Z"}
{"level":"info","msg":"Received push event repo: https://bitbucket.org/anandjoseph/argocd-sample-pvt, revision: change_version_28, touchedHead: false","time":"2025-05-18T07:53:51Z"}

@anandf anandf added this to the v3.1 milestone May 20, 2025
@jblsk
Copy link

jblsk commented Jul 29, 2025

Unfortunately, this fix is only for Bitbucket Cloud, not for Bitbucket DataCentre (DC, Server).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

manifest-generate-paths monorepo support for Bitbucket
4 participants