Skip to content

Conversation

rgmz
Copy link
Contributor

@rgmz rgmz commented Nov 2, 2024

Description:

This fixes #1565.

Note: the current 'fix' is technically a breaking change for path/paths regular expressions as mentioned here. I don't know if this is acceptable, or what the best way to handle this would be. (Test both styles of paths if runtime.GOOS == 'Windows'?)

TODO

  • Normalize finding paths
  • Normalize fingerprint paths
  • Normalize gitleaksignore paths
  • Add tests
  • Test .gitleaksignore on Windows
  • Test rule.path and rule.allowlist.paths regexes on Windows

Checklist:

  • Does your PR pass tests?
  • Have you written new tests for your changes?
  • Have you lint your code locally prior to submission?

@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch 3 times, most recently from 697b436 to 59a5e12 Compare November 2, 2024 23:21
@rgmz rgmz marked this pull request as ready for review November 3, 2024 00:03
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from 59a5e12 to b074b3f Compare November 3, 2024 00:07
@Ben-grmbl
Copy link
Contributor

I ran this on windows and as far as I could tell, it works well.

The unit test TestFromFiles now also passes on windows.

(some other unit tests still fail on windows for reasons entirely unrelated to this pull request, e.g. symlinks and deletion of temp files)

@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from b074b3f to 498034f Compare December 5, 2024 23:30
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch 23 times, most recently from 082fc96 to 4053c9a Compare January 18, 2025 14:40
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch 4 times, most recently from 710104b to 6fbc170 Compare January 18, 2025 15:45
@rgmz rgmz marked this pull request as draft January 18, 2025 15:46
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch 4 times, most recently from 4d1dac7 to 9d3555e Compare January 19, 2025 01:13
@rgmz rgmz marked this pull request as ready for review January 19, 2025 01:16
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from 9d3555e to c206912 Compare January 19, 2025 01:17
@rgmz
Copy link
Contributor Author

rgmz commented Jan 19, 2025

@zricethezav it's a bit awkward but I believe this is now fully backwards-compatible.

  • Any new findings or fingerprints will be normalized to Unix path separators
  • On Windows, both rule.path and allowlist.paths will test the normalized separators and the original separators
  • Paths in .gitleaksignore are also normalized to / on start

@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch 2 times, most recently from a574ad7 to 260f42d Compare January 20, 2025 15:48
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from 260f42d to ef7216f Compare January 29, 2025 02:39
@rgmz rgmz mentioned this pull request Jan 29, 2025
3 tasks
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from ef7216f to 5c71c5b Compare February 20, 2025 00:51
@@ -49,6 +49,7 @@ func TestWriteSarif(t *testing.T) {
t.Run(test.cfgName, func(t *testing.T) {
tmpfile, err := os.Create(filepath.Join(t.TempDir(), test.testReportName+".json"))
require.NoError(t, err)
defer tmpfile.Close()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops, thanks for adding this in

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surprisingly, this didn't seem to cause trouble on *nix. It was only a consistent issue on Windows.

this change also fixes, or skips, windows failures
@rgmz rgmz force-pushed the feat/normalize-fingerprint-path branch from 5c71c5b to 0a4ef20 Compare February 20, 2025 01:19
@zricethezav
Copy link
Collaborator

great work!

@zricethezav zricethezav merged commit c2afd56 into gitleaks:master Feb 20, 2025
2 checks passed
@rgmz rgmz deleted the feat/normalize-fingerprint-path branch February 20, 2025 01:51
alayne222 pushed a commit to alayne222/gitleaks that referenced this pull request May 28, 2025
* test: windows

* fix: make paths platform-agnostic

this change also fixes, or skips, windows failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Paths and Fingerprints are platform specific and not portable
3 participants