Changelog

• 4fb4382 cant count
• b1c9c7e Composite rules (#1905)
• 72977e4 feat: add Anthropic API key detection (#1910)
• 7b02c98 fix(git): handle port (#1912)
• 2a7bcff dont prematurely calculate fragment newlines (#1909)
• bd79c3e feat(allowlist): promote optimizations (#1908)
• 7fb4eda Fix: CVEs on go and go crypto (#1868)
• a044b81 feat: add artifactory reference token and api key detection (#1906)
• bf380d4 silly
• f487f85 Update gitleaks.yml
• 958f55a add just like that, no leaks

Optimizations

#1909 waits to find newlines until a match. This ends up saving a boat load of time since before we were finding newlines for every fragment regardless if a rule matched or not.
#1908 promoted @rgmz excellent stopword optimization

Composite Rules (Multi-part or required Rules) #1905

In v8.28.0 Gitleaks introduced composite rules, which are made up of a single "primary" rule and one or more auxiliary or required rules. To create a composite rule, add a [[rules.required]] table to the primary rule specifying an id and optionally withinLines and/or withinColumns proximity constraints. A fragment is a chunk of content that Gitleaks processes at once (typically a file, part of a file, or git diff), and proximity matching instructs the primary rule to only report a finding if the auxiliary required rules also find matches within the specified area of the fragment.

Proximity matching: Using the withinLines and withinColumns fields instructs the primary rule to only report a finding if the auxiliary required rules also find matches within the specified proximity. You can set:

• withinLines: N - required findings must be within N lines (vertically)
• withinColumns: N - required findings must be within N characters (horizontally)
• Both - creates a rectangular search area (both constraints must be satisfied)
• Neither - fragment-level matching (required findings can be anywhere in the same fragment)

Here are diagrams illustrating each proximity behavior:

p = primary captured secret
a = auxiliary (required) captured secret
fragment = section of data gitleaks is looking at


    *Fragment-level proximity*               
    Any required finding in the fragment
          ┌────────┐                       
   ┌──────┤fragment├─────┐                 
   │      └──────┬─┤     │ ┌───────┐       
   │             │a│◀────┼─│✓ MATCH│       
   │          ┌─┐└─┘     │ └───────┘       
   │┌─┐       │p│        │                 
   ││a│    ┌─┐└─┘        │ ┌───────┐       
   │└─┘    │a│◀──────────┼─│✓ MATCH│       
   └─▲─────┴─┴───────────┘ └───────┘       
     │    ┌───────┐                        
     └────│✓ MATCH│                        
          └───────┘                        
                                           
                                           
   *Column bounded proximity*
   `withinColumns = 3`                    
          ┌────────┐                       
   ┌────┬─┤fragment├─┬───┐                 
   │      └──────┬─┤     │ ┌───────────┐   
   │    │        │a│◀┼───┼─│+1C ✓ MATCH│   
   │          ┌─┐└─┘     │ └───────────┘   
   │┌─┐ │     │p│    │   │                 
┌──▶│a│  ┌─┐  └─┘        │ ┌───────────┐   
│  │└─┘ ││a│◀────────┼───┼─│-2C ✓ MATCH│   
│  │       ┘             │ └───────────┘   
│  └── -3C ───0C─── +3C ─┘                 
│  ┌─────────┐                             
│  │ -4C ✗ NO│                             
└──│  MATCH  │                             
   └─────────┘                             
                                           
                                           
   *Line bounded proximity*
   `withinLines = 4`                      
         ┌────────┐                        
   ┌─────┤fragment├─────┐                  
  +4L─ ─ ┴────────┘─ ─ ─│                  
   │                    │                  
   │              ┌─┐   │ ┌────────────┐   
   │         ┌─┐  │a│◀──┼─│+1L ✓ MATCH │   
   0L  ┌─┐   │p│  └─┘   │ ├────────────┤   
   │   │a│◀──┴─┴────────┼─│-1L ✓ MATCH │   
   │   └─┘              │ └────────────┘   
   │                    │ ┌─────────┐      
  -4L─ ─ ─ ─ ─ ─ ─ ─┌─┐─│ │-5L ✗ NO │      
   │                │a│◀┼─│  MATCH  │      
   └────────────────┴─┴─┘ └─────────┘      
                                           
                                           
   *Line and column bounded proximity*
   `withinLines = 4`                      
   `withinColumns = 3`                    
         ┌────────┐                        
   ┌─────┤fragment├─────┐                  
  +4L   ┌└────────┴ ┐   │                  
   │            ┌─┐     │ ┌───────────────┐
   │    │       │a│◀┼───┼─│+2L/+1C ✓ MATCH│
   │         ┌─┐└─┘     │ └───────────────┘
   0L   │    │p│    │   │                  
   │         └─┘        │                  
   │    │           │   │ ┌────────────┐   
  -4L    ─ ─ ─ ─ ─ ─┌─┐ │ │-5L/+3C ✗ NO│   
   │                │a│◀┼─│   MATCH    │   
   └───-3C────0L───+3C┴─┘ └────────────┘   

Changelog

• c7acf33 Merge branch 'master' of github.com:gitleaks/gitleaks
• 9faaa4a Add experimental allowlist optimizations (#1731)
• 79068b3 Detect Notion Public API Keys #1889 (#1890)

Changelog

• 80468ef Merge branch 'master' of github.com:gitleaks/gitleaks
• ef82237 fix(atlassian): reduce false-positives for v1 pattern (#1892)
• 2463f11 Fix log suppresion issue (#1887)
• 6f251ee Added Heroku API Key New Version (#1883)
• 20f9a1d Add Platform Bitbucket (#1886)
• 722ce82 Add Platform Gitea (#1884)
• 79780b8 Merge branch 'master' of github.com:gitleaks/gitleaks
• c5683ca prevent default warn message when max-archive-depth not set (#1881)
• 0357c3c prevent default warn message when max-archive-depth not set

Changelog

• 782f310 Archive support (#1872)
• 489d13c Update README.md
• d29ee55 Reduce aws-access-token false positives (#1876)
• 611db65 Set pass_filenames to false for Docker hook (#1850)
• 0589ae0 unicode decoding (#1854)
• 82f7e32 Diagnostics (#1856)
• f97a9ee chore: include decoder in debug log (#1853)

Got another @bplaxco release. Cheers!

Archive Scanning

Sometimes secrets are packaged within archive files like zip files or tarballs,
making them difficult to discover. Now you can tell gitleaks to automatically
extract and scan the contents of archives. The flag --max-archive-depth
enables this feature for both dir and git scan types. The default value of
"0" means this feature is disabled by default.

Recursive scanning is supported since archives can also contain other archives.
The --max-archive-depth flag sets the recursion limit. Recursion stops when
there are no new archives to extract, so setting a very high max depth just
sets the potential to go that deep. It will only go as deep as it needs to.

The findings for secrets located within an archive will include the path to the
file inside the archive. Inner paths are separated with !.

Example finding (shortened for brevity):

Finding:     DB_PASSWORD=8ae31cacf141669ddfb5da
...
File:        testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod
Line:        4
Commit:      6e6ee6596d337bb656496425fb98644eb62b4a82
...
Fingerprint: 6e6ee6596d337bb656496425fb98644eb62b4a82:testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod:generic-api-key:4
Link:        https://github.com/leaktk/gitleaks/blob/6e6ee6596d337bb656496425fb98644eb62b4a82/testdata/archives/nested.tar.gz

This means a secret was detected on line 4 of files/.env.prod. which is in
archives/files.tar which is in testdata/archives/nested.tar.gz.

Currently supported formats:

The compression
and archive
formats supported by mholt's archives package
are supported.

Changelog

• 78eebac Percent/URL Decoding Support (#1831)
• 6f967ca fix(kubernetes): remove slow element from pat (#1848)
• 88f56d3 feat: identify slow file (#1479)
• 9609928 rm 1password detect test since we test it in cfg gen
• 23cb69f feat(rules): Add 1Password secret key detection (#1834)

Calling this one @bplaxco's release as he introduced a really clever method for mixed decoding without sacrificing too much performance. As I stated in his PR, I think he's either a wizard or some time traveling AI. Dude is wicked smaht

Anyways, Gitleaks now supports the following decoders: hex, percent(url enconding), and b64. It's relatively straight forward to add a new decoder so if you're motivated, community contributions are welcomed!

Here's an example:

~/code/gitleaks-org/gitleaks (master) cat decode.txt
text below
aGVsbG8sIHdvcmxkIQ%3D%3D%0A
text above
~/code/gitleaks-org/gitleaks (master) ./gitleaks dir decode.txt --max-decode-depth=2 --log-level=debug

    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks

4:08PM DBG using stdlib regex engine
4:08PM DBG unable to load gitleaks config from decode.txt/.gitleaks.toml since --source=decode.txt is a file, using default config
4:08PM DBG found .gitleaksignore file: .gitleaksignore
4:08PM DBG segment found: original=[29,38] pos=[29,38]: "%3D%3D%0A" -> "==\n"
4:08PM DBG segment found: original=[11,38] pos=[11,31]: "aGVsbG8sIHdvcmxkIQ==" -> "hello, world!"
4:08PM INF scanned ~50 bytes (50 bytes) in 1.5ms
4:08PM INF no leaks found

Changelog

• d1c7759 fix(detect): test all allowlists (#1845)

Big thanks @rgmz

Changelog

• 4451b45 feat(config): define multiple global allowlists (#1777) (cause for the minor bump change)
• 7fb21a4 feat(rules): Add Perplexity AI API key detection (#1825)
• f6193bc feat(gcp): increase rule entropy (#1840)
• 9bc7257 Adding clickhouse scanner (#1826)
• b6cc71a fix(baseline): work with --redact (#1741)
• cfdeb0d feat(rule): validate & sort rule when generating (#1817)

Changelog

• 107a418 Add support for GitLab Runner Tokens (Routable) (#1820)
• 7fac002 bump repo version in pre-commit example (#1815)
• 4b54104 Fix currentLine out of bounds error (#1810)
• af7d5bc add support for Azure DevOps platform in SCM detection and link (#1807)
• 3e8cd2d Add MaxMind license key rule (#1771)
• ddcc753 implement new openai regex pattern (#1780)
• 9708e65 A first attempt adding hooks.slack.com/triggers/ (#1792)
• 198e410 feat(generic): tweak false-positives (#1803)
• e273a97 chore: tweak logging and readme for GITLEAKS_CONFIG_TOML feature (#1802)
• a503b58 feat: add option to set config from env var with toml content (#1662)

What's Changed

• Fix platform flag being ignored with gitleaks detect by @rgmz in #1765
• Make AddFinding public by @bplaxco in #1767
• FIX upgrade x/crypto to 0.31.0 to get rid of CVE-2024-45337 by @cgoessen in #1768
• Upgrade rs/zerolog, spf13/cobra, and spf13/viper by @rgmz in #1769
• Infer report-format from report-path extension if no value is provided by @rgmz in #1776
• generic-api-key: ignore csrf-tokens by @rgmz in #1779
• Prevent Yocto/BitBake false positives with generic-api-key rule by @Okeanos in #1783
• Fix decoded line allowlist by @zricethezav in #1788
• Readme badge revisions by @jessp01 in #1744
• feat(regexp): use standard regexp by default, make go-re2 opt-in by @twpayne in #1798
• gore2 release tags by @zricethezav in #1801

New Contributors

• @cgoessen made their first contribution in #1768
• @Okeanos made their first contribution in #1783
• @jessp01 made their first contribution in #1744
• @twpayne made their first contribution in #1798

Full Changelog: v8.24.0...v8.24.2

Changelog

• c2afd56 Make paths and fingerprints platform-agnostic (#1622)
• 818e32f Add Sonar rule (#1756)
• 3fa5a3a Minor false positive improvements (#1758)
• 2020e6a Add support for streaming DetectReader (#1760)
• 9122a2d chore: Update github.com/wasilibs/go-re2 to v1.9.0 (#1763)
• 398d0c4 docs: describe extended rules take precedence over base rules (#1563)
• ae26eff feat(git): disable link generation (#1748)
• c6424a6 added sourcegraph token rule (#1736)
• 6411402 feat(config): add rule for .p12 files (#1738)
• d71d95d add deno.lock to default exclusions (#1740)