feat: add HTTP request/response tracing support #9125

knqyf263 · 2025-07-02T11:01:14Z

Description

This PR adds a new --trace-http flag to enable HTTP request/response tracing for debugging purposes. It also renames the existing --trace flag to --trace-rego for clarity.

Key features:

Renamed --trace to --trace-rego (with deprecated alias for backward compatibility)
Added --trace-http flag for HTTP request/response tracing
Enhanced secret detection using Trivy's built-in secret scanner
Comprehensive redaction of sensitive data (auth headers, API keys, tokens, private keys)
Binary content detection and redaction
Colored output for better readability
Internal flag (hidden from help output) for maintainer debugging only

Security Features

The --trace-http flag includes robust security measures:

Advanced Secret Detection: Integrates Trivy's secret scanner to detect and redact private keys, tokens, and other sensitive data
Pattern-based Redaction: Additional regex patterns for JSON, form data, and Bearer tokens
Binary Content Protection: Automatically detects and redacts binary data
Internal Flag: Hidden from normal help output to prevent accidental usage by end users

Usage information

# Enable HTTP tracing for debugging (maintainer only - flag is hidden from help)
$ trivy image --trace-http alpine:latest

# With other debugging options
$ trivy image --trace-http --debug --insecure my-image:tag

# Client/server mode
$ trivy image --server http://localhost:4954 --trace-http alpine:latest

# Works in CI environments for maintainer debugging
$ CI=true trivy image --trace-http alpine:latest

Security Note: The --trace-http flag is intended for maintainer debugging only. It is hidden from normal help output (Internal: true) and should be used with caution as it may expose sensitive information in trace output despite redaction efforts.

Before and After

Before

When debugging network issues or authentication problems, users had no visibility into HTTP requests/responses:

$ trivy image alpine:latest
2025-07-02T10:00:00.000Z	FATAL	image scan error: scan error: unable to initialize a scanner: unable to initialize an image scanner: GET https://registry-1.docker.io/v2/: UNAUTHORIZED

After

With --trace-http, maintainers can see detailed HTTP traffic with sensitive data properly redacted:

$ trivy image --trace-http alpine:latest

--- HTTP REQUEST ---
GET /v2/library/alpine/manifests/latest HTTP/1.1
Host: registry-1.docker.io
User-Agent: trivy/dev
Accept: application/vnd.docker.distribution.manifest.v2+json
Authorization: <redacted>

--- HTTP RESPONSE ---
HTTP/1.1 200 OK
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Docker-Content-Digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.distribution.container.image.v1+json",
    "size": 1472,
    "digest": "sha256:324bc02ae1231fd9255658c128086395d3fa0aedd5a41ab6b034fd649d1a9260"
  }
}

alpine (alpine 3.20.3)
======================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

Secret Redaction Example

When processing requests with sensitive data, secrets are automatically detected and redacted:

--- HTTP REQUEST ---
POST /api/keys HTTP/1.1
Host: api.example.com
Content-Type: text/plain

-----BEGIN RSA PRIVATE KEY-----
<redacted>
<redacted>
<redacted>
-----END RSA PRIVATE KEY-----
Normal text here

Also, the existing --trace flag has been renamed to --trace-rego for clarity:

# Before
$ trivy config --trace configs/

# After (both work, --trace is deprecated)
$ trivy config --trace-rego configs/
$ trivy config --trace configs/  # deprecated alias

Implementation Details

Secret Scanner Integration: Uses Trivy's built-in secret scanner for advanced detection
Memory Efficient: Optimized regex compilation and byte operations
Internal Flag: Internal: true hides the flag from help output

Related issues

Discussed in Enhance `--trace` flag to support HTTP request/response tracing alongside Rego policy tracing #9093

Checklist

I've read the guidelines for contributing to this repository.
I've followed the conventions in the PR title.
I've added tests that prove my fix is effective or that my feature works.
I've updated the documentation with the relevant information (if needed).
I've added usage information (if the PR introduces new options)
I've included a "before" and "after" example to the description (if the PR is a user interface change).

- Add --trace-http flag for HTTP tracing with security warnings - Rename --trace to --trace-rego (keeping --trace as deprecated alias) - Implement comprehensive HTTP trace transport with sensitive data redaction - Add support for OCI and Docker manifest content types - Include CI environment protection for --trace-http flag - Add comprehensive test coverage for trace functionality

- Add WithWriter functional option to NewTraceTransport for configurable output - Implement comprehensive color support for HTTP methods, URLs, headers, and status codes - Add complete test coverage with exact output matching - Improve test infrastructure with RequestRecorder using functional options pattern - Normalize CRLF to LF in tests for cross-platform compatibility - Set consistent User-Agent in tests to avoid Go version dependency - Replace partial matching tests with complete output verification

- Add --trace-http flag documentation with security warning - Update --trace-rego flag documentation (renamed from --trace) - Regenerate CLI reference documentation across all commands

- Update debugging documentation to use --trace-rego instead of deprecated --trace - Reflect the flag rename in misconfiguration debugging examples

- Add HTTP request/response tracing section to troubleshooting guide - Include security warning about potential sensitive data exposure - Provide usage examples for debugging network issues - Clarify that Trivy attempts to redact sensitive information but exposure may still occur

The order of transport wrappers was incorrect, causing User-Agent headers to be set after the trace transport had already logged the request. Fixed by applying trace transport first, then user agent transport.

The --trace-http flag value can be included in telemetry data as it only indicates whether HTTP tracing is enabled or not.

# Conflicts: # docs/docs/advanced/telemetry-flags.md # pkg/flag/rego_flags.go

Update CLI documentation files to reflect recent flag changes.

DmitriyLewen

LGTM
left 1 suggestion.

DmitriyLewen · 2025-07-03T06:14:45Z

pkg/x/http/trace.go

+	})
+
+	// Handle private keys
+	privateKeyPattern := regexp.MustCompile(`-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----[\s\S]*?-----END (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----`)


We can use secret scanner here (we can add this (if required) in another PR).
wdyt?

Added in fa773aa

pkg/x/http/useragent_test.go

Replace example.com with example.invalid to use an invalid TLD as specified in RFC 6761. This makes it clearer that no external network communication should occur during tests.

Integrate Trivy's built-in secret scanner to improve secret detection and redaction capabilities in HTTP request/response tracing. Key changes: - Add secret scanner to traceTransport struct - Enhance redactBody function to use scanner findings - Replace asterisk patterns with colored redacted text - Add comprehensive test for private key detection - Optimize regex compilation for better performance This enhancement provides more accurate secret detection beyond basic pattern matching, leveraging Trivy's advanced secret scanning rules for better security in HTTP trace output.

Mark the --trace-http flag as internal to hide it from general users while keeping it available for maintainer debugging purposes. Changes: - Add Internal: true to TraceHTTPFlag to hide from help output - Remove CI environment check as flag is now hidden - Remove unused imports (strconv, xerrors) - Update CLI documentation to remove trace-http flag references This ensures the dangerous flag is only accessible to maintainers who explicitly know about it, reducing security risks from accidental usage by end users.

pkg/x/http/trace.go

pkg/x/http/trace_test.go

simar7

lgtm, left one comment.

DmitriyLewen

LGTM

pkg/x/http/trace.go

Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [mirror.gcr.io/aquasec/trivy](https://www.aquasec.com/products/trivy/) ([source](https://github.com/aquasecurity/trivy)) | minor | `0.64.1` -> `0.65.0` | --- ### Release Notes <details> <summary>aquasecurity/trivy (mirror.gcr.io/aquasec/trivy)</summary> ### [`v0.65.0`](https://github.com/aquasecurity/trivy/blob/HEAD/CHANGELOG.md#0650-2025-07-30) [Compare Source](aquasecurity/trivy@v0.64.1...v0.65.0) ##### Features - add graceful shutdown with signal handling ([#9242](aquasecurity/trivy#9242)) ([2c05882](aquasecurity/trivy@2c05882)) - add HTTP request/response tracing support ([#9125](aquasecurity/trivy#9125)) ([aa5b32a](aquasecurity/trivy@aa5b32a)) - **alma:** add AlmaLinux 10 support ([#9207](aquasecurity/trivy#9207)) ([861d51e](aquasecurity/trivy@861d51e)) - **flag:** add schema validation for `--server` flag ([#9270](aquasecurity/trivy#9270)) ([ed4640e](aquasecurity/trivy@ed4640e)) - **image:** add Docker context resolution ([#9166](aquasecurity/trivy#9166)) ([99cd4e7](aquasecurity/trivy@99cd4e7)) - **license:** observe pkg types option in license scanner ([#9091](aquasecurity/trivy#9091)) ([d44af8c](aquasecurity/trivy@d44af8c)) - **misconf:** add private ip google access attribute to subnetwork ([#9199](aquasecurity/trivy#9199)) ([263845c](aquasecurity/trivy@263845c)) - **misconf:** added logging and versioning to the gcp storage bucket ([#9226](aquasecurity/trivy#9226)) ([110f80e](aquasecurity/trivy@110f80e)) - **repo:** add git repository metadata to reports ([#9252](aquasecurity/trivy#9252)) ([f4b2cf1](aquasecurity/trivy@f4b2cf1)) - **report:** add CVSS vectors in sarif report ([#9157](aquasecurity/trivy#9157)) ([60723e6](aquasecurity/trivy@60723e6)) - **sbom:** add SHA-512 hash support for CycloneDX SBOM ([#9126](aquasecurity/trivy#9126)) ([12d6706](aquasecurity/trivy@12d6706)) ##### Bug Fixes - **alma:** parse epochs from rpmqa file ([#9101](aquasecurity/trivy#9101)) ([82db2fc](aquasecurity/trivy@82db2fc)) - also check `filepath` when removing duplicate packages ([#9142](aquasecurity/trivy#9142)) ([4d10a81](aquasecurity/trivy@4d10a81)) - **aws:** update amazon linux 2 EOL date ([#9176](aquasecurity/trivy#9176)) ([0ecfed6](aquasecurity/trivy@0ecfed6)) - **cli:** Add more non-sensitive flags to telemetry ([#9110](aquasecurity/trivy#9110)) ([7041a39](aquasecurity/trivy@7041a39)) - **cli:** ensure correct command is picked by telemetry ([#9260](aquasecurity/trivy#9260)) ([b4ad00f](aquasecurity/trivy@b4ad00f)) - **cli:** panic: attempt to get os.Args\[1] when len(os.Args) < 2 ([#9206](aquasecurity/trivy#9206)) ([adfa879](aquasecurity/trivy@adfa879)) - **license:** add missed `GFDL-NIV-1.1` and `GFDL-NIV-1.2` into Trivy mapping ([#9116](aquasecurity/trivy#9116)) ([a692f29](aquasecurity/trivy@a692f29)) - **license:** handle WITH operator for `LaxSplitLicenses` ([#9232](aquasecurity/trivy#9232)) ([b4193d0](aquasecurity/trivy@b4193d0)) - migrate from `*.list` to `*.md5sums` files for `dpkg` ([#9131](aquasecurity/trivy#9131)) ([f224de3](aquasecurity/trivy@f224de3)) - **misconf:** correctly adapt azure storage account ([#9138](aquasecurity/trivy#9138)) ([51aa022](aquasecurity/trivy@51aa022)) - **misconf:** correctly parse empty port ranges in google\_compute\_firewall ([#9237](aquasecurity/trivy#9237)) ([77bab7b](aquasecurity/trivy@77bab7b)) - **misconf:** fix log bucket in schema ([#9235](aquasecurity/trivy#9235)) ([7ebc129](aquasecurity/trivy@7ebc129)) - **misconf:** skip rewriting expr if attr is nil ([#9113](aquasecurity/trivy#9113)) ([42ccd3d](aquasecurity/trivy@42ccd3d)) - **nodejs:** don't use prerelease logic for compare npm constraints ([#9208](aquasecurity/trivy#9208)) ([fe96436](aquasecurity/trivy@fe96436)) - prevent graceful shutdown message on normal exit ([#9244](aquasecurity/trivy#9244)) ([6095984](aquasecurity/trivy@6095984)) - **rootio:** check full version to detect `root.io` packages ([#9117](aquasecurity/trivy#9117)) ([c2ddd44](aquasecurity/trivy@c2ddd44)) - **rootio:** fix severity selection ([#9181](aquasecurity/trivy#9181)) ([6fafbeb](aquasecurity/trivy@6fafbeb)) - **sbom:** merge in-graph and out-of-graph OS packages in scan results ([#9194](aquasecurity/trivy#9194)) ([aa944cc](aquasecurity/trivy@aa944cc)) - **sbom:** use correct field for licenses in CycloneDX reports ([#9057](aquasecurity/trivy#9057)) ([143da88](aquasecurity/trivy@143da88)) - **secret:** add UTF-8 validation in secret scanner to prevent protobuf marshalling errors ([#9253](aquasecurity/trivy#9253)) ([54832a7](aquasecurity/trivy@54832a7)) - **secret:** fix line numbers for multiple-line secrets ([#9104](aquasecurity/trivy#9104)) ([e579746](aquasecurity/trivy@e579746)) - **server:** add HTTP transport setup to server mode ([#9217](aquasecurity/trivy#9217)) ([1163b04](aquasecurity/trivy@1163b04)) - supporting .egg-info/METADATA in python.Packaging analyzer ([#9151](aquasecurity/trivy#9151)) ([e306e2d](aquasecurity/trivy@e306e2d)) - **terraform:** `for_each` on a map returns a resource for every key ([#9156](aquasecurity/trivy#9156)) ([153318f](aquasecurity/trivy@153318f)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1073 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>

Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>

knqyf263 added 12 commits July 1, 2025 17:49

docs: update CLI documentation for HTTP trace flags

2c30efc

- Add --trace-http flag documentation with security warning - Update --trace-rego flag documentation (renamed from --trace) - Regenerate CLI reference documentation across all commands

docs: update --trace flag references to --trace-rego

75345d6

- Update debugging documentation to use --trace-rego instead of deprecated --trace - Reflect the flag rename in misconfiguration debugging examples

fix: ensure User-Agent is captured in HTTP trace output

1291c02

The order of transport wrappers was incorrect, causing User-Agent headers to be set after the trace transport had already logged the request. Fixed by applying trace transport first, then user agent transport.

feat: mark --trace-http as telemetry safe

8233fde

The --trace-http flag value can be included in telemetry data as it only indicates whether HTTP tracing is enabled or not.

Merge remote-tracking branch 'upstream/main' into feat/http-trace

1ad277c

# Conflicts: # docs/docs/advanced/telemetry-flags.md # pkg/flag/rego_flags.go

fix: remove unnecessary ConfigName from deprecated trace alias

5a93ddd

fix: resolve lint issue in trace.go

fa7170c

fix: remove trailing whitespace in transport.go

193c99c

docs: update CLI documentation

1473565

Update CLI documentation files to reflect recent flag changes.

knqyf263 requested a review from DmitriyLewen July 3, 2025 05:36

DmitriyLewen reviewed Jul 3, 2025

View reviewed changes

nikpivkin reviewed Jul 3, 2025

View reviewed changes

pkg/x/http/useragent_test.go Show resolved Hide resolved

knqyf263 added 3 commits July 3, 2025 13:24

test: use invalid TLD in user agent test

76dd2e5

Replace example.com with example.invalid to use an invalid TLD as specified in RFC 6761. This makes it clearer that no external network communication should occur during tests.

knqyf263 marked this pull request as ready for review July 9, 2025 14:16

knqyf263 requested a review from simar7 as a code owner July 9, 2025 14:16

simar7 reviewed Jul 10, 2025

View reviewed changes

pkg/x/http/trace.go Show resolved Hide resolved

simar7 reviewed Jul 10, 2025

View reviewed changes

pkg/x/http/trace_test.go Show resolved Hide resolved

simar7 self-requested a review July 10, 2025 04:35

simar7 approved these changes Jul 10, 2025

View reviewed changes

nikpivkin approved these changes Jul 10, 2025

View reviewed changes

DmitriyLewen approved these changes Jul 10, 2025

View reviewed changes

pkg/x/http/trace.go Outdated Show resolved Hide resolved

Update pkg/x/http/trace.go

3f6db4e

Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>

knqyf263 enabled auto-merge July 10, 2025 06:27

knqyf263 added this pull request to the merge queue Jul 10, 2025

Merged via the queue into aquasecurity:main with commit aa5b32a Jul 10, 2025
17 checks passed

knqyf263 deleted the feat/http-trace branch July 10, 2025 07:06

aqua-bot mentioned this pull request Jul 10, 2025

release: v0.65.0 [main] #9108

Merged

yutatokoi pushed a commit to yutatokoi/trivy that referenced this pull request Aug 12, 2025

feat: add HTTP request/response tracing support (aquasecurity#9125)

145856a

Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add HTTP request/response tracing support #9125

feat: add HTTP request/response tracing support #9125

Uh oh!

knqyf263 commented Jul 2, 2025 •

edited

Loading

Uh oh!

DmitriyLewen left a comment

Uh oh!

DmitriyLewen Jul 3, 2025 •

edited

Loading

Uh oh!

knqyf263 Jul 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simar7 left a comment

Uh oh!

DmitriyLewen left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat: add HTTP request/response tracing support #9125

feat: add HTTP request/response tracing support #9125

Uh oh!

Conversation

knqyf263 commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Security Features

Usage information

Before and After

Before

After

Secret Redaction Example

Implementation Details

Related issues

Checklist

Uh oh!

DmitriyLewen left a comment

Choose a reason for hiding this comment

Uh oh!

DmitriyLewen Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

knqyf263 Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simar7 left a comment

Choose a reason for hiding this comment

Uh oh!

DmitriyLewen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

knqyf263 commented Jul 2, 2025 •

edited

Loading

DmitriyLewen Jul 3, 2025 •

edited

Loading