dac strip io from OpenAI #6156

jsimpher · 2025-07-23T15:41:19Z

What does this PR do?

OpenAI integration: removes I/O and request/response attribute tags from the APM spans for openAI LLM traced completion/chat/response requests and responses, which is duplicated in LLM Observability.

Motivation

Limit potentially sensitive or restricted prompt/response data around LLM's to only one track to simplify access controls.

Plugin Checklist

Additional Notes

github-actions · 2025-07-23T15:41:52Z

Overall package size

Self size: 11.11 MB
Deduped: 110.69 MB
No deduping: 111.08 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.0.1 | 20.3 MB | 20.3 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.14 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

codecov · 2025-07-23T15:42:08Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.80%. Comparing base (624e155) to head (5692e27).
Report is 4 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #6156   +/-   ##
=======================================
  Coverage   82.80%   82.80%           
=======================================
  Files         476      476           
  Lines       19661    19661           
=======================================
  Hits        16281    16281           
  Misses       3380     3380

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2025-07-23T15:48:54Z

Benchmarks

Benchmark execution time: 2025-07-24 19:51:27

Comparing candidate commit 5692e27 in PR branch jsimpher/dac-strip-io-from-openai with baseline commit 624e155 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1270 metrics, 53 unstable metrics.

datadog-datadog-prod-us1 · 2025-07-23T15:54:15Z

⚠️ Tests

⚠️ Warnings

❄️ 5 New flaky tests detected

cypress@10.2.0 esm can report code coverage if it is available from integration-tests/cypress/cypress.spec.js (Datadog)
Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/dd-trace-js/dd-trace-js/integration-tests/cypress/cypress.spec.js)
cypress@10.2.0 esm can run and report tests from integration-tests/cypress/cypress.spec.js (Datadog)
Cannot destructure property 'content' of 'testSessionEvent' as it is undefined.
cypress@10.2.0 esm catches errors in hooks from integration-tests/cypress/cypress.spec.js (Datadog)
Cannot read properties of undefined (reading 'content')
View all

ℹ️ Info

🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 5692e27 | Was this helpful? Give us feedback!}

sabrenner

nice! there are a few places we're still tagging stuff like filenames, permissions, prompts, other sensitive data, etc

createEditRequestExtraction → openai.request.instruction
commonFileRequestExtraction → openai.request.filename
commonLookupFineTuneRequestExtraction → openai.request.fine_tune_id
commonFineTuneResponseExtraction, createFineTuneRequestExtraction → records various input parameters
deleteFileResponseExtraction → openai.response.id
createRetrieveFileResponseExtraction → openai.response.filename

also, did we want to leave in all of the openai.organization.* tags as well? tbh for most of these i don't have a strong opinion at all just wanna make sure it's in line with DACs goal and what we did for Python as well.

otherwise overall great cleanup once again! 😎

packages/datadog-plugin-openai/src/tracing.js

sabrenner

lgtm! 🚢 i think you cleaned it up well, i'll have to come back to this separately to deprecate sending logs/metrics eventually which will hopefully remove even more code & code paths 😅

jsimpher · 2025-07-24T20:43:56Z

/merge

dd-devflow-routing-codex · 2025-07-24T20:44:01Z

View all feedbacks in Devflow UI.

2025-07-24 20:44:00 UTC ℹ️ Start processing command /merge

2025-07-24 20:44:08 UTC ℹ️ MergeQueue: queue is disabled

Added to the queue but the mergequeue is not enabled for now.

2025-07-25 01:47:00 UTC ℹ️ MergeQueue: This merge request was already merged

This pull request was merged directly.

* first run at openai * factored out param tagging * remove the rest * wasted case * image tagging * remove moderation tagging * org tag * remove model check * uncalled logger * remove a few more tags * fine tuning removed * cleanup * fine tune ids * remove org name * update tests * remove instruction

jsimpher changed the title ~~Jsimpher/dac strip io from OpenAI~~ dac strip io from OpenAI Jul 23, 2025

jsimpher added the semver-patch label Jul 23, 2025

first run at openai

6e1d8d6

jsimpher force-pushed the jsimpher/dac-strip-io-from-openai branch from 6e6a2e9 to 6e1d8d6 Compare July 23, 2025 16:54

jsimpher and others added 5 commits July 23, 2025 13:19

factored out param tagging

6bac9e3

remove the rest

e1cef29

Merge branch 'master' into jsimpher/dac-strip-io-from-openai

10b4133

Merge branch 'master' into jsimpher/dac-strip-io-from-openai

90ae478

wasted case

4a312c1

jsimpher force-pushed the jsimpher/dac-strip-io-from-openai branch 2 times, most recently from 0b1d055 to 4a312c1 Compare July 24, 2025 14:04

image tagging

d496a4e

jsimpher force-pushed the jsimpher/dac-strip-io-from-openai branch from b923b53 to d496a4e Compare July 24, 2025 14:25

jsimpher added 5 commits July 24, 2025 11:13

remove moderation tagging

9f27dc2

org tag

371075c

remove model check

f325d2e

uncalled logger

0422757

remove a few more tags

c1b5261

jsimpher marked this pull request as ready for review July 24, 2025 16:15

jsimpher requested a review from a team as a code owner July 24, 2025 16:15

sabrenner reviewed Jul 24, 2025

View reviewed changes

packages/datadog-plugin-openai/src/tracing.js Show resolved Hide resolved

packages/datadog-plugin-openai/src/tracing.js Outdated Show resolved Hide resolved

jsimpher added 3 commits July 24, 2025 14:28

fine tuning removed

7536463

cleanup

d81c038

fine tune ids

7ecd325

jsimpher force-pushed the jsimpher/dac-strip-io-from-openai branch from cf5ba32 to 7ecd325 Compare July 24, 2025 18:47

remove org name

e30edfb

jsimpher added 2 commits July 24, 2025 15:22

update tests

70cdc55

remove instruction

5692e27

sabrenner approved these changes Jul 24, 2025

View reviewed changes

dd-devflow bot added the mergequeue-status: queued label Jul 24, 2025

jsimpher merged commit 919a919 into master Jul 25, 2025
728 of 730 checks passed

dd-devflow bot added mergequeue-status: done and removed mergequeue-status: queued labels Jul 25, 2025

jsimpher deleted the jsimpher/dac-strip-io-from-openai branch July 25, 2025 01:46

dd-octo-sts bot mentioned this pull request Jul 25, 2025

v5.61.2 proposal #6170

Closed

dd-octo-sts bot mentioned this pull request Jul 30, 2025

v5.62.0 proposal #6200

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

dac strip io from OpenAI #6156

dac strip io from OpenAI #6156

Uh oh!

jsimpher commented Jul 23, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 23, 2025 •

edited

Loading

Uh oh!

codecov bot commented Jul 23, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jul 23, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Jul 23, 2025 •

edited

Loading

Uh oh!

sabrenner left a comment

Uh oh!

Uh oh!

Uh oh!

sabrenner left a comment

Uh oh!

jsimpher commented Jul 24, 2025

Uh oh!

dd-devflow-routing-codex bot commented Jul 24, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

dac strip io from OpenAI #6156

dac strip io from OpenAI #6156

Uh oh!

Conversation

jsimpher commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Plugin Checklist

Additional Notes

Uh oh!

github-actions bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overall package size

Uh oh!

codecov bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Uh oh!

datadog-datadog-prod-us1 bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

sabrenner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sabrenner left a comment

Choose a reason for hiding this comment

Uh oh!

jsimpher commented Jul 24, 2025

Uh oh!

dd-devflow-routing-codex bot commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jsimpher commented Jul 23, 2025 •

edited

Loading

github-actions bot commented Jul 23, 2025 •

edited

Loading

codecov bot commented Jul 23, 2025 •

edited

Loading

pr-commenter bot commented Jul 23, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Jul 23, 2025 •

edited

Loading

dd-devflow-routing-codex bot commented Jul 24, 2025 •

edited

Loading