Skip to content

Conversation

philipphofmann
Copy link
Member

📜 Description

When having app launch profiling and TTFD enabled, a race condition when reading alwaysWaitForFullDisplay could occur. This is fixed now by reading if TTFD is enabled from the options instead of the UIViewControllerPerformanceTracker.

💡 Motivation and Context

Running the tests with the thread sanitizer enabled surfaced this data race.

==================
WARNING: ThreadSanitizer: data race (pid=47792)
  Read of size 1 at 0x00030aa799f8 by thread T350:
    #0 -[SentryUIViewControllerPerformanceTracker alwaysWaitForFullDisplay] <null> (Sentry:arm64+0x18c298)
    #1 __sentry_sdkInitProfilerTasks_block_invoke <null> (Sentry:arm64+0x12fca8)
    #2 __53-[SentryDispatchQueueWrapper dispatchAsyncWithBlock:]_block_invoke <null> (Sentry:arm64+0x144bc)
    #3 __tsan::invoke_and_release_block(void*) <null> (libclang_rt.tsan_iossim_dynamic.dylib:arm64+0x7f97c)
    #4 _dispatch_client_callout <null> (libdispatch.dylib:arm64+0x1c274)

  Previous write of size 1 at 0x00030aa799f8 by main thread (mutexes: write M0):
    #0 -[SentryUIViewControllerPerformanceTracker initWithTracker:dispatchQueueWrapper:] <null> (Sentry:arm64+0x1860bc)
    #1 -[SentryDependencyContainer uiViewControllerPerformanceTracker] <null> (Sentry:arm64+0x178bc8)
    #2 SentryTests.SentrySDKTests.testReportFullyDisplayed() -> () <null> (SentryTests:arm64+0x783388)
    #3 @objc SentryTests.SentrySDKTests.testReportFullyDisplayed() -> () <null> (SentryTests:arm64+0x783728)
    #4 __invoking___ <null> (CoreFoundation:arm64+0x13a20c)

  Location is heap block of size 80 at 0x00030aa799f0 allocated by main thread:
    #0 calloc <null> (libclang_rt.tsan_iossim_dynamic.dylib:arm64+0x5ed24)
    #1 _malloc_type_calloc_outlined <null> (libsystem_malloc.dylib:arm64+0xf308)
    #2 SentryTests.SentrySDKTests.testReportFullyDisplayed() -> () <null> (SentryTests:arm64+0x783388)
    #3 @objc SentryTests.SentrySDKTests.testReportFullyDisplayed() -> () <null> (SentryTests:arm64+0x783728)
    #4 __invoking___ <null> (CoreFoundation:arm64+0x13a20c)

  Mutex M0 (0x000109f0eb60) created at:
    #0 objc_sync_enter <null> (libclang_rt.tsan_iossim_dynamic.dylib:arm64+0x7d384)
    #1 -[SentryDependencyContainer crashReporter] <null> (Sentry:arm64+0x178094)
    #2 __32-[SentryCrashWrapper systemInfo]_block_invoke <null> (Sentry:arm64+0x4e73c)
    #3 dispatch_once <null> (libclang_rt.tsan_iossim_dynamic.dylib:arm64+0x80988)
    #4 -[SentryCrashWrapper systemInfo] <null> (Sentry:arm64+0x4e660)
    #5 -[SentryCrashWrapper enrichScope:] <null> (Sentry:arm64+0x4ec28)
    #6 -[SentryHub initWithClient:andScope:andCrashWrapper:andDispatchQueue:] <null> (Sentry:arm64+0xaf850)
    #7 -[SentryHub initWithClient:andScope:] <null> (Sentry:arm64+0xaf2cc)
    #8 __30+[SentrySDK startWithOptions:]_block_invoke <null> (Sentry:arm64+0x1427a8)
    #9 -[SentryDispatchQueueWrapper dispatchAsyncOnMainQueue:] <null> (Sentry:arm64+0x14638)
    #10 +[SentrySDK startWithOptions:] <null> (Sentry:arm64+0x1424e0)
    #11 +[SentrySDK startWithConfigureOptions:] <null> (Sentry:arm64+0x142ba8)
    #12 SentryTests.DataSentryTracingIntegrationTests.(Fixture in _F9C28CBE30EAC9BE15107FEF3A455A1A).getSut(testName: Swift.String, isSDKEnabled: Swift.Bool, isEnabled: Swift.Bool) throws -> Foundation.Data <null> (SentryTests:arm64+0xb4f298)
    #13 SentryTests.DataSentryTracingIntegrationTests.testInitContentsOfWithSentryTracing_fileIsIgnored_shouldNotTraceManually() throws -> () <null> (SentryTests:arm64+0xb5f9f4)
    #14 @objc SentryTests.DataSentryTracingIntegrationTests.testInitContentsOfWithSentryTracing_fileIsIgnored_shouldNotTraceManually() throws -> () <null> (SentryTests:arm64+0xb609bc)
    #15 __invoking___ <null> (CoreFoundation:arm64+0x13a20c)

  Thread T350 (tid=446262, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race (/Users/philipp.hofmann/Library/Developer/Xcode/DerivedData/Sentry-bahizrqpvqbrnycjvclkuwoibtgv/Build/Products/Test-iphonesimulator/Sentry.framework/Sentry:arm64+0x18c298) in -[SentryUIViewControllerPerformanceTracker alwaysWaitForFullDisplay]+0x40
==================

The problem is that the SentryPerformanceTrackingIntegration writes alwaysWaitForFullDisplay on the main thread here

SentryUIViewControllerPerformanceTracker *performanceTracker =
[SentryDependencyContainer.sharedInstance uiViewControllerPerformanceTracker];
performanceTracker.alwaysWaitForFullDisplay = options.enableTimeToFullDisplayTracing;

While the SentryProfiler reads the same value on a BG thread here

[SentryDependencyContainer.sharedInstance.dispatchQueueWrapper dispatchAsyncWithBlock:^{
// get the configuration options from the last time the launch config was written; it may be
// different than the new options the SDK was just started with
const auto configDict = sentry_appLaunchProfileConfiguration();
const auto profileIsContinuousV1 =
[configDict[kSentryLaunchProfileConfigKeyContinuousProfiling] boolValue];
const auto profileIsContinuousV2 =
[configDict[kSentryLaunchProfileConfigKeyContinuousProfilingV2] boolValue];
const auto v2LifecycleValue
= configDict[kSentryLaunchProfileConfigKeyContinuousProfilingV2Lifecycle];
const auto v2Lifecycle = (SentryProfileLifecycle)
[configDict[kSentryLaunchProfileConfigKeyContinuousProfilingV2Lifecycle] intValue];
const auto v2LifecycleIsManual = profileIsContinuousV2 && v2LifecycleValue != nil
&& v2Lifecycle == SentryProfileLifecycleManual;
BOOL shouldStopAndTransmitLaunchProfile = YES;
# if SENTRY_HAS_UIKIT
const auto v2LifecycleIsTrace = profileIsContinuousV2 && v2LifecycleValue != nil
&& v2Lifecycle == SentryProfileLifecycleTrace;
const auto profileIsCorrelatedToTrace = !profileIsContinuousV2 || v2LifecycleIsTrace;
SentryUIViewControllerPerformanceTracker *performanceTracker =
[SentryDependencyContainer.sharedInstance uiViewControllerPerformanceTracker];
if (profileIsCorrelatedToTrace && performanceTracker.alwaysWaitForFullDisplay) {
SENTRY_LOG_DEBUG(@"Will wait to stop launch profile correlated to a trace until full "
@"display reported.");
shouldStopAndTransmitLaunchProfile = NO;
}
# endif // SENTRY_HAS_UIKIT

💚 How did you test it?

Running the tests again with the thread sanitizer enabled didn't surface this data race anymore. Ideally, we would write an integration test for this, but that is quite complicated. Enabling the thread sanitizer again in CI will surface such issues again.

📝 Checklist

You have to check all boxes before merging:

  • I added tests to verify the changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • Review from the native team if needed.
  • No breaking change or entry added to the changelog.
  • No breaking change for hybrid SDKs or communicated to hybrid SDKs.

When having app launch profiling and TTFD enabled, a race condition when
reading alwaysWaitForFullDisplay could occur. This is fixed now by
reading if TTFD is enabled from the options instead of the
UIViewControllerPerformanceTracker.
Copy link

codecov bot commented May 27, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.274%. Comparing base (701b301) to head (2df58d6).

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##              main     #5300       +/-   ##
=============================================
+ Coverage   86.153%   86.274%   +0.121%     
=============================================
  Files          401       402        +1     
  Lines        34773     34905      +132     
  Branches     15033     15132       +99     
=============================================
+ Hits         29958     30114      +156     
+ Misses        4774      4746       -28     
- Partials        41        45        +4     
Files with missing lines Coverage Δ
Sources/Sentry/SentryProfiler.mm 90.184% <100.000%> (+1.699%) ⬆️

... and 26 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 701b301...2df58d6. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

github-actions bot commented May 27, 2025

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1217.77 ms 1243.96 ms 26.19 ms
Size 23.75 KiB 867.06 KiB 843.32 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
b714cb9 1224.71 ms 1238.04 ms 13.33 ms
5cfc768 1220.74 ms 1245.06 ms 24.32 ms
4e3915a 1230.02 ms 1258.90 ms 28.88 ms
35c962f 1207.61 ms 1235.90 ms 28.29 ms
8ea5293 1242.70 ms 1262.25 ms 19.55 ms
d38165b 1211.41 ms 1242.49 ms 31.08 ms
8047b99 1226.37 ms 1246.63 ms 20.26 ms
b13e93a 1236.24 ms 1247.33 ms 11.08 ms
55f739c 1226.06 ms 1248.78 ms 22.71 ms
2691350 1224.92 ms 1255.82 ms 30.90 ms

App size

Revision Plain With Sentry Diff
b714cb9 23.75 KiB 858.69 KiB 834.93 KiB
5cfc768 23.75 KiB 850.73 KiB 826.98 KiB
4e3915a 23.75 KiB 858.69 KiB 834.94 KiB
35c962f 23.75 KiB 854.77 KiB 831.02 KiB
8ea5293 23.75 KiB 852.24 KiB 828.49 KiB
d38165b 23.75 KiB 855.37 KiB 831.62 KiB
8047b99 23.75 KiB 855.37 KiB 831.62 KiB
b13e93a 23.75 KiB 855.37 KiB 831.62 KiB
55f739c 23.75 KiB 858.73 KiB 834.98 KiB
2691350 23.75 KiB 850.73 KiB 826.98 KiB

Copy link
Member

@philprime philprime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the detailed PR description explaining the issue.

Copy link
Member

@philprime philprime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the detailed PR description explaining the issue.

@philipphofmann
Copy link
Member Author

All the UI tests are failing. I need to double check if I actually broke something.

Copy link
Member

@armcknight armcknight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this @philipphofmann . We just need to check a few more options for it to be complete.

@@ -31,6 +31,27 @@ extension SentryAppLaunchProfilingTests {
XCTAssertNil(sentry_launchTracer)
}

// TTFD only works when UIKit is available, so we only test this on iOS.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should just change the #if os(iOS) || os(macOS) || targetEnvironment(macCatalyst) to this at the beginning/end of this file. copypasta on my part!

SentryUIViewControllerPerformanceTracker *performanceTracker =
[SentryDependencyContainer.sharedInstance uiViewControllerPerformanceTracker];
if (profileIsCorrelatedToTrace && performanceTracker.alwaysWaitForFullDisplay) {

if (profileIsCorrelatedToTrace && options.enableTimeToFullDisplayTracing) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll also need to check a few other options, like options.enableUIViewControllerTracing, since you could theoretically misconfigure the SDK like:

options.enableUIViewControllerTracing = false
options.enableTimeToFullDisplayTracing = true

Checking SentryUIViewControllerPerformanceTracker.alwaysWaitForFullDisplay would have implicitly included all the option combination validation. I don't know what other options would be applicable, maybe enableAutoPerformanceTracing and tracesSampleRate/tracesSampler?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually why the UI tests are failing now. Since we automatically configure enableUIViewControllerTracing to true unless the launch flag "--disable-time-to-full-display-tracing" is set in SentrySDKWrapper/SentrySDKOverrides, changing this now caused the conditional branch to be taken here where it was not before.

@philipphofmann
Copy link
Member Author

Thanks for the review @armcknight. Due to the hotfix and some unexpected immediate family member care leave yesterday, I was unable to include your feedback yet. As I'm on PTO the whole next week, this PR will lie around for a bit. If this is important, feel free to pick it up and as @philprime for another review. I can also tackle it once I'm back.

@philipphofmann
Copy link
Member Author

philipphofmann commented Jun 26, 2025

Thanks for the review @armcknight. I'm sorry about the hassle. The actual problem is in the test testReportFullyDisplayed. It sets a new TestTimeToDisplayTracker to the SentryDependencyContainer after the SDK starts. As the sentry_sdkInitProfilerTasks runs on a BG thread, it can happen that setting this new TestTimeToDisplayTracker occurs during sentry_sdkInitProfilerTasks. Instead, we must set the TestTimeToDisplayTracker to the SentryDependencyContainer already before starting the SDK. Then the data race is gone. I'm going to open a new PR for this and close this one.

@philipphofmann philipphofmann deleted the fix/race-condition-app-launch-profiling branch July 7, 2025 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants