Skip to content

Conversation

ajpallares
Copy link
Member

@ajpallares ajpallares commented Jul 15, 2025

Motivation

We have some flaky StoreKit integration tests. This PR tries to make them less flaky.

Description

Some unit tests were randomly failing because the receipt posting would happen after the purchase expiration, which would cause the product entitlement become inactive.

This PR changes the SKTestSession's timeRate of some tests from .oneRenewalEveryTwoSeconds to .oneSecondIsOneDay. Because these affected tests simulate the purchase of a product with a 3-day trial, with this change, the first renewal time of the these tests changes from 2 seconds to 3 seconds, which should reduce their flakiness.

@ajpallares
Copy link
Member Author

@RCGitBot please test

Copy link

emerge-tools bot commented Jul 15, 2025

📸 Snapshot Test

705 unchanged

Name Added Removed Modified Renamed Unchanged Errored Approval
RevenueCat
com.revenuecat.PaywallsTester.mac-catalyst-optimized-for-mac
0 0 0 0 235 0 N/A
RevenueCat
com.revenuecat.PaywallsTester.mac-catalyst-scaled-to-match-ipad
0 0 0 0 235 0 N/A
RevenueCat
com.revenuecat.PaywallsTester
0 0 0 0 235 0 N/A

🛸 Powered by Emerge Tools

// 'SKTestSession.TimeRate.monthlyRenewalEveryThirtySeconds'
// However, we've found that their behavior is not equivalent since using `monthlyRenewalEveryThirtySeconds`
// results in a crash in our tests.
testSession.timeRate = .init(rawValue: 6)! // == .oneSecondIsOneDay
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of this, but using .oneSecondIsOneDay directly creates a warning with a fix me button that changes it to .monthlyRenewalEveryThirtySeconds. And using .monthlyRenewalEveryThirtySeconds always results in a crash in my local machine.
I think this is acceptable since we are in unit tests and it is clearly (?) documented with comments.

Lmk if you think otherwise!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could the rawValue change between versions of iOS?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question. I don't think so, as it would mean breaking the API.

I wouldn't rely on this in the main code, but in unit tests I think it's fine (better this, with no warning and no fix me button that could lead to a crash.

For reference, this is the crash I'm getting locally when using .monthlyRenewalEveryThirtySeconds instead of . oneSecondIsOneDay.

https://forums.swift.org/t/sigabrt-freed-pointer-was-not-the-last-allocation-crash/78772

@ajpallares ajpallares marked this pull request as ready for review July 15, 2025 14:23
@ajpallares ajpallares requested review from a team July 15, 2025 14:29
Copy link
Contributor

@facumenzella facumenzella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try it out!!

@ajpallares ajpallares merged commit 6655bb3 into main Jul 17, 2025
12 checks passed
@ajpallares ajpallares deleted the fix-flaky-tests branch July 17, 2025 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants