Skip to content

Conversation

yakkomajuri
Copy link
Contributor

Adding some initial caching to experiment results.

This isn't in its ideal state, but I opted for getting the ball rolling and iterating. Currently caching experiment results for 30min only with some bespoke logic meant for them only. This should at least prevent us from recalculating on refreshes and changes during a single session.

Essentially got the essence of the @cached_function logic and implemented it for experiments. Unsure if it's the best approach. Also we seem to not test @cached_function anywhere? Although I could add some tests here

Also includes a fix for the fact that setFilters isn't called when setting an experiment goal which was leaving the sampling toggle off even though the insight was sampled.

@yakkomajuri yakkomajuri requested a review from neilkakkar March 14, 2023 16:38
@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

  • chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted
  • firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@yakkomajuri yakkomajuri requested a review from pauldambra March 15, 2023 14:38
@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

  • chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted
  • firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

Copy link
Member

@pauldambra pauldambra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick pass before going afk without running the code, looks good

it's a shame the cached_function decorator isn't in a place where we can easily re-use it

@@ -525,7 +525,7 @@ export const experimentLogic = kea<experimentLogicType>([
const response = await api.get(
`api/projects/${values.currentTeamId}/experiments/${values.experimentId}/results`
)
return { ...response, fakeInsightId: Math.random().toString(36).substring(2, 15) }
return { ...response.result, fakeInsightId: Math.random().toString(36).substring(2, 15) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

presumably the errors in the visual regression tests come from this?

Suggested change
return { ...response.result, fakeInsightId: Math.random().toString(36).substring(2, 15) }
return { ...(response?.result || {}), fakeInsightId: Math.random().toString(36).substring(2, 15) }

but I didn't run the code so 🙈

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I thought about this as well, but {} will throw an error down the line.

I didn't improve things here, but didn't make them worse either, because if the result that now comes inside response.result wasn't at the top-level before, we'd also throw

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this will also throw right on deploy, but that shouldn't be too long.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll consider how to best handle things

Copy link
Contributor

@neilkakkar neilkakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this!

Totally fine if out of scope, just mentioning so you're aware, we also make calls to secondary_results which could use the same treatment.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

  • chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted
  • firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

  • chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted
  • firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@yakkomajuri
Copy link
Contributor Author

ready for a final look @neilkakkar @pauldambra

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol this is clearly not good - looking

Copy link
Contributor

@neilkakkar neilkakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to update the API response structure going to the frontend snapshots so those look correct, but otherwise LGTM!

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

  • chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted
  • firefox: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@yakkomajuri yakkomajuri merged commit 49803a9 into master Mar 16, 2023
@yakkomajuri yakkomajuri deleted the experiments-caching branch March 16, 2023 19:41
fuziontech added a commit that referenced this pull request Mar 16, 2023
* master: (101 commits)
  feat: feedback tab - improve the UI and switch to HogQL (#14777)
  feat(experiments): cache experiment results (#14742)
  fix(empty-state): properly fix query timeout state (#14793)
  chore(feature-flag): release json payload flag (#14775)
  feat(hogql): make sure joins work with properties (#14790)
  test(frontend): Tune visual regression test failure threshold (#14766)
  feat(hogql): count distinct and count star (#14786)
  feat(perf): only load experiment results once (#14772)
  feat: allow new insight type choice from sidebar (#14759)
  fix(persons): hide persons in the future (#14308)
  feat: allow query cards in saved insights grid (#14784)
  fix(lemon-ui): Align padding for small/large buttons with icons (#14750)
  chore(deps): Upgrade TypeScript from 4.8 to 4.9 in the frontend (#14609)
  feat: rename insight query tab to json tab (#14781)
  feat: use same top heading for insight card and exported insight (#14780)
  feat: define new insight urls in one place (#14778)
  fix(hogql): person properties in a funnel breakdown (#14765)
  fix: hide query based insights in recently viewed list (#14774)
  feat: improve query summaries (#14768)
  feat(hogql): add sampling support (#14733)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants