Skip to content

Conversation

mariusandra
Copy link
Collaborator

@mariusandra mariusandra commented Jan 19, 2023

Problem

It would be great to use HogQL in property filters.

This is a lighter version of #13264 . Probably makes a lot of old tickets obsolete.

Changes

Adds support for HogQL filters in a lot of places. Everything is behind the hogql_expressions feature flag.

image

HogQL support in insights, including access to event and person properties:

  • trends global filters - works & has test
  • trends step filters - works & has test
  • trends breakdown - works & has test
  • funnels global filters - works & has test
  • funnels steps - works & has test
  • lifecycle global filters - works & has test
  • paths global filters - works & has test
  • retention global filters - works & has test
  • stickiness - works & has test

HogQL support elsewhere:

  • hogql in action step filters, show action on events list - works & has test
  • action with hogql in property as trend steps - works, has test
  • session recording list - works, has test
  • hogql in elements query - works, just not person.properties unless person-on-events is enabled
  • action with hogql in property as part of a cohort - seems to work, but no explicit tests
  • action with hogql in property as funnel step - I'm not sure it works

Technical changes:

  • Passes the hogql_context object deeply into queries.
    • This context contains a values: Dict[str, Any], which collects encountered string literals in HogQL
    • Every new entry into values is just hogql_context.values[f"val_{val.length}"] = 'string', so the keys are rather predictable.
  • These hogql_context.values need to be passed onto the SQL query, and hence get passed around a lot via hogql_context=filter.hogql_context or equivalent code. That's where a lot of the changes come in.
  • Connect the HogQLContext to a Filter, to make it easier to pass the context around queries. Note that calling filter.with_data (cloning a filter) passes the context by reference, meaning both the old and new filter will gather variables into the same dict.
  • Adds support for "global filters" onto the lifecycle chart. It worked in the backend, but was hidden in the frontend 🤔
  • Adds a lot of tests to make sure hogql person and event property access works in all the different insights and other conditions (e.g. as part of an action).
  • Removed a dead cohort query from get_entity_cohort_subquery that I just couldn't find used anywhere
  • Patches a few tests with @override_settings(PERSON_ON_EVENTS_OVERRIDE=False) # :KLUDGE: avoid making a bunch of extra queries that may be cached instance-wide. Forcing this setting avoids all manner of SQL queries (up to 5 per test) that run when is_cloud(), posthoganalytics.feature_enabled() and actor_on_events_ready() (checks async migrations) functions are called. Since we're going to "solve" the person on events issue soon, and I couldn't work out a reliable pattern between how to mock all of this with all the tests that run in CI, I decided to patch 5 test files and tell them to consider person on events disabled. For now.

How did you test this code?

Work in progress

@mariusandra mariusandra changed the title Hogql filters again feat(data-exploration): HogQL filters again Jan 19, 2023
@mariusandra
Copy link
Collaborator Author

image

🤣

@mariusandra mariusandra merged commit 5b4a3bf into master Jan 31, 2023
@mariusandra mariusandra deleted the hogql-filters-again branch January 31, 2023 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants