As noted in #6360, our ambitious goal this coming sprint is to build the best possible quant analysis tool.

In prep for this, we did some user testing on the MVP, and an industry analysis + feature brainstorm. (thanks Paolo!)

I interpret best as a tool that does its job without getting in the way. And the job for correlation analysis is to help diagnose causes.

There's two things we need to accomplish to be successful here:

1. Surface meaningful & easy to understand results
2. Easily allow further actions on surfaced results to make them actionable.

For example, I might see that "Video Played" is a signal for success, but I can't understand why: Maybe it's specific videos that specially convert well, or something else to do with them. So, once I see "Video Played", I expect to be able to answer the following question: What is it about Video Played that makes it a good signal?

Implicit in the 2 things above is not cluttering the UI with 1000s of different options that might be useful: If it isn't clear how to answer your question, the tool isn't useful.

Things we ought to do: (rough prioritisation for now, evolving list). Ideally we'd do all of these, but given the 2 week constraint, we want to choose maximum impact things first. I think both lists go in parallel, and we should pick up the first thing from either and continue & see where we get till the end of the sprint.

Surface meaningful & easy to understand results

We're already in a place where results are useful to some people. Making these better involves allowing discarding 'obvious' events, surfacing things users are familiar with (actions), and making the numbers (if any) easy to understand & removing discrepancies.

3. Allow global property exclusions from Property correlation
4. Allow excluding events from Events Correlation
5. Polish our UI, make things clearer.
   1. Make a call on what all information to show: Entire contingency tables, success & failure counts, or just the odds ratio.
   2. Actionable tips to reduce skew (a.k.a. give more confident results) vs mentioning a confidence level
6. Include Actions in Event signals
   1. Figure out how to do this & timings & how long this takes, since action matching has moved back to plugin-server
7. [ ]

Allow further actions on surfaced results

Where we're lacking right now is what to do with the results. It's hard to go from "oh, this is a signal" to "What I should do about it". Drilling down on signals via paths & breakdowns; and then qualitative feedback on this small set, like session recordings, help figure out the problems.

For example, on a property breakdown like browser_version=87 being a signal of failures, I'd love to see some session recordings to see what went wrong.

1. Given an event, allow breaking down by event properties - @neilkakkar
   1. Figure out technical constraints around this - can we do all event properties (would be unique to us) quickly?
   2. Figure out how & where to display this
   3. Some special automatic handling for internal events like $autocapture and $pageview
2. Given an event / property signal, show me all persons who did this,
   1. and link to relevant session recordings
   2. [ ]
3. Given a signal, show me the paths these specific users took.
4. (questionable value): Add this event signal to the funnel

I think executing the above things gets us to a place where we are integrated pretty well into other tools. It complements, and shortcuts what users would anyways be doing to diagnose causes, thus achieving its goal of helping diagnose causes.

To judge success: Get 3 users to LOVE correlation analysis

cc: @clarkus @paolodamico @marcushyett-ph @hazzadous @liyiy @EDsCODE @macobo for any ideas on what we should / should not be doing.