Skip to content

Experimentation MVP Things to Do #7462

@neilkakkar

Description

@neilkakkar

Following up from #7418 , here's what we need to do to get the MVP out in these 2 weeks:

Main Tasks:

  1. Figure out & build the UI Creation Flow - @liyiy
    • @clarkus & @paolodamico : we could use some input here about the basic flow. (specifically on how we should structure this, disregard full fledged wireframes for now) There's 2.5 steps to creating an experiment: (1) Create an insight & (2) Choose a FF. (2.5): Name and description of experiment.
    • As you can tell, there's lots of ambiguity above^ that we need to clean up.
    • (We considered a flow where funnels have a "create experiment from insight" button, but @macobo mentioned how this increases complexity on the standard funnels view, and reduces discoverability quite a bit in simple mode)
    • (future context): There's a back & forth when choosing an insight & FF: They choose an insight, we tell them how long the experiment will run for. They tweak FF rollout % / definition, we tweak how long we expect experiment to run for. They tweak insight definition, we tweak how long to run the experiment for. What's a good way of doing this?
  2. Build Experiment Page with the Insight and experiment results (also see here)
  3. Create Plugin to add $active_feature_flags & $feature/{key} to every event (we only do it for web captures today) - @neilkakkar - deferred. Just focusing on posthog-js for now.
    • This implicitly allows breakdowns by Feature Flags on Funnels
  4. Given a funnel with FF breakdown, calculate experiment results
  5. Given a funnel and FF, calculate how long an experiment should run

New constraints:

  1. To give accurate results on expected running time, we need to answer the question: Given a FF, how many people in the past X days on this insight would've belonged to this FF? And use this to estimate. For now, restricting scope to basic FFs only (i.e. flag with the only property being the rollout % to all users).

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions