Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/web-console-docs/goals-and-metrics/metrics/create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,20 @@ You can choose one of four methods to manage outliers:
:::caution
Outlier limits can reduce variance and make metrics more stable, but they also change the underlying data.
When used incorrectly, they may hide meaningful effects or introduce bias in experiment results.
Always ensure your limits reflect real-world constraints rather than engineering convenience, and avoid tuning them in response to specific experiment outcomes.
Always ensure your limits reflect real-world constraints rather than engineering convenience, and avoid tuning them in response to specific experiment outcomes.
:::

#### How limits are calculated

When using the **Quantile** or **Standard Deviation** methods, the outlier limits (lower and upper bounds) are calculated across **all participants in the experiment combined**, not separately for each variant.

This means:
- The same thresholds apply uniformly to all variants (control and treatments)
- The calculation uses the larger combined sample size, resulting in more stable threshold estimates
- All variants are treated identically, preventing any bias that could arise from variant-specific capping

This approach ensures fair comparisons between variants while benefiting from the statistical power of the full experiment population.

#### Unlimited

This is the default. This option applies no outlier treatment. All values are kept exactly as they appear in the event data.
Expand Down