docs: clarify outlier limits are calculated across all variants

[joalves]

Summary

• Adds documentation clarifying that outlier limits (for Quantile and Standard Deviation methods) are calculated across the entire experiment population, not per-variant
• Explains the benefits of this approach: uniform thresholds, stable estimates from larger sample size, and unbiased comparisons

Context

This addresses a common question about whether outlier capping uses control group bounds or variant-specific bounds. The answer (confirmed by code analysis) is that limits are computed from all participants combined.

Test plan

• Preview the docs locally to verify formatting
• Review the added section for clarity and accuracy

Summary by CodeRabbit

• Documentation
  • Enhanced guidance on limit calculations for outlier detection methods.
  • Added comprehensive explanations and examples for Quantile, Standard Deviation, and Fixed outlier methods.
  • Clarified that outlier bounds are computed uniformly across all variants for consistent thresholds.
  • Expanded the Unlimited option documentation with practical usage recommendations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

This pull request updates the web console documentation for metrics creation, specifically the section on outlier handling and limit calculations. The changes introduce a new subsection explaining how limits are calculated for Quantile and Standard Deviation methods—noting that bounds are computed across all experiment participants collectively rather than per-variant. The documentation expands the Unlimited option section with a clarifying note and provides detailed explanations with examples for Quantile, Standard Deviation, and Fixed outlier methods.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 Hop, hop, hooray!
With metrics now so clear and bright,
Outliers caught in crystalline light,
Quantiles dance across every friend,
No variant bias, just fair-handed blend! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title clearly and specifically summarizes the main change: clarifying that outlier limits are calculated across all variants rather than per-variant, which aligns directly with the documentation additions and PR objectives.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)

docs/web-console-docs/goals-and-metrics/metrics/create.mdx (2)

185-201: Fix “removed” vs “capped” inconsistency in Fixed limits description.

Line 185 says values outside bounds are “removed”, but the example on Line 201 shows capping, and earlier text defines outlier handling as capping. This is contradictory and will mislead users. Align the wording with the actual behaviour.

🔧 Proposed doc fix

-This option lets you define explicit numeric boundaries. Any value below the lower limit or above the upper limit **is removed**.
+This option lets you define explicit numeric boundaries. Any value below the lower limit or above the upper limit **is capped to the nearest boundary**.

---

174-178: Clarify multiplier = 0 behaviour to match the stated formula.

The formula defines limits as mean ± (stdev × multiplier), but the bullets say “keeps all values below/above the mean” when multiplier is 0. With capping, multiplier 0 implies values beyond the mean are capped to the mean, not kept. Either update the bullets to reflect capping, or adjust the formula/text to explain a special-case “disabled side” behaviour if that is the real implementation.

🔧 Proposed doc fix (if capping follows the stated formula)

-Setting multipliers to 0 effectively disables trimming on that side:
-- Lower multiplier 0 → lower limit = mean → keeps all values below the mean
-- Upper multiplier 0 → upper limit = mean → keeps all values above the mean
-
-Setting both to 0 is the equivalent of setting no outlier limit.
+Setting multipliers to 0 sets the limit to the mean on that side:
+- Lower multiplier 0 → lower limit = mean → values below the mean are capped to the mean
+- Upper multiplier 0 → upper limit = mean → values above the mean are capped to the mean
+
+Setting both to 0 caps all values to the mean.

[netlify]

✅ Deploy Preview for absmartly-docs ready!

Name	Link
🔨 Latest commit	d51704a
🔍 Latest deploy log	https://app.netlify.com/projects/absmartly-docs/deploys/6977788be00f840008f412f7
😎 Deploy Preview	https://deploy-preview-245--absmartly-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.