How to Strategically Run an Experiment

Last updated: June 2026

Overview

Running an experiment for too short a period is the most common reason teams draw wrong conclusions from their tests. This article covers minimum run times, traffic thresholds, and how to interpret early results.

Minimum Guidelines

Why 3 weeks? A full three weeks accounts for day-of-week traffic variation. Stopping after one weekend or one weekday spike risks drawing conclusions from unrepresentative data.

Why 500 sessions per variant? Below this threshold, statistical confidence scores on the dashboard are unreliable regardless of what they display.

Why 95% confidence? This is the standard minimum for making decisions based on experiment results. The Jurni dashboard displays confidence as a score; wait for 95+ before declaring a winner.

Budget and Timeline Expectations

There is no minimum spend requirement in Jurni, but more traffic means faster results.

• Brands spending ~$500/day on a campaign can typically reach significance in 1 to 2 weeks

• Brands spending ~$100/day may need 4 to 6 weeks

Push as much traffic as possible to reach significance faster.

What to Avoid

• Do not pause or adjust an experiment in the first 48 to 72 hours. Early data is noisy and consistently misleading.

• Do not act on results below 95% confidence, even if the trend looks strong.

• Do not change the traffic allocation mid-experiment. This resets the signal quality.

Early Winners

If one variant is performing dramatically better (for example, 2x or more on CVR after 7 days), you can choose to end early. Document that the result is directional and not statistically confirmed. Use it to inform your next test, not as a final conclusion.

Multi-Variant Experiments

Experiments with 3 or more variants need proportionally more traffic. Add roughly 50% more sessions per additional variant beyond the original two.

Tip: Keep early experiments to two variants (control vs. one change) to reach significance faster and isolate what is actually driving the difference.

Screenshot placeholder: Jurni Experiments dashboard showing confidence score, sessions per variant, and run duration.

Frequently Asked Questions

Can I extend an experiment that is close to significance?

Yes. Just let it run longer. The clock does not reset unless you change the traffic allocation.

What if traffic is too low to ever reach significance?

Focus traffic on fewer funnels, increase ad spend on the experiment Smart Link, or run a simpler single-variable test with a tighter creative change. Simpler tests require less traffic to reach significance.

Should I run an experiment on a brand new funnel or a proven one?

Start with your best-performing funnel as the control. Testing against a weak baseline inflates apparent lift and makes it harder to know if your change is genuinely effective.