Group Sequential Testing
Group Sequential Testing (GST) allows us to stop or ship an A/B test early when there is a large effect, while also retaining the ability to detect smaller effects within a reasonable timeframe.
Unlike mSPRT, Group Sequential Testing does not require a drastic increase in sample size to be able to detect small effects. And unlike fixed-horizon A/B testing, Group Sequential Testing does not require one to wait until the end of the experiment to make a decision. An A/B test using GST requires pre-determining the maximum sample size with a power calculation based on the minimum effect of interest, as well as the number of analyses to be conducted throughout the duration of the experiment. Our intuitive user interface guides users through this process in an easy way.
In the case of a 30-day experiment where daily data review for decision-making flexibility is desired, the test can be configured with 30 analysis points. While this may slightly reduce statistical power compared to fixed-horizon testing, it greatly speeds up decision-making, leading to 20% to 80% faster decisions on average, as significance is commonly reached before the full sample is collected.
Importantly, Group Sequential Testing does not increase the false positive rate, and it can accelerate experiments by up to two times compared to the traditional mSPRT approach used in other A/B testing tools on the market.
Try ABsmartly on A PoV
You can acquire a 60-day PoV (Proof of Value) to explore the features and experiment with the platform before committing to an annual subscription.
