Better business results
Group Sequential Tests stop tests earlier than equivalent fixed-horizon A/B tests, on average, while retaining statistical rigour. This results in direct business benefits by reducing exposure to harmful variants and through earlier shipping of beneficial changes. The business effect is proportional to how early a test stops, compared to a fixed-horizon equivalent.
That’s why the experimentation experts use it. Organisations prioritising product-lead growth need a tool that helps them innovate faster and more efficiently. The added value and agility is too good to miss out on. Experiment like the pros with Group Sequential Testing.
Best of both worlds
The Group Sequential approach is the middle ground between fixed-horizon tests and fully sequential tests like SPRT, mSPRT and variants of Always-Valid Inference. GSTs offer the right level of speed, agility, and generalizability without incurring the significant loss of power typical for fully sequential tests. Group sequential tests are a best-of-both-worlds method which is ideally suited for product experimentation at scale.
Want to become a GST expert?
Decisions 80% faster
Group Sequential Testing (GST) requires a lower sample size than a fully sequential test (mSPRT) with equivalent statistical guarantees while allowing more agility and faster decision-making than fixed-horizon testing. Tests typically take significantly less time and fewer users to reach a conclusion. On the lower end, a GST would take 20% fewer users to complete, whereas in favourable scenarios it may end 80% faster, on average.
Scale experimentation efficiently
Organisations looking to boost decision-making through experimentation should focus on running more tests to refine insights, which increases value and promotes company-wide interest. With the right tool and the Flywheel approach, teams can scale experimentation and run multiple A/B tests simultaneously across their organisation.
Group Sequential Testing empowers this approach even more by shortening the average time it takes to complete an A/B test. This allows your team to make timely decisions, adapt faster, and achieve faster iteration. Maximise your testing velocity and unlock a higher rate of innovation by identifying winning strategies sooner and cutting losing ones short.
DYK? GST is used for Clinical Trials
Can GST save lives? Well, not in eCommerce! but Pharmaceutical development and Clinical Trials use GST because stopping early means potentially saving lives through early identification of either efficacy or futility. “Sequential Trials” allows for interim analyses and potential early stopping based on accumulating data.
Unlike traditional fixed-sample designs where researchers collect data until the study is completed, group sequential designs incorporate interim analyses at prespecified points during the trial, enabling ongoing assessments of efficacy and safety. Note, in general online controlled experimentation, a prespecified schedule is not strictly necessary and is optional when trying to balance agility and speed.
LATAM Airlines
