Faster Decisions with Group Sequential Tests

ABsmartly’s proprietary Group Sequential Testing (GST) engine empowers companies to make faster data-driven decisions—up to 80% faster than conventional Fixed-Horizon A/B testing.

Better business results 

Group Sequential Tests stop tests earlier than equivalent fixed-horizon A/B tests, on average, while retaining statistical rigour. This results in direct business benefits by reducing exposure to harmful variants and through earlier shipping of beneficial changes. The business effect is proportional to how early a test stops, compared to a fixed-horizon equivalent.

That’s why the experimentation experts use it. Organisations prioritising product-lead growth need a tool that helps them innovate faster and more efficiently. The added value and agility is too good to miss out on.  Experiment like the pros with Group Sequential Testing.

Group Sequential Tests stop tests earlier than equivalent fixed-horizon A/B tests, on average, while retaining statistical rigour. This results in direct business benefits by reducing exposure to harmful variants and through earlier shipping of beneficial changes. The business effect is proportional to how early a test stops, compared to a fixed-horizon equivalent.

That’s why the experimentation experts use it. Organisations prioritising product-lead growth need a tool that helps them innovate faster and more efficiently. The added value and agility is too good to miss out on.  Experiment like the pros with Group Sequential Testing.

Best of both worlds

The Group Sequential approach is the middle ground between fixed-horizon tests and fully sequential tests like SPRT, mSPRT and variants of Always-Valid Inference.  GSTs offer the right level of speed, agility, and generalizability without incurring the significant loss of power typical for fully sequential tests. Group sequential tests are a best-of-both-worlds method which is ideally suited for product experimentation at scale.

Want to become a GST expert?

The Group Sequential approach is the middle ground between fixed-horizon tests and fully sequential tests like SPRT, mSPRT and variants of Always-Valid Inference.  GSTs offer the right level of speed, agility, and generalizability without incurring the significant loss of power typical for fully sequential tests. Group sequential tests are a best-of-both-worlds method which is ideally suited for product experimentation at scale.

Want to become a GST expert?

Download the whitepaper

Decisions 80% faster

Group Sequential Testing (GST) requires a lower sample size than a fully sequential test (mSPRT) with equivalent statistical guarantees while allowing more agility and faster decision-making than fixed-horizon testing. Tests typically take significantly less time and fewer users to reach a conclusion. On the lower end, a GST would take 20% fewer users to complete, whereas in favourable scenarios it may end 80% faster, on average.

Group Sequential Testing (GST) requires a lower sample size than a fully sequential test (mSPRT) with equivalent statistical guarantees while allowing more agility and faster decision-making than fixed-horizon testing. Tests typically take significantly less time and fewer users to reach a conclusion. On the lower end, a GST would take 20% fewer users to complete, whereas in favourable scenarios it may end 80% faster, on average.

Scale experimentation efficiently

Organisations looking to boost decision-making through experimentation should focus on running more tests to refine insights, which increases value and promotes company-wide interest. With the right tool and the Flywheel approach, teams can scale experimentation and run multiple A/B tests simultaneously across their organisation.Group Sequential Testing empowers this approach even more by shortening the average time it takes to complete an A/B test. This allows your team to make timely decisions, adapt faster, and achieve faster iteration. Maximise your testing velocity and unlock a higher rate of innovation by identifying winning strategies sooner and cutting losing ones short.

DYK? GST is used for Clinical Trials

Can GST save lives? Well, not in eCommerce! but Pharmaceutical development and Clinical Trials use GST because stopping early means potentially saving lives through early identification of either efficacy or futility. “Sequential Trials” allows for interim analyses and potential early stopping based on accumulating data. 

Unlike traditional fixed-sample designs where researchers collect data until the study is completed, group sequential designs incorporate interim analyses at prespecified points during the trial, enabling ongoing assessments of efficacy and safety. Note, in general online controlled experimentation, a prespecified schedule is not strictly necessary and is optional when trying to balance agility and speed.

Can GST save lives? Well, not in eCommerce! but Pharmaceutical development and Clinical Trials use GST because stopping early means potentially saving lives through early identification of either efficacy or futility. “Sequential Trials” allows for interim analyses and potential early stopping based on accumulating data. 

Unlike traditional fixed-sample designs where researchers collect data until the study is completed, group sequential designs incorporate interim analyses at prespecified points during the trial, enabling ongoing assessments of efficacy and safety. Note, in general online controlled experimentation, a prespecified schedule is not strictly necessary and is optional when trying to balance agility and speed.

Home

Benefits

Resources

About

Pricing

Benefits

Resources

About

Pricing

Benefits

Resources

About

Pricing

54% efficiency gain with GST

54% efficiency gain with GST

"Through continuous experimentation, we've seen remarkable results in just a few short months: a 5.17% increase in unique flight quotes and a 54% efficiency gain in stopping experiments earlier with GST, compared to using a fixed horizon approach."

"Through continuous experimentation, we've seen remarkable results in just a few short months: a 5.17% increase in unique flight quotes and a 54% efficiency gain in stopping experiments earlier with GST, compared to using a fixed horizon approach."

Rafael Buitrago, Digital Experimentation Consultant

Rafael Buitrago, Digital Experimentation Consultant

LATAM Airlines

Run and discuss experiments at scale.

Sign up for a demo

View all benefits

Run and discuss experiments at scale.

Sign up for a demo

View all benefits

Run and discuss experiments at scale.

Sign up for a demo

View all benefits