Experimentation Made Smarter With Our December Update

This December, we’re focused on simplifying the process of setting up and running tests, empowering teams to experiment with greater confidence and clarity. Our latest platform update makes your experimentation experience smoother, faster, and more flexible than ever. 

Enhanced Group Sequential Testing (GST)

We made significant usability improvements to our GST to make it easier and faster to create and monitor experiments.

One of the key improvements is real-time estimates for interim analysis. You can now see exactly when the next analysis will take place, so there’s no guesswork involved. We’ve also streamlined GST setups, allowing you to configure tests using just a maximum runtime without the need to define a Minimum Detectable Effect (MDE).

Another useful addition is our new metric performance graph. This gives you a clear picture of how a metric behaves before you commit to it, helping you fine-tune your experiment for more reliable insights. Finally, we’ve tidied up the experiment creation form, moving fields around so that it’s easier to navigate and quicker to use.

These upgrades make GST more accessible and flexible, so teams can focus on running meaningful experiments without getting bogged down in setup details. For the full rundown of these changes, have a look at our detailed documentation. 

Field Locking in Templates

If you’re part of a team running lots of experiments, you’ll know how important it is to have consistency in how tests are set up. With field locking in templates, you can now standardise experiment designs by locking down certain fields so they can’t be changed.

This is particularly useful for fields like guardrail metrics or anything you don’t want people to alter. It’s a simple way to reduce errors, save time, and ensure everyone is on the same page when creating experiments. Field locking helps organisations with multiple teams or high experimentation velocity minimise errors by ensuring every experiment is set up correctly from the start. You’ll find a full guide to setting up templates in our documentation hub. 

Default Platform Settings

Experiment setup can sometimes feel repetitive if you’re configuring the same fields over and over again. That’s why we’ve introduced default platform settings to take care of those repetitive tasks for you.

Now you can predefine defaults for key analysis fields, including:

• Your preferred analysis type (Group Sequential or Fixed Horizon).

• Minimum and default confidence and power levels to keep experiments aligned with your standards.

• Default MDEs for both Group Sequential and Fixed Horizon experiments.

• Pre-set values for GST-specific fields like time to first analysis, analysis intervals, and maximum runtime.

These defaults mean less fiddling with settings and more time spent designing impactful tests. They’re particularly helpful for organisations running experiments at scale, where consistency and efficiency are critical. You can configure these settings in Settings > Platform Settings, and once they’re set, they’ll apply automatically to all new experiments.

Making Experimentation Smoother, Faster, and More Flexible

Looking back over 2024, it’s been a big year for making experimentation smoother, faster, and more scalable. We’ve focused on practical improvements that remove friction and help you get more out of every test you run.

New features like enhanced Group Sequential Testing and metric performance graphs make it easier to design and monitor experiments that deliver reliable insights. At the same time, field locking and default platform settings give your teams the structure they need to run experiments efficiently and consistently, even at scale.

As we head into 2025, we’ll keep listening to your feedback and evolving the platform to meet your needs. To explore how our latest features can streamline your experimentation process and deliver better insights, book a personalised demo today and see ABsmartly’s updates in action.