As we step into spring, the ABsmartly team is thrilled to announce a suite of innovative features designed to enhance and optimise your business's A/B testing experiments. This April, we're rolling out even more groundbreaking tools that solidify our commitment to advancing your analytical capabilities and providing actionable insights to propel your business forward. Among these, our main highlight is the introduction of Interaction Detection now available in the ABsmartly online experimentation platform. 

Interaction Detection in A/B Testing

In the dynamic world of A/B testing, ensuring accurate and reliable results is paramount. That's why we've developed a cutting-edge tool that automatically detects interactions between A/B tests, streamlining your online experimentation process and giving you confidence in your outcomes.

Our experience at Booking.com, coupled with observations from Microsoft, reinforces our understanding that unplanned interactions are incredibly rare. However, despite the rarity of such interactions, simply the fear of them (interactions) often prevents many teams from fully leveraging the potential of running overlapping tests at scale.

Here at ABsmartly, we understand the importance of alleviating these concerns. Drawing from our experience and insights from the industry, we've integrated interaction detection directly into our platform.

Our goal is simple: to empower elite experimenters to run online experiments confidently and efficiently, without the fear. With our Interaction Detection feature launch, you can now leverage statistics to detect interactions seamlessly.

Additional Innovations

Feature Flags for Easy Activation

Feature Flags (or Feature Toggles) means you can now easily activate or deactivate features independently of your experiments, maintaining a clear separation between experimental hypotheses and feature management. With Feature Flags, you're equipped with a powerful "kill switch" for immediate action, which gives you more flexibility and makes it simpler to create a list for experiments and a separate list for features. 

Imagine a developer puts code in the codebase that has a serious bug in it, but nobody notices it until it gets to production. Normally, you would have to find the bug, remove it, commit that code, re-deploy the site, by which point it could have been seen by many customers already. If it was set up as a feature flag though, you could simply come into the ABsmartly experimentation platform and 'turn it off', which would take no time at all. 

When adding a large feature, it can be hard to do a deployment without also sending every new feature. Before Feature Flags, developers might keep a copy of their codebase separate from the main one, so they can do a release in the middle. Then, when it's ready to be released,  devs have to double-check that copy (which is a lot of code by this point), before merging it into the main codebase. This takes a long time and makes it more prone to bugs, not to mention the time it takes to keep the "copy" up to date with other changes in the main codebase. What they can do now is push everything into the main codebase in much smaller packages, but with a feature flag that makes sure that users cannot see this new feature, even though it's already there. Feature Flags save precious dev time (a rough estimate of 10 hours down to 10 minutes) and decreases the likelihood of bugs in the code. The mantra that devs use for that is called CI/CD, which stands for Continuous Integration and Continuous Delivery/Deployment. This time savings and improved quality is extremely valuable when scaled across the organisation.

Metric Threshold Alerts

To make absolutely sure that you don’t miss anything that’s happening with your experiments, we're implementing a real-time warning system for metric thresholds, whether that be primary, secondary, exploratory or guardrail metrics. Users can define specific metrics and their acceptable ranges.

Should any metric unexpectedly deviate, an alert will be promptly issued within the platform. Imagine page load time suddenly increases by 40 seconds, your team needs to know right away that this is happening. This proactive metric threshold monitoring provides peace of mind and reassurance, allowing you to programmatically end the experiment as with guardrail metrics like revenue, page load time, and cart abandonment.

Regex Goal Property and Segmentation Filters

Our platform now supports regex (match) comparison, enabling more general string targeting for your metrics. This feature allows you to segment your audience based on a pattern instead of a specific string, permitting you to create metrics and segment data more quickly and more openly.

Assignment Conflict Alerts

Alongside our brand new Metric threshold alerts, we have also added assignment conflict alerts for your (and your experiments’) peace of mind. The assignment conflict health check looks for units in your experiment that have been exposed to multiple variants or variants that do not exist. This is not possible when using the ABsmartly SDKs’ regular tracking functions, but is a potential issue when using custom assignments. This alert will give you absolute certainty that your participants are only seeing what they should when they should. 

The Features You Need to Scale Experimentation

At ABsmartly, we are continually iterating and improving our platform and our robust statistical engine for improved accuracy and true real-time reporting. We want to offer your business the opportunity to use cutting-edge digital tools and the right features that empower you to achieve your online experimentation goals efficiently and confidently. From the revolutionary Experiment Interaction Detection to the practical agility offered by Feature Flags, our latest feature releases are designed to help you easily embed a culture of experimentation in your company and continue to develop and improve the products and services you offer your customers.