Uncovering Unknown Unknowns: Serko Ltd. Partners with ABsmartly to Drive Experimentation Culture and Boost Revenue

Overview

Serko Ltd., the B2B arm of Booking.com, recognised that in a competitive market, traditional approaches to product development and decision-making were no longer enough to meet their customers' complex and shifting needs. The company understood that to stay ahead, they needed to embrace a culture of experimentation that would allow them to quickly test, learn, and adapt. 

By systematically validating assumptions and uncovering unknown unknowns, Serko aimed to reduce experimentation risks and ensure that they were responding to customer demands. This culture of experimentation was seen as essential not only for driving business growth but also for fostering a more agile, data-driven organisation that could respond swiftly to changes in the market.

Challenge

The Need for a Structured Experimentation Approach

Before partnering with ABsmartly, Serko faced significant challenges in how they approached innovation and product development. Serko's leadership recognised that customers often do not know what they want until they are shown something tangible, making experimentation a critical component of their strategy. The company had considered using various tools, including developing their own, but these options either lacked the flexibility or required a level of scientific rigour that made it difficult for cross-functional teams to engage effectively.

Serko were keen to rapidly scale their experimentation efforts and improve their ability to make data-driven decisions. They wanted to validate risky assumptions early in the product development process to save time and money and quickly adapt their strategies based on real-world customer feedback. Implementing a robust experimentation framework reduced the time it took to launch and iterate on new features, ultimately enhancing customer satisfaction and driving immediate business outcomes. 

In the long term Serko aimed to deeply embed a culture of experimentation across the entire organisation. Their vision is to make experimentation a core component of their decision-making process, enabling continuous innovation and reducing the risks associated with product development. By fostering this culture, Serko wanted to empower all their teams—across product, engineering, and marketing—to engage in experimentation, and ensure that insights gained were shared and leveraged throughout the company. The ultimate experimentation objective is to build a more agile, resilient organisation that can confidently anticipate and respond to market changes.

Solution

Implementing a Cross-Functional Experimentation Platform

ABsmartly provided the tools and support necessary to democratise experimentation across Serko’s teams, from product managers and engineers to designers and marketers. The platform's intuitive interface and advanced features allowed Serko to implement experiments quickly without the technical barriers that had previously slowed them down. ABsmartly's proactive approach—where the team often anticipated Serko’s needs before they were even articulated—was a major factor in their decision. 

The coaching ABsmartly provided was a huge value-add for Serko as it allowed team members to quickly understand how to use the tool as effectively as possible. ABsmartly was implemented as a cross-functional tool at Serko, empowering various teams to participate in and benefit from experimentation. 

One of the key successes in this collaboration was ABsmartly's ability to simplify complex experimentation processes, making it easier for non-technical team members to engage with the results and insights. While Serko initially encountered challenges in balancing scientific rigour with practical application, ABsmartly’s support helped streamline these processes, enabling the company to maintain the necessary scientific standards without overwhelming the teams.

Benefits

Rapid Scaling of Experimentation and Significant Business Impact

Within the first year of partnering with ABsmartly, Serko evolved from running almost no experiments to launching over 150 unique experiments—ranging from simple feature tests to complex initiatives—within just 12 months. This shift has led to millions of dollars in incremental revenue and has helped Serko avoid significant potential losses, demonstrating the power and necessity of an experimentation-driven approach.

One of the standout results was the ability to identify and validate risky assumptions quickly, allowing Serko to pivot or double down on strategies with confidence. The platform's ability to handle a variety of test types enabled Serko to test new features, validate assumptions, and even run "fake door" tests to gauge customer interest before committing to full launches. ABsmartly also provided Serko with key support in understanding when to use advanced statistical techniques like GST (Group Sequential Testing) instead of FH (Fixed Horizon) for timely results and accelerated learning.

The cultural shift at Serko has been equally significant. Experimentation has become a central part of the company’s decision-making process; this visibility and widespread use of experimentation has increased the visibility of experimentation outcomes and reinforced the importance of data-driven decisions at all levels of the company.

Conclusion

“Experimentation helps us prioritise what we want to focus on next and uncover the unknown unknowns” (Stuart McLoughlin). Experimentation has become essential to Serko’s product development strategy, helping them optimise existing features and uncover future opportunities. Stuart McLoughlin, Lead Product Manager - Retention & Growth, notes, "Using experimentation helps us actually discover that future,". By continuously testing and learning, Serko has seen significant improvement; their data-driven approach has "de-risked" product development, allowing the team to validate assumptions before fully committing resources. 

Ready to take the risk out of experimentation? Learn how ABsmartly can help you drive innovation and make data-backed decisions with confidence. For more information, visit our website here: www.absmartly.com