Bringing an Experimentation Culture to B2C Companies: The Story of ABsmartly

ABsmartly had its origins in the pioneering work of Jonas Alves, one of the founders (along with Márcio Martins as the CTO and Bruno Silva as the Cloud Architect), and the mastermind behind the Booking.com Experimentation platform.

Jonas' journey into the world of A/B testing began in 2008 when he joined Booking.com as a full-stack developer. At that time, A/B testing was not as widespread in the digital space, making him one of the early proponents of this powerful methodology.

His initial role at Booking.com was to run A/B tests together with the designer (Luciano Kohan) and revamp the existing experimentation platform, which was in its nascent stages. He introduced features like visualizations for confidence intervals and the ability to track various metrics with a single line of code (basically what is now called a CDP), making the platform more sophisticated and user-friendly. With his technical skills, he significantly improved the system and made it more accessible to experimenters.

Enabling Experimentation at Scale

One of the notable improvements during Jonas's tenure was simplifying the process of running A/B tests by moving away from HTML templates, which made the experiments easier to set up. Booking.com saw a surge in experimentation, including successful tests like the "urgency messages" that became a staple on e-commerce websites.

Recognizing the need to accommodate a growing number of experimenters, Jonas fought to expand experimentation outside of the website team, to various development teams, democratizing the practice within the company. Jonas took on the role of Product Owner of Experimentation in 2012, overseeing training for new hires and for seasoned experimenters, feature development, report creation, advanced statistics and the addition of essential guardrail metrics to increase velocity and peace of mind. His efforts were instrumental in maintaining a strong experimentation culture during Booking.com's rapid growth.

To further enhance the quality of experimentation, Jonas contacted industry experts like Ronny Kohavi, who was still running experimentation at Microsoft at that time and referred to Roger Longbotham, who provided valuable insights into statistical methods, data analysis and training. Experimentation at Booking.com reached a scale where they were starting between 100 and 200 AB tests per day, and running more than 1000 experiments simultaneously, an impressive achievement by any standard.

The Motivation Behind ABsmartly

In 2015, Jonas decided to leave Booking.com, passing the responsibility to his friend Lukas Vermeer, and embarked on a new journey.

Jonas worked one year with Catawiki to help them build their experiment platform and build a custom training program to help them grow their experimentation culture.

Jonas joined Tripaneer.com as a cofounder, together with Mário Silva and Márcio Martins. Together they built a very sophisticated experimentation platform for Tripaneer.

At the same time Jonas continued offering experimentation consulting services to various companies like GetYourGuide, Tinder, Viagogo (which acquired StubHub), OkCupid, Remitly, Coupang, PicNic, Preply, SpotAHome, ParkMobile, Adidas and many others. 

ABsmartly emerged as a response to the specific needs of companies seeking tailored experimentation solutions aligned with their unique business requirements. These companies were looking for better guardrail metrics, data ownership, and the ability to integrate with custom and internal tools, and ABsmartly aimed to provide these solutions.

Jonas's deep expertise and passion for experimentation, coupled with the contributions of Márcio Martins, Mário Silva and Bruno Silva, led to the creation of a platform that could cater to the diverse needs of businesses. They recognized that many companies, despite the availability of various experimentation platforms, still preferred to build their own internally due to customization, data ownership, and flexibility requirements.

A Statistically Robust Experimentation Platform

ABsmartly's platform was designed to address these concerns and offer a robust solution for experimentation.

In their pursuit of excellence, ABsmartly made a strategic partnership that has set them apart as one of the few and most innovative A/B testing platforms in the market. The company joined forces with Georgi Georgiev, the author of the book "Statistical Methods in Online A/B Testing: Statistics for Data-driven Business Decisions and Risk Management in E-commerce», and incorporated Georgi's AGILE sequential method into the platform.

Group Sequential Testing is a game-changing approach to A/B testing that allows for more dynamic and efficient decision-making. With this partnership, ABsmartly positioned itself as a pioneer in the industry, offering businesses a powerful tool that not only adheres to industry standards but also embraces cutting-edge statistical methodologies to deliver superior results in the world of online experimentation.

Jonas extensive experience in experimentation and the team’s commitment to fostering a culture of experimentation have played a pivotal role in the company's success.

Their holistic approach to experimentation is reflected in the way they've tailored ABsmartly to meet the distinctive needs of companies seeking data-driven decisions, while also nurturing a culture of experimentation and shared learning throughout the organization. This dedication to fostering an environment of ongoing experimentation and knowledge exchange sets ABsmartly apart in the industry.

ABsmartly Today

Today, ABsmartly stands at the forefront of the A/B testing and experimentation landscape, serving B2C companies across the globe. As a trusted partner for organizations seeking to make data-driven, informed decisions and stay competitive in the dynamic world of online business, they have become the go-to platform for companies in different sectors, including healthcare, e-commerce, media, travel, and more.