Cultivating a Culture of Experimentation with A/B Testing Tools

Our founder Jonas Alves was recently interviewed by a PhD candidate for their thesis on experimentation platforms, and this blog is an excerpt from that interview. In it, Jonas shares his thoughts on experimentation culture, the five pillars of the framework, and the impact an experimentation culture can have for consumer websites and apps. Read on, and join the conversation on Experimentation Culture by following Jonas on LinkedIn.

Jonas Alves, Experimentation Pioneer

Jonas Alves pioneered one of the most successful online experiments of the twenty-first century when he and the team at Booking.com were the first to think of adding scarcity messaging to motivate faster conversions. Jonas was part of a two-person team that, at the time, with a relatively unsophisticated A/B testing tool, created a hugely efficient website and experimentation culture that they started scaling across the organization.

Jonas went to evolve the platform for Booking, making it more self-service and thus easier to add new metrics. This meant by early 2015, when he became responsible for experimentation, his team was tracking more than 5000 metrics across the website, and team members were putting live between 100 and 200 A/B tests per day, with upwards of 1000 experiments running at any point in time.  

With over 16 years of experience in experimentation, Jonas has established himself as a pivotal figure in A/B testing and product experimentation. In addition to his cutting-edge experimentation strategies at Booking.com, he also led the way in fostering a data-driven culture within the organization. His work has paved the way for a more inclusive and collaborative approach to experimentation, enabling a broader range of teams to contribute to the company's growth and success.

A Data-Driven Culture

In his subsequent roles at Catawiki and Tripaneer, Jonas continued to drive the culture of experimentation. He has consistently championed the importance of building a data-driven culture and understands that pooling his teams’ knowledge allows them to evolve best practices in experimentation.

Jonas firmly believes that a culture of experimentation should start from the top. Leaders play a crucial role in shaping the organization's mindset and approach towards experimentation. By promoting a growth mindset and embracing a little risk-taking and innovation, leaders set the tone for the entire company. In a business that revolves around experimentation, creating a culture that encourages curiosity, iteration, and learning from failures is essential if you want to move forward. 

The Five Pillars of a Culture of Experimentation

The ABsmartly philosophy on building a culture of experimentation is grounded in several key principles:

Embrace Failure as a Stepping Stone

In a culture of experimentation, failure shouldn’t be seen as a setback but an invaluable learning opportunity. This mindset encourages team members to take calculated risks, knowing that each experiment, regardless of its outcome, is a step toward uncovering insights that can help move the business forward. A great example of this philosophy exemplified is Stephen Bartlett hiring a Head of Failure. 

Foster Curiosity and Continuous Learning

For Jonas, curiosity is the fuel that drives experimentation. Encouraging team members to ask questions, challenge assumptions, and continuously seek out new knowledge creates a dynamic environment where innovation is not just encouraged but expected.

Democratize Decision-Making

Another critical aspect of building a culture of experimentation is the democratization of decision-making. By empowering team members at all levels to propose and conduct experiments, organizations can tap into a wide range of perspectives and ideas, therefore enhancing the potential for groundbreaking innovations.

Invest in the Right Tools and Technologies

To effectively implement a culture of experimentation, having the right infrastructure is essential. This means investing in tools and technologies that enable setting up experiments seamlessly, data analysis, and the rapid iteration of ideas. This is why ABsmartly focuses so much on collaboration, best practices and the cultural side of building an experimentation program, in addition to being one of the best A/B testing tools on the market, allowing unlimited A/B tests which allows organizations to facilitate experimentation at all stages of the decision making process. .

Celebrate Successes and Learn from Every Outcome

Finally, an experimentation culture thrives on recognition and learning. Celebrating successes, no matter how small, fosters a positive environment that makes team members excited to keep pushing boundaries. Similarly, every experiment, whatever the outcome, provides important insights, and digging into these results often leads to improvement and innovation.

The Impact of Experimentation Culture

Under Jonas' leadership, ABsmartly has become a beacon for companies looking to embrace a culture of experimentation. Their results speak for themselves: organizations that adopt this approach see improvements in agility, faster time-to-market, faster time to revenue and savings, and, most importantly, a deeper understanding of their customers' needs and preferences.

A culture of experimentation reaches well beyond just business metrics. It promotes a workplace environment where creativity is nurtured, collaboration is prioritized, and teams are invested in the company's mission..

Have the Right People in the Right Seats on the Bus 

First Who, Then What, otherwise known as get the right people in the right seats on the bus, is a concept developed in the book Good to Great by Jim Collins. In the book he notes that the people who build great organizations make sure they have the right people in the key seats on the bus before they start driving. Jonas acknowledges the truth in this when you’re building or scaling a tech company. Not only do you need to have experienced people on board, you also need to make the time investment required to build a culture that supports rapid implementation of new strategies. 

Any culture needs to be baked into the structure of a company, but you, as a founder also need to leave enough elasticity for the culture to evolve as the organization grows. As Lukas Vermeer says in this article, it’s like an engine, “the flywheel starts spinning slowly, and then you need to start making it go faster and faster and faster by working on all those parts of the wheel.”

The Future Is Experimental

Jonas' vision for ABsmartly wasn’t just about creating a A/B testing platform; it was about building a movement toward a more experimental, data-driven approach to business. In fact, one of their core company values is Experimentation. “We push higher, embracing innovation and experimentation through a fast-fail mindset over the status quo.” as taken from their company manifesto that all team members sign-off on. The other ABsmartly company values are Empathy, Energy and Excellence. They aspire to embrace and embody these 4 key values that help build a great company to work for and a great platform for customers to succeed in building their experimentation culture.

As we look to the future, the principles championed by the ABsmartly team serve as a roadmap for companies keen to stay ahead of the curve in an ever-changing landscape.

Join the conversation on Linkedin by following Jonas and ABsmartly.