Hire for These Five Traits to Grow an Experimentation Culture Like Booking.com

Building a world-class experimentation culture is all about finding people with the right mindset. After they’re hired, you’ve got to give them the tools and training they need to succeed. Then, get out of their way.

When I worked at Booking.com, we knew the company grew fast because we experimented. A lot. Similar tech giants (e.g. Meta, Spotify, Google, Airbnb, etc.) grew exponentially with experimentation, too.

When you A/B test every idea and each line of code, you catch bugs and bad ideas before they stunt your growth. Plus, you discover unassuming great ideas that could otherwise be ignored. Then, you amplify their impact.

This double-whammy of “less bad/more good” is how experimentation creates growth. But you need people willing to make decisions with trustworthy data for it to work. And, unfortunately, not everyone has what it takes to put all their work to the literal test.

So, how do you find people with an experimental mindset? Glad you asked.

Five Traits that Signal an Experimental Mindset

I was a hiring manager under legendary Booking.com design leader, Stuart Frisby where I helped the design team grow from 30 designers to more than 300. That means I interviewed hundreds of people, and most importantly—I saw how my “new hires” performed after joining. It was a learning loop that refined my hiring approach throughout my nine years at Booking.

So, here are five critical character traits I look for when building an experimentation culture…

1. Curiosity: Can Your Candidate Push Ideas Further?

As world-renowned education philosopher, Sir Ken Robinson, once said: “Curiosity is the engine of achievement.”

When you’re not around to tell people what to test, you need a team that does it themselves. That makes curiosity the fuel for every good hypothesis. Good experimenters don’t simply launch a test and let it run. They push.

• They push beyond the fear of a risky idea to learn what actually matters.
• They push the boundaries of an idea to increase their chances of detecting (and sometimes flipping) an effect.
• They push beyond an idea to figure out the “why” behind a result.

For example, at Booking.com during customer interviews, people always mentioned the price on the search results page was too small. “I can’t find it quickly and easily,” they’d say. Myself and other designers tested increasing the price size by a few pixels, usually with negative results.

It was baffling that what people said they needed led to fewer bookings. But because we practiced “null hypothesis testing,” unless we had strong evidence to show that increasing the price was the right thing to do, we’d leave the price as it was. So, the problem persisted.

It wasn’t until Isaack Okello Brian joined the design team that we finally solved the price display problem. Like other designers before him, he noticed the price should be bigger. But instead of exercising restraint and making it just a few pixels bigger, Issack quadrupled it.

Everyone around him raised an eyebrow, but no one stopped him. Why? Because they were curious! And pushing ideas was the norm.

“Curiosity is the engine of achievement.”

– Sir Ken Robinson

The result? Isaack’s test grew bookings because he wasn’t afraid to push a concept further than everyone else. And because the rest of us were curious enough to learn if his bold idea would work.

How to Assess a Job Candidate for Curiosity

• What’s one interesting thing you’ve learned about lately? How did you learn about it?
• What do you like to do in your free time? Why do you like it?

2. Bravery: Being Scared But Doing It Anyway

Like Isaack’s story taught us, you can’t find unlikely improvements without a bit of guts. Bravery means being willing to test a bold, controversial idea to see what happens. This trait separates the people who create new growth from the people who only protect existing revenue.

How to Assess a Job Candidate for Bravery

Ask, “What’s the most ‘out there’ or controversial idea you’ve ever proposed for a test? Why did you think it was worth the risk, and how did you handle the pushback?”

3. Humility: Prioritizing Truth Over Ego

This is the hardest one for high-achievers. In a testing culture, your “best” ideas will fail. Often and hard.

Humility is the ability to prioritize the truth over your own ego. If you can’t admit your favorite feature kind of sucked, you’ll cherry-pick data just so you can look right. (People do this, even if they think they don’t. It’s just human nature.)

Humble people value intellectual honesty over their opinions. And good experimenters choose to go with the data—even when it says they’re wrong.

How to Assess a Job Candidate for Humility

Ask them to tell you about an idea that crashed and burned. Look for someone who speaks about the failure with scientific detachment and genuine interest rather than shame. Bonus points if they get excited about how wrong they were and what they learned from it.

4. Resilience: The Grit to Keep Going

Experimentation is a volume game. You’re going to hit “flat” results and “losers” most of the time. Resilience is what keeps a team from getting discouraged when a string of tests don’t land.

On the flip side, there’s wisdom in knowing when to quit. Resilience means quickly picking up another test idea when things don’t work out as you hoped.

How to Assess a Job Candidate for Resilience

Ask about a period where they felt like they weren’t making progress. How did they decide when to pivot vs. when to push ahead?

5. Ethics: Honesty and Integrity In Action

Just because you can do something doesn’t mean you should. And for better or worse, experimenters have a lot of power to influence behavior.

An ethical experimenter considers not only the short-term impact, but also the long-term impact of their actions. They avoid cheap tricks (e.g. “dark patterns”) that exploit psychological vulnerabilities for a short-term metric bump. They know that sacrificing user trust is always a losing strategy.

How to Assess a Job Candidate for Ethics

Come up with an ethically questionable test idea. Ask them what they think of it and why. Listen to how they reason through the idea and frame their opinion of it. Bonus points if they understand the goal behind the idea and suggest ways to achieve the outcome in an ethical way.

Does Your Culture Feed (or Starve) an Experimentation Mindset?

You can hire the most brave, ethical, and curious people on earth, but if your company operates under “command and control,” those people will burn out (and you squander their potential).

Here are some questions to ask yourself so you don’t crush what you worked hard to cultivate:

Does your incentive structure reward “The Win” or “The Learning?”

If praise, recognition, and bonuses only come from “revenue generated,” people stop being brave and start playing it safe.

Tip: Set aside a “failure budget.”
When people know their bad ideas won’t immediately bankrupt the company, they’re more likely to take a risk. The former CTO of Booking.com, Brendan Bank, did this, and he would report on whether or not we took enough risks and were going fast enough depending on how much of it we used.

Is there space for the proverbial “The Rabbit Hole?”

Curiosity takes time. If your team is expected to ship at a frantic pace, they can’t learn deeply and go as far as they need to innovate. An experimentation culture like Booking’s gives people time to go deep into unassuming topics to see what might be there. Just like Alice from “Alice in Wonderland,” the curious minds you hire need to explore weird places not on the (road)map to discover interesting new worlds of opportunity.

Tip: Ditch output-based roadmaps.
Instead, embrace outcome-based roadmaps that focus on impact—not timeframes and tick boxes. Build in exploration time to let your employees dig into topics they find interesting to learn what value might be hiding there.

Do your leaders model “Situational Humility?”

If leadership never admits they were wrong about a product direction, the rest of the org won’t either. Situational humility is when a leader knows when to show strength and certainty to instill confidence in their employee base and when to be vulnerable and admit when they don’t know something. Employees often adjust their behaviors to align with what leaders show through their own actions care about. When leaders role model humility, it becomes easier for their employees to embrace, too.

Tip: Test some HiPPO (Highest Paid Person’s Opinion) ideas.
Let the leaders role model what graceful failure looks like. Ask questions publicly without fear of looking “silly” or “stupid.”

Final Thoughts About Building an Experimentation Culture

Tools like ABsmartly give you the data you can trust. But curiosity, bravery, humility, resilience, and ethics turn that data into a culture of sustainable growth.

When you find these people (and get out of their way), you aren’t just hiring employees—you’re hiring your company’s current and future success.

Curious to learn about a great tool to give to your curious minds? Get an ABsmartly demo.