Back to Insights
Paul Hamshere

Test & Learn Culture (Success Factor 3 of 20)

Adopting lean ways to validate across the product lifecycle will ensure long-term success.

Test & Learn Culture (Success Factor 3 of 20)

What is the ideal outcome and what benefits will it bring?

To ensure your customers are engaged ongoing and you meet a higher proportion of your product business cases adopting a test & learn culture in the early stages (and throughout the product life cycle), is fundamental. By empowering product teams to foster a mindset of experimentation, organisations can ensure that the products built will be more aligned to customer needs and ultimately, business success.

Leaders that sponsor this approach, create a culture of less talking, and more action, with decisions that are based less on opinion and more on evidence.

What changes have we seen in the market?

Recently, there has been a significant shift towards embracing a test and learn culture in the market. In the past, large enterprises prioritised meticulous documentation and strict adherence to predetermined plans. Persevering was considered the sole path to success, even if the needs of the business and customers had changed since the final approval. Unfortunately, this approach resulted in a high failure rate for newly launched products within their first year in the market.

In response to the surge in digital technologies and the growing competitiveness of the market, organisations have acknowledged the pressing need for a dynamic and data-driven approach.

Consequently, there has been a notable cultural shift where persevering is no longer the only path to success. The decision for teams to pivot or perish a product, or change after validation, is as valuable for long term success. Numerous tools and approaches have emerged to support this mindset, including new upfront quantitative and/or qualitative research techniques, pretotyping (e.g., fake front doors), prototyping, A/B testing, Low-Code Development, and utilising AI to increase productivity across the product development process.

What successful examples have we seen?

According to Gartner survey analysis, organisations that significantly outperform their competitors are almost twice as likely to make testing and experimentation a priority. This highlights the vital role of testing and experimentation in driving competitive advantage, and serving more value to the end customer.

What types of test & learn strategies do these successful organisations implement?
  1. A/B Testing

Salesforce partnered with one of our partners Optimizely, to accelerate experimentation leveraging A/B testing. This has enabled Salesforce to become the customer success managers, answering more complex customer questions by empowering their product teams through the adoption of best practices. For growing and scaling organisations, Optimizley’s A/B experimentation tool is the key to continuous customer experience improvement and delivering the best business results.

“Experimentation is a core value to those of us on the digital experience team and Optimizely is a key partner for us. I think one of the major things that I have seen since we partnered with Optimizely is the number of tests we do, the sophistication of those tests.” – Eric Stahl, SVP Digital Experience at Salesforce
  1. Pretotyping

Airbnb created a fake front door website with photos and descriptions of their own apartment, positioning it as an opportunity for users to find affordable accommodations. By pretotyping the idea, they were able to mitigate the risk of over-investing in a product that might not have delighted users, and reducing the potential financial and reputational risks. Also, by involving users and gathering real-time feedback, they ensured that their final product would align with user expectations and provide a delightful experience, increasing the likelihood of user adoption and retention.

Where should you start?

  • Does senior leadership support an experimentation mindset and culture? If not, why?
  • How much priority is given to upfront research and validation before progressing to build?
  • Do you possess rapid optimisation capability for testing product micro changes that benefit both the business and customer experience?
  • How many of your product business cases fail and why?
  • How often do you pivot from your agreed plans or not progress at all with products or ideas (i.e. Pivot or Perish)?
  • At what point are you currently positioned on this maturity scale, and what benefits would your customers and business gain by advancing further?

Test & Learn Culture is one of IE’s 20 success factors. To learn more, read our first article, ‘Winning in a Product-Led World’.

Looking to solve big problems? Let’s talk.

Partner With Us

Stay in the loop

Get occasional newsletters about IE’s insights. We won’t spam your inbox.
Thank you! You've now been subscribed.
Something went wrong while submitting the form.
IE recognises the Aboriginal and Torres Strait Islander peoples as the Traditional Owners of the lands on which we work, and we acknowledge those communities' continuing connections to their lands, waters, and cultures. We pay our respects to their Elders past and present.