The Customer Evidence Pyramid - A Proven Method To Build Successful Products
“To avoid decision-making blind spots, aim to sequentially increase strength of customer evidence during the product development process. Combine different qualitative and quantitative research methods to establish a three-dimensional picture of the customer problem you’re trying to solve. No one research method is perfect. All research approaches have upside and downside.”
You might also enjoy these articles:
Introduction
Not all findings from customer research are created equal.
Common forms of customer research that are used every day in organisations can be error prone, introduce bias and provide weak customer evidence. Some research methods, purely by their design, are geared to providing better evidence than other methods.
The validity of the results from the different forms of customer research can vary just by the different research methods used. That’s fine, as long as we’re aware of the pros and cons of each research method that we use.
No one research method is perfect. All research methods have upside and downside. Think about customer research methods as right time, right place.
Select the appropriate research method to meet your learning objectives, budget, and time constraints. Combine different qualitative and quantitative research methods to establish a three-dimensional perspective on the customer problem you’re trying to solve.
Be aware of the implications to evidence strength and bias when selecting your customer research method.
The customer evidence pyramid
A good way to conceptualise the hierarchy of customer evidence is as a pyramid.
While this model is based off the medical model of clinical evidence, the intent is to provide a practical model for consumer / market research.
The research methods with the strongest evidence base are located at the top of the pyramid, while methods that produce the least reliable evidence are at the bottom.
The customer research methods towards the top of the pyramid are underpinned by strong research design, quantitative analysis, and a disciplined application of the scientific method of learning. These methods include randomised controlled experiments and meta-analysis of experiments.
The systematic review and meta-analysis of randomised controlled experiments is the highest level of evidence in evidence-based decision-making.
At the bottom of the pyramid, the least reliable evidence comes from ideas, opinions, past experiences, assumptions, and anecdotes.
Practical applications for building winning products
It’s generally best to start your journey at the bottom of the pyramid to sequentially gather evidence and build understanding over time.
Remember, try to use customer research methods in combination, rather than selecting one isolated method as a cover all.
Sometimes, there can be business constraints, particularly time and budget, which may force your hand one way or another. Even still, think about how you can work within business constraints to move from a position of weak evidence to strong evidence.
Try to race to the top of the pyramid too quickly and you’ll introduce gaps in your understanding of the customer problem or opportunity space.
Think about your customer research in three distinct phases:
Bottom of the pyramid
Moving up the pyramid
Top of the pyramid
1. BOTTOM OF THE PYRAMID
I recommend starting at the base of the pyramid.
While these research methods (reports, case studies, macro analysis) provide weaker customer evidence, it’s a good way to start forming a preliminary mental model or heuristic around your opportunity space.
This can help you to quickly understand the following:
Market sizing / profitability
Market growth rates
Market share / competitive dynamics
Market drivers
Industry outlook
Customer behaviours
These forms of market research are typically easily accessible and low-cost (free). There’s no excuses.
Learn all you can quickly and cheaply.
Preliminary research findings will help to inform your hypotheses when engaging in future quantitative and qualitative research.
2. MOVING UP THE PYRAMID
I’m a big believer in “zero distance to customers”. As product developers, we should be always aiming for the shortest distance between organisation and customers.
This ensures that our product intuition is always kept up to date and we don’t develop a Perception – Reality Gap – what we think customers want and don’t want is different.
Conduct exploratory or investigative qualitative research to develop an initial understanding of customer journeys, problems, pain points, needs and struggles.
In large organisations, there’s so many rich forms of qualitative evidence:
Customer discovery
In-depth customer interviews
Group discussions
Call listening
Sales customer feedback
Customer service feedback
Marketing customer feedback
User research
Combine your qualitative research with quantitative research (surveys, questionnaires, simulations, user data, product data) as an additional checkpoint and learning mechanism.
Do the insights from the qualitative and quantitative research align? Is customer sentiment the same, or different? With a larger quantitative sample size, what changes?
The insights from your quantitative and qualitative inform your experiments and provide a base for future decision-making.
New innovations are generated from a Customer Insight or a Data Insight.
3. TOP OF THE PYRAMID
We can choose to make our decisions from intuition or data.
Experimentation is critical because it enhances our ability to make decisions by refining our product intuition with objective customer data.
Remember, we need to keep our product intuition closely aligned to our customers and the marketplace.
The top of the pyramid is where organisations frequently come unstuck. Many businesses still don’t perform experiments, with decision-makers not using experimentation data in the decision-making process.
Experimentation helps us to bring all our previous research together to understand if our solution delivers value for our target customers.
It helps us to understand where we are right, and where we are wrong. It helps us to keep learning continuously.
Experimentation and meta-analysis of experiments is the strongest form of evidence as we’re able to attribute cause and effect of customer behaviours.
In summary
To avoid decision-making and learning blind spots, aim to sequentially increase strength of customer evidence during the product development process.
Think about these five things:
Don’t be in a hurry to reach the top of the pyramid (constraints aside)
Source fast and cheap evidence to form early heuristics
Stay close to customers to keep your product intuition up to date
Combine research methods to develop knowledge over time
Use experiments to determine if customers truly value your solution
If you can understand the truth behind your customer behaviours, and validate assumptions, it will help you to make more accurate decisions, increasing business growth.
Need help with your next experiment?
Whether you’ve never run an experiment before, or you’ve run hundreds, I’m passionate about coaching people to run more effective experiments.
Are you struggling with experimentation in any way?
Let’s talk, and I’ll help you.
References:
Before you finish...
Did you find this article helpful? Before you go, it would be great if you could help us by completing the actions below.
By joining the First Principles community, you’ll never miss an article.
Help make our longform articles stronger. It only takes a minute to complete the NPS survey for this article. Your feedback helps me make each article better.
Or share this article by simply copy and pasting the URL from your browser.