Welcome to the UX Myth blog series, where I debunk common design misunderstandings you keep hearing about in product meetings (and attempting to fix them).
In 1985, Coca-Cola (or Coke) introduced a new reformulation of its iconic drinks, dubbed “New Coke”. It went through thousands of “sip tests” and many iterations to carefully adjust the taste based on the supposed “customer” feedback. Despite all the positive feedback from testing before the initial launch, the New Coke turned out to be a total disaster -- causing the company to lose millions of dollars in return. To combat the worldwide dissatisfaction, the parent company had to rename the original formula “Coke Classic” and discontinue the New Coke altogether in late 2002.
That serves as a cautionary tale about many things, one of which is how to conduct user research properly.
What’s The “Real” Problem For Those Tests?
Given the above, is user research and testing still relevant for the product development process?
Yes, it is! The issue with Coke’s new formula is that people love it when they’re doing a “sipping” test. They tell the truth; those prototypes are suitable for literally a sip test where a person would take in only a tiny amount at a time. However, when it comes to chugging the entire can in one go? Not so much.
There’s nothing wrong with those feedback loops at all. They’ve just interpreted the wrong solution for the problem.
Understanding The Context
Sure, you can be literal and say the term “design” exists to solve problems. It does solve problems, just not that literally.
When a person says, “I’m hungry”, you don’t dash to the nearest supermarket and buy 40 packs of candy because you solve a “hunger” problem. You first try to understand the intention or meaning of how and why they say it, then make reasonable assumptions that could be a potential solution.
Suppose all the great researchers and pioneers focus only to “ask” customers the existing problem they want to be solved. We wouldn’t have all these innovations and groundbreaking creative inventions we have today.
"Some people say give the customers what they want, but that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, 'If I'd ask customers what they wanted, they would've told me a faster horse.' People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page." - Steve Jobs
When conducting user research and feedback sessions, test subjects make confident but false assumptions about their future needs. Make sure you know what to ask and how to digest those answers into usable insights.
What To Do?
Here are some suggestions:
Considers Efficient Insights Acquisition Methods
Not all of the research has to be a 1-on-1 interview. The amount of excessive user research using unnecessary by-the-book tools and framework could drive everyone up the wall. Spend a bit more time on planning -- how to gather those results to prove UX assumptions.
Be articulate and thoughtful when it comes to research and testing. Time is your most important resource. Ask yourself (and your team) these crucial questions before conducting any research:
Will these tasks be sufficient via a quick-and-dirty internal workshop?
What assets and resources [e.g. people, data, requirements] are already at hand? What needs to be requested?
Can we leave out any time-consuming tasks? (fully-fledged user journey with documentation, guerilla interviews, competitor’s analysis, etc.)
How can we refine and process this information with efficiency in mind?
Factor in ideas from stakeholders
Although it may seem counterintuitive at first, trusting in your team’s (and sometimes your client’s) ability to provide crucial user insights is also an essential aspect of building a strong empathy towards the product’s audience.
Make sure you take every input as a suggestion and not as an executive order. There is a thin line between listening to reason and just blindly following orders made in a walled garden.
Quickly Track And Enact Changes
No one is perfect. It doesn’t matter how many times you have carefully planned things out. Your product may fail to impress your audience, people may make errors; Creating a proper testing environment and running it through an internal review/testing that focuses on minimising the risk of unforeseen accidents could go a long way toward improving your product.
Build a standardised system to track and respond to users feedback to ensure you can get to the cause of the complaint. The most important thing is adjusting to changes and deal with those problems properly.
