The 9 Rules of Design Research

Lately, I’ve noticed a lot more ambient enthusiasm for research among both early stage start-ups and established organizations. Businesses have embraced the idea that meaningful innovation requires understanding their customers as humans with complex lives.

This is fantastic.

I’ve also been hearing quite a few of the same myths, misperceptions, and hedges repeated. So, in the interests of being helpful—because I do like to be helpful—here is a snackable listicle of simple correctives designed to share far and wide (I’ve been assured that research proves readers enjoy snackable listicles. And puppies.)

1. Get comfortable being uncomfortable

“All I know is that I know nothing.” — Socrates

We’ve all been brought up to value answers and fear questions. We were rewarded for right answers at school and we are rewarded for bright ideas at work. No wonder so many people look for reasons to avoid doing research, especially qualitative research. Anxiety around looking less knowledgable runs deep. At least quant stuff has the comforting familiarity of standardized testing. Maintaining a research mindset means realizing that bias is rampant, certainty is an illusion, and any answer has a short shelf life. A good question is far more valuable in the long run. And you can’t ask good questions—meaning you can’t learn—until you admit you don’t have the answers.

2. Ask first, prototype later

“If we only test bottle openers, we may never realize customers prefer screw-top bottles.”—Victor Lombardi, Why We Fail

So, of course there is a rush to prototype and test the prototype. A prototype is an answer, and it’s tangible, even if it’s simply a sketch on paper. This is comfortable, much more comfortable than just asking questions, even if it is tantamount to setting a large pile of money on fire. To anyone concerned about demonstrating their value by making fast, visible progress, simply asking questions feels as productive as a raccoon washing cotton candy.

The danger in prototyping too soon is investing resources in answering a question no one asked, and ignoring the opportunity cost. Testing a prototype can help you refine an idea that is already good, not tell you whether you’re solving the right problem. And it’s easy to mistake the polish of a prototype for the quality of the idea (*cough* Juicero *cough*). FWIW, it’s also easy to mistake the gloss of a research report for the value of the insights.

Instead of saving and defending weak ideas, asking the right questions helps you identify and eradicate bad ideas faster. You just have to be strong enough to embrace being wrong.

3. Know your goal

Asking questions is a waste of time unless you know your reason for doing so in advance. And you have to publicly swear that your reason is not “to be proven right.” That is everyone’s secret goal. See #1.

Often, in the enthusiasm to embrace research, teams will start talking to customers without a clear, shared goal. And then afterwards, they feel like they spent precious time with no idea how to apply what they learned, hence nothing to show for it. This leads to statements like “We tried doing research last year and it was a waste of time.” And, thus, a return to the comfort of building and testing. Or, they walk away with different interpretations of what they heard, which leads to more arguments about who was proven right.

In large organizations, the unspoken goal is sometimes “demonstrate a commitment to research while allowing our product leaders to do what they want.” This might sound cynical but I’ve talked to many skilled practitioners in well-funded research departments who generate magnificent reports that have zero impact on decision-making. Acknowledging this happens is the first step to stopping it.

It is perfectly fine, and a great place to start, for your goal to be “We need to level-set and quickly understand the perspective of people who aren’t us.” Just don’t tack on other goals after the fact.

Only after you have a goal will you know what you need to know. And you have to know your question before you can choose how to answer it.

4. Agree on the big questions

“At its core, all business is about making bets on human behavior.”
— The Power of ‘Thick’ Data, WSJ

The quality of your question determines the utility of the results. Asking the wrong question is the same as prototyping a solution to the wrong problem. They will both give you something other than what you need. Start with your high-priority questions. These come from the assumptions or areas of ignorance that carry the most risk if you’re wrong.

The big research question is what you want to know, not what you ask in an interview. In fact, asking your research question directly is often the worst way to learn anything. People often don’t know or are unwilling to admit to their true behaviors, but everyone is really good at making up answers.

Design research gets conflated with user research all the time. Talking to representative users is just one of many ways of answering high-priority research questions. Not everything you need to know is about users.

Often the most critical question is a variation of “Based on evidence, what do we really know about our customers/competition/internal capabilities?” This can be a particularly terrifying one to approach in total honesty, but you should be able to answer it within the hour.

5. There is always enough time and money

When research is defined as a type of work outside of design, it’s easy to define gathering evidence as something extra and find reasons not to do it.

Often, teams have to ask permission of someone with authority in order to do work that is categorized as research. Asking questions is inherently threatening to authority. If you’ve ever worked with a leader who was resistant to doing qualitative research as part of a million dollar project, ask yourself whether they would skip doing their own research before buying a $50,000 car. Stated objections are often cover for a fear of being undermined or proven wrong or not looking productive in the right way.

If you are clear and candid about your goals and high-priority questions, you can learn something useful within whatever time and budget is available to you. Find studies online. Go outside during lunch and observe people. Usability test someone else’s product. Get creative.

Just avoid doing surveys.

6. Don’t expect data to change minds

“It is difficult to get a man to understand something, when his salary depends on his not understanding it.”—Upton Sinclair

This is often a hard one for highly-trained, specialist researchers to embrace, even though research has demonstrated it to be true. If you are used to working with a community of peers who value a certain kind of data, you may be ill-equipped to convince people who reject it out of hand. And it can feel insulting to one’s professional competence that the data is not enough.

The whole point of gathering evidence is to make evidence-based decisions. If that evidence undermines or contradicts the ideas of beliefs of the person with authority to make decisions, they will find reasons to reject it or ignore it. This is also at the heart of why qualitative researchers have a hard time in some engineering-driven organizations. People who are comfortable and competent with numbers want answers in numbers, even if the question demands something more descriptive.

So you have to turn ethnography inward and learn how your peers and leaders make decisions before you try to use data to influence those decisions.

7. Embrace messy imperfection

“We’re fickle, stupid beings with poor memories and a great gift for self destruction.” ― Suzanne Collins, Mockingjay

Human lives are messy. If people didn’t have problems, there would be no need for products and services to solve them and we wouldn’t have jobs. Figuring out the best way to solve problems for people requires some time out in the real, messy world and letting go of a certain amount of control. While an ethical, sufficiently rigorous approach is necessary, there is no qualitative clean room. A clear goal and a good question can withstand all sorts of unpredictable conditions.

The desire for tidy, comfortable activities that look and feel like expertise made visible leads to the inappropriate use of focus groups, usability labs, eye-tracking, surveys, and glossy reports when something much less formal would be much more effective.

Incorporating evidence into design decisions is itself a learning process. You will never find the right answer and be done. If the process is working, you will continue to make decisions with increasing levels of confidence.

8. Commit to collaboration

Everyone working on the same thing needs to be operating in the same shared reality. The people making decisions about the product need to be the best informed. It doesn’t matter how good the knowledge is, if it’s only in one person’s head (unless you are in London and that person is your cab driver).

Research without collaboration means that one group of people is learning and creating reports for another group to acknowledge and ignore. Knowledge leaks out of even the most well-meaning teams working like this. Collaboration without evidence means everyone has tacitly agreed whose personal preferences win. Neither of these is the most productive approach.

Directly involving the people creating the product in asking and answering the questions is the most productive approach. Plus it’s fun. And there are several ways to accomplish this depending on the organization.

The whole point of asking questions is to establish a shared framework for making decisions so that you can make better decisions faster. I run a workshop on this. It changes lives.

9. Find your bias buddies

“We can be blind to the obvious, and we are also blind to our blindness.” ― Daniel Kahneman, Thinking Fast and Slow

So, you did the work and you found some answers. Now you need to decide what they mean. When it comes to interpreting the results of research, collaboration becomes particularly critical. Everyone with a human brain is burdened by human biases. And there is no way to sense one’s own. We all see what best fits our existing beliefs. So, we have to refer to an external standard (including the pre-established goals and questions) and work together to check each other.

This has nothing to do with how smart or how well-informed you are. Once you accept this, and as long as you work in a team that evinces psychological safety and mutual respect, it can be a fun game to identify biases and call them out.

The Wikipedia page has a nice list, along with the Cognitive Bias Codex to print and post on your wall.

Maybe, just call it design done right

In sum, what we’re talking about when we’re talking about design research is really doing evidence-based design. Creation, criticism, and inquiry are all integral parts of the design process. Separating them leads to optimizing for the wrong things out of ignorance, ego, or fear.

Design is an exchange of value. You have to ask what people really need and value and what business value you expect to get in return, before putting anything at all into the world.

It doesn’t matter what questions you ask or how you find the answers, as long as you are ethical in your approach, honest about what you know, and apply yourself towards a worthwhile goal. There is no one right way and no one right answer. Enjoy the uncertainty! It never ends.