The Keys to Successful Concept Testing: Testing

This is the last part of a three-part series on how to successfully conduct concept testing with users. Check out part one (planning) and part two (prototyping) to learn more.

Test Your Concepts

You’ve got your plan, you’ve got your prototype — now you’re ready to test. There are a few important things that will help testing run smoothly and bring the insights you need.

Prepare ahead of time

Any user researcher could tell you that nothing goes as planned on the big day, especially when you have prototypes. But there are ways to set yourself up for success ahead of sessions.

First, test your prototypes ahead of time. Running through them more than once is important to catch any issues beforehand. Chances are something will break or not work as you expected. Be sure to have a backup plan in case the prototypes stop working mid-interview. This can include having lower fidelity versions of concepts at hand (i.e., earlier sketches or still images of the wireframes) or relying more on discussion for that interview.   

Then, conduct at least one pilot session beforehand. A pilot session is conducted at the onset of your research and helps you:

• Ensure the flow of your session makes sense.
• Uncover gaps in your thinking, and potential changes to the research guide. 
• Identify issues with your prototype or potential areas of confusion.

Conducting a pilot session with a real participant or someone that isn’t familiar with the project mimics what an actual session might be like and gives you the opportunity to course correct. 

Mitigate your bias 

Bias can easily skew your concept testing feedback if you aren’t keeping it in check. Confirmation bias, for example, is one of the most pervasive forms of bias in user research. In confirmation bias, people subconsciously notice or analyze information in a manner that validates their existing beliefs. We all like to believe our initial assumptions were actually correct, right? 

It’s important to be aware of you and your team’s biases before research sessions, so that you’re heading into testing sessions with open eyes and ears to unexpected information. You can mitigate bias by: 

• Detaching yourself from concepts and any expectations for how you want research participants to respond to those concepts. 
• Having team members observe sessions so findings can be informed by multiple perspectives.
• Recording sessions so you have an unbiased reference. 
• Avoiding leading questions like “How much do you like this concept?” or “What did you think of this new, improved idea?”

Bias can also creep into your research in unintuitive ways. If you’re testing multiple concepts in one session, you’ll need to present concepts in a different order across sessions to avoid order bias. Order bias is the tendency for people to prefer options or make decisions based on the order in which information is presented. This can lead to biased decision-making, as participants may give greater weight to information presented first or last rather than considering all the information equally.

Let’s imagine that I’m testing Concepts A, B, and C. If I present these concepts in the same order for all my participants, I may be giving more weight to Concept A or C, and leaving out B. By randomizing the order of the concepts, you’ll reduce the potential for bias. 

Build rapport with people  

At the heart of it, concept testing is all about learning from the people you’re designing for and with. During testing, your research participants shouldn’t feel like lab rats. It’s important to let them know that you value their time, perspective and life experiences. 

One way you can do this is by building rapport with your participants. Building rapport through a concept testing session can look like:

• Warm up questions to ask more about who they are (ie “tell me more about what you do”).
• Reinforcing that you’re interested in their perspectives and they’re welcome to vocalize any feedback (the good, the bad, the ugly.)
• Confirming that you’re testing the prototype, not them. This is especially important in moments when you can’t answer their questions about how the prototype “should” work.

Uncover the deeper need

Creating user-centered testing also means leaving room to understand your users’ needs and getting to the root of those needs through concept testing. You can do this through a research strategy called the “5 Why’s”. The 5 Why’s is a strategy where the moderator follows up with several why’s (doesn’t have to be exactly five) to help gain a deeper understanding of what someone actually means or why they hold a certain belief. It can be particularly useful in an interview with a quieter participant.

Let’s imagine that your team is redesigning a food delivery app and you’re testing completely different concepts. In practice, a “5 Why’s” conversation with a more reserved participant could look a little something like this in the session —

• User: “I think this concept is a lot better.”
• Moderator: “Oh, interesting! Why? What makes you say that?”
• User: “It feels more... socially engaging, compared to other concepts.”
• Moderator: “Socially engaging? Can you tell me a bit more about what you mean by that?”
• User: “Yeah… I can see what my friends are ordering and which dishes they’re into. It’s not just about me deciding on my own.”
• Moderator: “Why does being able to see what your friends are ordering matter to you?”
• User: “It makes it feel like we’re eating together, even if we’re not. We’re still sharing an experience, you know?”
• Moderator: “Why is that kind of connection important to you?”
• User: “Well, sometimes I feel bad about spending money on takeout. But if I can make it more about connecting with my friends, it feels... less wasteful, like there’s still value in it.”

In this discussion, we learn that this participant feels guilty ordering takeout, but that some of that guilt is relieved if takeout becomes a way they can connect with friends. The “5 Why’s” strategy helps better uncover your users’ beliefs, behaviors and needs, not just simply preferences.

It can be easy for teams to quickly jump into concept testing. Rushing in without purposeful planning, prototyping and testing can create challenges later on in a project. By keeping these key concept testing tips in mind, you’ll set your team up for a smoother process and leave testing sessions with clear, actionable next steps. 

Have any questions about how to test out your new ideas? Feel free to email us at business@viget.com.