Closing the feedback loop with a little help from your friends

I was at the Agile 2009 conference last week to give a talk based on a project we did with Choice Hotels, this is the short version of my presentation. A quick thanks to everyone who came to my talk at the conference, and to everyone who provided feedback afterwards!

I like to think that most designers accept that integrating user feedback is critical to the success of a project. I also like to think that I’m not alone in struggling to find time and budget to undertake a thorough research phase on every project. As a group, UX designers have found a lot of ways do design research quickly (and cheaply) in order to make the best use of the time they have available on a project. On a recent project we found just such a solution that allowed us to gain critical insights while overcoming hurdles that made doing user research difficult.

Marketing Saves The Day

We were working on a re-design of web presence for Choice Europe and Choice Australia, and wanted to incorporate user feedback based on our previous design work for Choice Scandinavia and Choice Canada. We were able to produce a prototype site for use in usability testing, but quickly ran into some problems:

• We didn’t speak German, French, or Spanish
• We didn’t have budget for travel
• We would have to work around significant time differences

We worked around the problem by partnering with Choice’s local marketing teams. These teams were local, had the language skills we lacked, and were more aware of local customs than we were. We provided training in remote usability methods, and working together we were able to complete almost 50 usability tests in 4 countries.

The test results gathered by the Choice marketing teams provided critical insights that we used during the design process, and provided a means for the Choice teams to get involved directly in that process.

The Fourth Way

Generally, there are three ways to gain insight into customer needs:

1. Talking to customers
2. Thinking like a customer
3. Thinking like yourself

The higher you are on the scale, the higher the quality of the feedback and insight you can gather. The problem is that we rarely have time for #1, and that we spend too much time using #2 and #3, relying on our ability to think like our customers for insight. What we realized was that we could add a fourth option by taking advantage of the fact that Choice already had teams who were interacting with customers.

Integrating Collaborative Research

If you think about it, there are lots of groups within any company that:

• Already talk to users or have access to users
• Are already working to help users
• Have an interest in getting feedback directly from users

So the challenge for every agile product team is to connect with those groups, and partner with them on research activities in order to get insight into customer needs and get timely customer feedback on their work. The process for doing so isn’t all that different from the approach we normally take to design research:

1. We identify our information needs
2. We create a research plan
3. We recruit participants
4. We do the research
5. We analyze the results

It’s in recruiting and undertaking the research that we find ready opportunities to collaborate. In order to do so, we need to create a good research plan. One of the challenges we’ll face is that the UX team has the knowledge and skills to do research, but someone else has the time and opportunity to do the research. As a result, we’ll need to share our expertise via the research plan (and a testing script) and via training.

Plan to spend time training your partner based on the methods you’re going to use, and make sure to practice. You should be doing practice runs anyways, but it becomes even more useful to do so when the researchers are inexperienced.

Once both teams understand the research plan, and are comfortable with the methods, then we can start to make use our combined efforts. Of course, how we do so will be dependent on who we’re working with.

Finding Allies

So how do we find groups in our organization that we can partner with? We’ve identified three criteria that can help you:

• Access: Who in your organization has access to contact information for your customers, or better yet, who in your organization is already talking to customers directly?
• Mutual Interest: Looking at the research questions you have, is there anyone else in the organization that would benefit from this information? Think about your project stakeholders, who will benefit if you can better meet your project goals?
• Complementary Skills: Are there teams within your organization that already have day-to-day tasks that can be leveraged as research methods?

Which groups fit these criteria will differ between organizations, but we can identify broad groups that frequently match at least one:

• Marketing: Marketing may have access to existing users via mailing lists and other customer data, and share an interest in gathering insight into customer needs.
• Customer Service: Not only does your customer service team already talk directly to users, they have a vested interest in removing barriers to customer success in your product.
• Community Teams: Like customer service, they talk directly to users, and can help identify individuals who might be well suited to a particular research project.

Once we’ve identified our research partners, we can look for appropriate ways to split up the work that needs to be done. It may be that they take over all recruiting and research activities, or one team can do the recruiting while the other does the research. Regardless, one last challenge will be getting good results.

To make sure we do get good results, everyone needs to have some understanding of what good results look like, and how to capture them. This could be audio, video, or screen recordings, it could be observations on sticky notes, or structured results forms. This is just one more reason to practice.

Does It Work?

We got great results when we put this approach into practice with Choice Hotels, but there are limits to its effectiveness. Most importantly, you’re faced with reduced control over the feedback you’re getting. You’re relying on researchers with limited training, and as a result they’ll need help from you to gather information effectively. Similarly, you’ll need to take into account your partners’ strengths and weaknesses in creating your research plan. To make this approach most effective, it will help to make it a regular part of your research efforts. This will give everyone a chance to gain experience with the research methods and data collection. Thankfully, making research a habit not only improves the results you get, but also creates a more efficient channel for getting user feedback, and can even change the way your organization approaches product development.

Customer Centric Organizations

Jared Spool likes to talk about the three things that define an customer-centric organization:

1. Vision: Everyone needs to have a shared vision for the future
2. Feedback: You need to watch your customers use yours or a competitors product at least once every six weeks
3. Culture: The company needs to embrace failure as a learning tool

By working together with groups outside of the UX team on customer research — by directly exposing these groups to direct customer feedback — you’re directly contributing to making your organization more customer centric. I think this point alone makes this collaborative approach to research worth considering.