Taming the elephant in the room

Brand perception and bias

I’ve been a bit of a pizza snob ever since my first job making pizza at the age of 15. I’ve always had a pretty low opinion of Domino’s Pizza®, so I was very impressed when I saw Domino’s recent efforts to reinvent both their pizza and their brand identity by tackling their problems head on. When I first saw their ads and online video I was amazed that they had chosen to use actual user feedback that denigrated their product. It’s extremely rare to see companies admit their own faults so bluntly and publicly, but the best way to deal with the elephant in the room is to acknowledge it, confront it, and ultimately overcome it. We have yet to see whether Domino’s can successfully overcome their negative brand perception, but the message they’ve sent has been enough to motivate me—someone who had previously sworn never to eat Domino’s Pizza again—to give them another try.

Their rebranding effort has brought up quite a few memories for me. I’ve seen the pain on the faces of designers and engineers when users eviscerate the products they’ve spent months or years developing. I also know convincing stakeholders they need to rethink a product can be a painful and costly endeavor. However, one of my most salient memories involves interacting with research participants who had their minds made up about the quality of a product before they ever laid eyes on it. Such preconceived notions can be another elephant in the room—a barrier to achieving accurate and actionable feedback on a concept or design.

The Angry Elephant

A short time ago, my business partner and I were giving a talk about communication and research at an ecommerce company, and someone asked a very interesting question. What was our most difficult interaction with a research participant? The question instantly brought to mind this memory: Some time ago, we were asked to conduct a focus-group session with a specialized group of participants. In this particular case, the focus-group participants had previously tested builds of the product, and their feedback had been overwhelmingly negative. Our client, to their credit, had already decided to redesign the product from the ground up and realized they needed direction. The company had chosen to bring in a fresh research team, so they hired our company to engage users, find out exactly what was wrong with the current design, and provide guidance on how they could redesign the product.

I’ll never forget the experience of walking into the room where the focus group was to take place to greet this group of participants and being met with glares bordering on open hostility. We explained that we were performing research the company intended to use to improve the design of the product, but that wasn’t anything they hadn’t heard before, and they were understandably skeptical. Our initial attempts to extract meaningful information fell flat as the participants just spat out short comments, then quickly clammed up. It was obvious we would have to win them over and show them we were on their side. Vague phrases such as we recognize that there are problems with the design were not enough to assuage their doubts. To show them we were truly ready to make serious changes, we had to do what Domino’s has done in their recent advertisements: be brutally honest about the failings of the existing design. So, after going through an extemporaneous list of the product’s design shortcomings, we were finally able to show the participants that we understood their pain, cared about their feedback, and were ready to make changes.

This kind of experience interacting with user research participants emphasizes two essential parts of successfully communicating with participants: establishing objectivity and building alliances.

Establishing Objectivity

When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design. If you have any biases or personal connections with the product, participant may feel uncomfortable giving you negative feedback, because they unconsciously sense your disappointment. For this reason, we commonly advise that designers not lead user research for products they’ve designed. If a dedicated user researcher is not an option, we typically advise companies to use a different designer who has no personal attachment to the project. When we are interacting with participants, very early on in our conversations with them, we make a point of letting them know that we did not design the product, so anything they might say will not hurt our feelings. This gives participants the freedom to offer negative feedback without the social stigma of being mean or hurtful.

Building Alliances

When taking on the role of user researcher—whether you are actually a user researcher or are a designer, engineer, product manager, or in an another profession—it is important to take to heart that, for that period of time, you are on the side of the users. A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team. In doing this, a researcher builds alliances with participants, so participants feel that the researcher—and thus, the company—values their feedback and will use it to influence the design of the product. While establishing objectivity can help participants feel free to speak openly, building alliances can motivate them to participate actively in a user research session. It’s another strategy you can use to maximize the amount of actionable user feedback you’ll get from a research session.

The Happy Elephant

Of course, preconceived notions about the quality of a product can also swing in the other direction. A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product, because participants have already made up their minds to love the product before they even lay eyes on it. We’ve tested products with participants who were brand loyalists and overlooked numerous faults in a product’s design. In one particular instance, a participant had difficulty performing a majority of the tasks during a usability test, but reported that he loved the design, so we shouldn’t change a thing. It was only through our pointing out and inquiring about specific areas of difficulty he had encountered that he finally admitted that there were aspects of the user interface that should change. There are two methods that come to mind for dealing with this type of issue: anonymous testing and prioritizing objective data.

Anonymous Testing

The easiest way to overcome such brand bias—whether participants have positive or negative perceptions of a company’s brands—is to simply avoid it entirely. If it is possible to conceal the identity of the company developing the product, it can be useful to do so. Often, by not associating a brand with a product or concept, we can acquire a more accurate understanding of users’ reactions to the product itself. As consultants, we can accomplish this by performing user research under our own company name. We can bring participants to our offices and use prototypes and builds that show no company logos. It can be much more difficult to establish anonymity if we are testing at our clients’ offices, but, of course, it isn’t always necessary to do so. We do usually suggest some type of anonymous testing when we’re working with companies that are trying to overcome a strong negative or positive brand perception. However, anonymous testing isn’t necessary for companies that are less well-known or have a more neutral brand perception.

Prioritizing Objective Data

Objective data refers to information user researchers collect through direct observation of participants, while subjective data refers to data researchers collect through participants’ verbal reports. In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research. For example, during a usability test, we note specific instances in which a participant has difficulty with a device or user interface. Then, when interviewing the participant, we typically make statements like this: It’s great that you loved the Web site, but you seemed to have difficulty doing XYZ. To which a participant usually responds by acknowledging the difficulty, then making comments about their expectations or suggestions for improving the user interface. There are also ways to objectively document emotional responses such as frustration, disgust, and happiness. Learn to recognize such emotional cues, document them, and inquire about them when doing any type of product research, including generative research such as ethnography. Typically, objective data is more accurate than subjective data, so it is important to prioritize data accordingly when findings are inconsistent.

Conclusion

Participant biases, both positive and negative, can get in the way of obtaining useful feedback when performing concept or usability testing. There are a number of ways to work around these biases, as follows:

· Establish objectivity. Show participants that you are not personally attached to a concept or design, allowing them to provide feedback without hurting your feelings.

· Build alliances with participants. Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them to use.

· Use anonymous testing. Filter usability testing through a subsidiary or a contracted agency to avoid any brand bias entirely.

· Prioritize objective data. What users actually do is more important that what they say they do.

If you take these lessons to heart and apply them appropriately, they can be enormously helpful in making some of your more difficult interactions with users much more productive.

This article was first published as a column on UXmatters.com