Concept Testing: How to test a concept without killing it

  • SumoMe


Secret #7: The psychological atmosphere of the group is as important as the concept statement itself.

Tell participants clearly what the task is, how you want them to approach it, and make it desirable and safe to do so. Set up an atmosphere of psychological safety, trust and constructive helpfulness. Make it clear that they are there to help you develop the basic idea into something genuinely useful to them. You want their suggestions and constructive criticisms.

You are interested primarily in their reactions, not as much in how they think other people would feel, although the latter also can be helpful.

Certain kinds of people, particularly physicians, have been trained to think scientifically, but not hypothetically and creatively. If the concept is an undeveloped idea, make it clear that you will be asking them to make certain assumptions without supporting data.

The point here is that you have to make it safe for people to think creatively and hypothetically. Otherwise, you run the risk of people saying, “How can we comment before we know the exact characteristics of the product?”

In order to further encourage psychological comfort, I stress how important it is for them to express any dissenting opinions or contrary experiences, even if they are a minority of one. It is important to the client to hear any negatives that exist. So, I am perceived as a neutral third party with no vested interest in anything other than an accurate and thorough discussion. In other words, make it OK to express different views.

Secret #8: The role of the researcher in concept development is highly creative.

The job of the researcher in concept development is not only analytical, but extremely creative. The researcher’s mission is not just to dissect what is, but build what can be.

The first groups usually start out skeptical

Almost every good concept starts off with strong skepticism, which can be devastating to the client unless the process is understood. Sometimes concept testing is unfortunately stopped at this initial flood of objections. However, objections are actually desirable, because in the first sessions you want to understand people’s concerns. In fact, I am very suspicious in the rare cases of concepts which are extremely positive from the beginning. I worry that the participants are getting carried away by the enthusiasm of the moment, and are not realistically considering usage questions. That’s when I play devil’s advocate and make them sell me.

Feed back the comments of earlier groups to new groups.

As successive groups begin to become more positive, bring up the negatives encountered in former groups and ask them how they would answer such objections.

Pay particular attention to opinion shifts.

When a participant changes from positive to negative or vice versa, make sure that you find out why. Often, this will provide you with the key to making the concept work.

Watch for “Hot Buttons” and “Achilles’ Heels”

Occasionally, you will find something about the product which is so positive or negative that it overwhelms all other issues.The most important positive response you can get to a new concept is the “Aha response.” When you describe the concept, people say “Aha, that’s what I need.” It is often accompanied by a sort of satisfied laughter. It is exactly the same gratifying realization that people get when they make any kind of discovery, or hear a joke with a message to be learned. You’ve hit a hot button.

For instance, I have worked with some medical products which physicians immediately believed would become “standards of community care.” They believed that they would have to use the product to defend against the malpractice risk of not using it. While situations like this are the dream of every product manager, these cases are rare. On the negative side, I can think of several Achilles’ Heels which made further development of products inadvisable. The reason that something is an Achilles Heel is usually because there is a fundamental contradiction inherent in the product which cannot be designed around. One was a bank check which would allow people to write in several payees at the same time. So, you could pay your rent, car payment, and several credit cards with the same check. This idea ran up against peoples’ practice of waiting a period of time before paying bills or writing checks before they had transferred funds into their account. There was no way at the time to design around this problem, so the concept was killed. Another time, a particular packaging was designed for unit dose ophthalmic products which could be used during eye surgery. The product was not able to be opened under operating room conditions by the ophthalmic surgeons, who are among the most dexterous people alive! An entirely new package concept had to be developed.

Secret #9: Plan a sufficient number of groups to refine the concept

The kind of development process I have been describing usually requires 4-12 sessions, depending on how many market segments are involved. I realize that the $20,000 to $100,000 required to do this is way beyond the budget of most companies. I’ll offer some alternative methods later. The attitude, “Lets just run one or two focus groups and see if we have anything,” is usually asking for trouble because negative feedback will kill a lot of great concepts that can be easily be fixed. Here’s an example of one of the most successful concepts in business history that would have been killed, without doubt, if the initial feedback was taken as a failed “test.”

I was asked by the ad agency of CitiBank to test the idea for the first automatic teller machine (ATM). They wanted to do it on one group, but didn’t really know if they were testing a card, a machine or a service. I recommended six groups because of the complexity of the issue.

The first two groups were among the most negative I have ever conducted, as only New York groups can be. The participants attacked the concept, the bank and banks in general. When I went into the client room after the second session, it was like going to a funeral, which is exactly what it could have been (for the concept). I pointed out that the session went exactly as I predicted. I told them what we had really learned and together we were able to successfully reposition the concept for the next groups. We put into the concept statement a guarantee that the bank would rectify mistakes, a security code the bank employees themselves couldn’t access, a phone to speak to a human, a friendly screen (“computer” was a dirty word in those days). The next two sessions became more positive as we kept learning. We then took all that we had learned in the first four sessions and essentially wrote a brochure offering this service to the next two groups. At the end of these sessions (the 5th and 6th) people were so positive that they were angry that I couldn’t sign them up immediately!

The bank felt confident enough to install the machines and get a jump on the competition in this highly competitive industry. They could not put them in fast enough. If we had viewed this as a “concept test,” rather than “concept development,” and conducted only the first one or two groups, you can see what could have happened.

Secret #10: Include influencers, not just the ultimate users

Your product often has to be “bought” psychologically first by someone else than the person who ultimately buys it. Your concept may live or die by the recommendations and word of mouth influencers such as medical experts and specialists, purchasing managers, distributors, department store buyers, store clerks, engineers, hospital administrators, and even insurance underwriters. The video tape recorder is one example, The Holter Monitor is another (see Case Studies section).

Secret #11: Use a consultant who is skilled in concept development

For concept work, you want someone who has been through enough successful product introductions and seen enough failures to have a basis of comparison. This usually requires many years. The moderator should have extensive experience in creative idea generation sessions, using a variety of methods, particularly CPSI, Synectics, Morphological Synthesis, Brainstorming, and the like. Such experience will allow the moderator to get the participants to modify parts of the concept which are not working.

Want help? Click here.

Alternative methods to focus groups

There are an increasing variety of alternatives to focus groups. Ideally, you want a method that is a live, interactive means of developing concepts. This is a complex area that could be the subject of several articles. I’ve used user groups, advisory groups, forums, online focus groups (which I’m not keen on), listserve groups, plain vanilla group email, and individual interviews. If you are a large corporation with the money, I’d stick with face-to-face or telephone focus groups. If not, you need a less expensive alternative , all of which have more pitfalls than even focus groups. Picking the right research methodology is crucial and very situation dependent. I can help you sort it out for your particular situation.

A note to entrepreneurs: Before you mortgage your house on that great new idea, please realize that your idea is in its infancy. It needs loving nurturing, which you are giving it. Bring it to a neonatal specialist, a pediatrician and a child development specialist. You are doing due diligence with patents and service marks, looking at engineering feasibility, getting site names, and consulting lawyers and accountants. Please get a consultation as well with a specialist in concept development and new product/service marketing. That would be me 😉 .

Next, we’ll look at pitfalls to avoid.