“Concept testing” is probably the most valuable and challenging application of qualitative research. Although it is subtle, complex and difficult, it can be de-mystified and clarified into a set of principles. I believe that this is the first time that anyone has set forth the foundations of concept testing. 

What it is 

Concept testing is the attempt to predict the success of a new product idea before it is marketed. It usually involves getting people’s reactions to a statement describing the basic idea of the product. As such, it is usually pass/fail, go/no go. As I will explain later, this is usually a very good way to kill a concept.

A much more fruitful approach is Concept development: the gradual refinement of new ideas into a form that is most likely to be accepted in the marketplace. It not only gives promising ideas a fighting chance, it provides guidance for the communication of benefits, uses, packaging, advertising, sales approaches, product information, distribution, and pricing. 

A risky undertaking 

This is a risky undertaking for several reasons:

• People cannot be expected to know or tell you how they would behave in  the future, especially in hypothetical situations where the product isn’t even developed.

• People are often skeptical of new ideas, and occasionally hostile to them. Radically new products shake up people’s established way of doing things, putting them at or over the edge of their own incompetence. So, new ideas are unsettling at best and a threat at worst. People often do not have the imagination to see how the new product would benefit them.

These people are not likely to receive a new product with open arms.For these reasons, even “obviously” good ideas which were ultimate successes, like automated teller machines, extended battery-life pacemakers, VCRs, and many once-a-day medications had major problems when they were first described to people in the concept testing stage of development. New product ideas presented in the form of a half-page concept statement run a terrible risk of being killed prematurely.

It can be done right

Concept testing is sometimes practiced rather naïvely. All too often, a couple of focus groups are assembled, read a simple description of the concept and people are then asked for their reactions. This approach is not only naïve, it is dangerous. How many wonderful, underdeveloped, defenseless concepts were killed when a few groups of people mugged them with brutal criticism in the dark alley of concept testing?

How to do it right

Concept development is a very powerful tool when done right. It has saved hundreds of millions of dollars. It will let you avoid false starts, wrong positionings, poor strategy and selling to the wrong people. It is not only vital insurance, but more importantly, it guides you throughout the entire development process—from initial concept to successful launch.

Principle #1: An idea is not a product.

All too often, people forget that the idea is not the product. This seems obvious, but believe me it’s not ease to keep this in mind, especially for people who are intimately involved with the product. It is disastrous to believe that a new, superlative product will sell itself. You have to see your product from the point of view of your customers. A new product is unsettling to most people. It usually requires a new way of doing things.

Principle #2: For an idea to become a viable product, it must be more than just superior.

People only change to new products when they perceive that there is a significant gain. People have to believe it is more valuable than the money, time and comfort that they will have to give up in order to purchase it. You have to convince people that: it will ultimately be an improvement over what they have now, it is worth changing from what they already have, there is a relatively simple way to verify its superiority, that it will live up to its promises, plus a whole host of other issues. Marginal improvements rarely succeed in displacing a market leader. (I have developed the Decision Map(tm) to show the steps that must occur for someone to go from not knowing about a product, to becoming an enthusiastic user.)

Principle #3: It is not what you know your product is, but what others think it is, that determines product acceptance.

Even the most straightforward product will be perceived differently by different people. It can be seen from various perspectives, used for a variety of purposes, in different contexts, with different expectations. So, you can’t develop a product only on paper. For most products you must develop a prototype, to work out the inevitable bugs, because a product exists in the real world , not just in theory. In addition, your product must survive not only in the real world, but in psychological realty , i.e. the world as perceived by people ï as filtered through their beliefs and emotions. You must be people driven, not product driven.

Principle #4: To develop your product in the psychological world, you need a psychological marketing laboratory.

A laboratory is a safe place in which to try new things. There has been no better laboratory invented for new products than focus groups. Because people are motivated to communicate in focus groups, an experienced moderator can infer what is going on in their heads and hearts. What they say is important, but how they say it, what is behind what they say and what is not said is also just as important. Surveys of people voting on whether or not they would use the product are usually very misleading.

Principle #5: Concept “testing” is not a pass/fail test.

A test is something that passes or fails, or that gets a score. As I have already mentioned, you can’t take a fragile new idea and bounce it off a bunch of people. It will usually shatter. Concepts must be developed and nurtured. Concept development is a more accurate name for the process than concept testing.

Principle #6: Concept development is a process of successively refining, amplifying and enhancing the basic idea of the product.

While concept development is often conducted haphazardly, it can and should be a highly organized process:

1. Assess the present situation.You must thoroughly understand what people are doing now. You want to know their present satisfaction levels, whether they perceive a problem for which your product may be a solution, and what are their present beliefs, misconceptions, attitudes and emotions. Look particularly for the issues that they have the most energy around, rather than what they tell you is important. You want to know the words they use, and the way they categorize and conceptualize the area. More often than not, people will not make the same distinctions and discriminations the manufacturers make. Users’ concepts will be much more use and function oriented. The manufacturers’ concepts will be more product and feature oriented. I can’t stress enough that respondents’ actual words, perceptions and felt needs are what you are after. You can only take people from here to there. So, you better know where “here” is.

Sometimes people say that they don’t need the product when it is the kind of product that solves a problem that they don’t see, or are embarrassed about or don’t want to acknowledge. If so, it doesn’t mean that your concept is dead. You just have to get them to see or acknowledge the problem. Sometimes, you come up against product rejection when the respondents take professional pride in their ability to overcome the difficulties that your product will eliminate. Some examples are: automatic cameras for professional photographers, or surgical staples and glues (surgeons pride themselves on their knot tying dexterity), prepared anything (when the preparation is part of the professionalism), programmed instruction for teachers and simplified (Macintosh or Windows) computers for computer professionals.

Many concepts have been killed when such situations were discovered. But what this really means is that other benefits must be found. If your new concept indeed represents a significant advance, it must be clearly communicated.

2. Write the concept statement. This step is the most difficult. The way the concept is presented is crucial. There are two pitfalls to be avoided:

(1) Don’t make the statement just a straightforward “objective” description of the features of the product.

(2) Don’t make the statement too “salesy.” Instead, a balance must be struck between a dry spec sheet and a slick brochure.You do this by starting with a brief description of the present situation, in words the target audience uses. It is usually best to describe the concept as a solution to a problem. For instance to physicians: “You know how hard it is to get ‘patient compliance’ [physicians’ words] with your recommendations...” Or, “You know how frustrating X is...” Then, continue with the product description in benefit terms: “So, my client has developed a..., which is designed to..., without causing ...[undercutting a qualm]” Then, and only then, describe the product itself, and how the product’s claims will be substantiated.

Make it as real as you can, but if something is hypothetical or undeveloped about the product, mention this. Then, ask for their guidance in making the concept viable.

3. Refine and elaborate the statement in successive groups.The concept statement is written and rewritten, for successive groups, continually taking into account what has been learned before. Each time it is presented, you will elicit qualms, objections and concerns as well as praise, new uses and new ways of describing it. You take the positives and incorporate them into the concept statement. With the negatives, you look for ways to underplay them or turn them into benefits. This comes from the creativity of the researcher and the client. We find out directly from participants how we can make the product more appealing. We continue to strengthen the presentation of the concept. You will find that you have almost developed a sales presentation even before you have an actual product.

HOW TO CONDUCT THE SESSIONS

Principle #7: The psychological atmosphere of the group is as important as the concept statement itself.

Tell participants clearly what the task is, how you want them to approach it, and make it desirable and safe to do so. Set up an atmosphere of psychological safety, trust and constructive helpfulness. Make it clear that they are there to help you develop the basic idea into something genuinely useful to them. You want their suggestions and constructive criticisms.

You are interested primarily in their reactions, not as much in how they think other people would feel, although the latter also can be helpful.

Certain kinds of people, particularly physicians, have been trained to think scientifically, but not hypothetically and creatively. If the concept is an undeveloped idea, make it clear that you will be asking them to make certain assumptions without supporting data.

The point here is that you have to make it safe for people to think creatively and hypothetically. Otherwise, you run the risk of people saying, “How can we comment before we know the exact characteristics of the product?”

In order to further encourage psychological comfort, I stress how important it is for them to express any dissenting opinions or contrary experiences, even if they are a minority of one. It is important to the client to hear any negatives that exist. So, I am perceived as a neutral third party with no vested interest in anything other than an accurate and thorough discussion. In other words, make it OK to express different views.

Principle #8: The role of the researcher in concept development is highly creative.

The job of the researcher in concept development is not only analytical, but extremely creative. The researcher’s mission is not just to dissect what is, but build what can be.

The first groups usually start out skeptical

Almost every good concept starts off with strong skepticism, which can be devastating to the client unless the process is understood. Sometimes concept testing is unfortunately stopped at this initial flood of objections. However, objections are actually desirable, because in the first sessions you want to understand people’s concerns. In fact, I am very suspicious in the rare cases of concepts which are extremely positive from the beginning. I worry that the participants are getting carried away by the enthusiasm of the moment, and are not realistically considering usage questions. That’s when I play devil’s advocate and make them sell me.

Feed back the comments of earlier groups to new groups.

As successive groups begin to become more positive, bring up the negatives encountered in former groups and ask them how they would answer such objections.

Pay particular attention to opinion shifts.

When a participant changes from positive to negative or vice versa, make sure that you find out why. Often, this will provide you with the key to making the concept work.

Watch for “Hot Buttons” and “Achilles’ Heels”

Occasionally, you will find something about the product which is so positive or negative that it overwhelms all other issues.The most important positive response you can get to a new concept is the “Aha response.” When you describe the concept, people say “Aha, that’s what I need.” It is often accompanied by a sort of satisfied laughter. It is exactly the same gratifying realization that people get when they make any kind of discovery, or hear a joke with a message to be learned. You’ve hit a hot button.

For instance, I have worked with some medical products which physicians immediately believed would become “standards of community care.” They believed that they would have to use the product to defend against the malpractice risk of not using it. While situations like this are the dream of every product manager, these cases are extremely rare.On the negative side, I can think of several Achilles’ Heels which made further development of products inadvisable. The reason that something is an Achilles Heel is usually because there is a fundamental contradiction inherent in the product which cannot be designed around. One was a bank check which would allow people to write in several payees at the same time. So, you could pay your rent, car payment, and several credit cards with the same check. This idea ran up against peoples’ practice of waiting a period of time before paying bills or writing checks before they had transferred funds into their account. There was no way at the time to design around this problem, so the concept was killed. Another time, a particular packaging was designed for unit dose ophthalmic products which could be used during eye surgery. The product was not able to be opened under operating room conditions by the ophthalmic surgeons, who are among the most dexterous people alive! An entirely new package concept had to be developed.

Principle #9: Plan a sufficient number of groups to refine the concept

The kind of development process I have been describing usually requires 4-12 sessions, depending on how many market segments are involved. But the attitude, “Lets just run one or two focus groups and see if we have anything,” is usually asking for trouble.

I was asked by the ad agency of one of the largest banks in the country to test the idea for the first automatic teller machine in New York City. They wanted to do it on one group, but didn’t really know if they were testing a card, a machine or a service. I recommended six groups because of the complexity of the issue.

The first two groups were among the most negative I have ever conducted, as only New York groups can be. The participants attacked the concept, the bank and banks in general. When I went into the client room after the second session, it was like going to a funeral, which is exactly what it could have been (for the concept). I pointed out that the session went exactly as I predicted. I told them what we had really learned and together we were able to successfully reposition the concept for the next groups. We put into the concept statement a guarantee that the bank would rectify mistakes, a security code the bank employees themselves couldn’t access, a phone to speak to a human, a friendly screen (“computer” was a dirty word in those days). The next two sessions became more positive as we kept learning.We then took all that we had learned in the first four sessions an essentially wrote a brochure offering this service to the next two groups. At the end of these sessions (the 5th and 6th) people were so positive that they wanted me to sign them up immediately, or put them on a preferred list as a reward for being in the research!

The bank felt confident enough to install the machines and get a jump on the competition in this highly competitive industry. They could not put them in fast enough.If we had viewed this as a concept test, rather than concept development, and conducted only the first one or two groups, you can see what could have happened.

Principle #10: Include influencers, not just the ultimate users

Your product often has to be “bought” psychologically first by someone else than the person who ultimately buys it. Your concept may live or die by the recommendations and word of mouth influencers such as medical experts and specialists, purchasing managers, distributors, department store buyers, store clerks, engineers, hospital administrators, and even insurance underwriters. The video tape recorder is one example, The Holter Monitor is another (see Case Studies section).

Principle #11: Use a consultant who is skilled in concept development

For concept work, you want someone who has been through enough successful product introductions and seen enough failures to have a basis of comparison. This usually requires many years. The moderator should have extensive experience in creative idea generation sessions, using a variety of methods, particularly CPSI, Synectics, Morphological Synthesis, Brainstorming, and the like. Such experience will allow the moderator to get the participants to modify parts of the concept which are not working.

PITFALLS TO AVOID

• Product rather than customer orientation.
• Using an inside moderator, or moderator inexperienced in concept development.
• Testing, rather than developing.
• Testing things that can’t, or shouldn’t be tested, like some (definitely not all) personal products, products which must be experienced but can’t be produced in prototype, or ideas which are so vague that they cannot yet be communicated meaningfully.
• Not doing concept development until it is too late to do anything about it. Particularly true in the pharmaceutical industry where many products are tested after the clinical trials are committed to. Here, concepts should be developed prior to and during clinical trials.
• Not giving the concept enough of a chance.
• Taking votes for the purpose of projecting whether the concept will be successful (there are other legitimate reasons for votes). You can project from groups, but you must do it on the strength of the opinions, the logic of the statements, and the other factors previously mentioned.
• Taking what people say at face value. Be careful of reactions like “This will be a big seller among X’s, but I wouldn’t buy it.” Or, “I’d buy it, but no one else would.”

CASE STUDIES

The following examples illustrate these principles.

Medical devices - Principle #1

Most new medical devices contradict the established way of doing things rather dramatically. In any such situation, it is very important to understand the sources of pride and other benefits of doing things the old way.

The IBM Jr. and the IBM Convertible - Principle #1

The IBM Personal computer was one of the greatest business successes of all time. But their introduction of the PC Jr. and the Convertible were amazingly short of what consumers wanted. These monumental setbacks would probably have been avoided, and possibly turned into a success, with better concept development work. Don’t think that a series of smashing successes guarantees anything for your next product.

The joggers stopwatch - Principle #1

The first digital jogger’s stopwatch was a great idea - except that you could not switch from the stopwatch mode to see the real time, without wiping out the elapsed time on the stopwatch function. So, a jogger could not tell what time it actually was, only how long he/she had been running. The execution of the concept is as important as its abstract description. Also, it often pays to concept test your competitors’ products. You can often find a fatal flaw in an otherwise good concept, which is easily correctable.

Buccal tablets - Principles #2 & 10

These are tablets which are placed between the upper gum and the cheek, in the “buccal cavity.” The tablet slowly releases its drug over a period of hours directly into the small blood vessels in the gum. It is one of many “new drug delivery systems,” designed to bypass the stomach, where many drugs are destroyed. The need for this type of system is unquestioned, especially for the very large and fragile molecules of the newer bioengineered drugs. The first drug to use that system was believed by physicians to be unacceptable to patients, even though the few patients who were on it, loved it. There were several flaws in how the product was demonstrated, which the company was not aware of. Make sure that there isn’t an unsold segment of the marketplace, who can kill the product. Also, how the product will be demonstrated is an integral part of the product and should be carefully considered.

The Lisa - Principle #2

When Apple Computer’s Lisa was introduced, telephone groups of non competing computer store owners were conducted (not for Apple). The participants were very positive toward the Lisa and thought that it would be a smashing success. However, they were unable to support their response with a reason that made sense and could not justify its price. I concluded that their “positive reaction” was nothing more than a “Gee Wiz” reaction expressing amazement over the technological achievements of the machine. They clearly would not be able to sell it to their customers on that basis. It was clear that the machine would be a failure, but that a much cheaper version would be a success. The Lisa (which failed) was subsequently turned into the Macintosh, a tremendous success. The enthusiastic unsupportable acceptance of the concept should not be interpreted as a positive outcome in the case of major purchases.

Multiple payee check - Principle #2

A bank check with multiple payees. It contradicted peoples’ fundamental practices of check writing. While convenience is a very powerful benefit and wins more often than not (the Instamatic camera, frozen meals, etc.), it doesn’t win when it is in direct opposition to inherent preferences.

Diagnostic tests - Principle #3

I have worked with several diagnostic tests which excited physicians for reasons other than the uses for which it was designed. What could be more straightforward than a test that will confirm whether someone has a disease? But sometimes the tests were not perceived as necessary for making the evaluation. Instead, the tests were perceived as helping to decide whether to use more expensive tests and treatments, how aggressively to pursue treatment and which drugs to use. The lesson here is that as much attention should be paid to the perceived uses of the product as to its original intended use.

Enuretic device - Principle #3

A device which was to be worn by bed-wetters. It was perceived as having a potential for electric shock. There were other aspects of the product which were also perceived as offensive, punitive and degrading. Pediatricians were so offended by the product that it was clear that no amount of persuasion could have sold it. Initial perceptions can (rightly or wrongly) make it impossible for people to listen to any subsequent persuasion.

Unit dose ophthalmic - Principle #4

To simulate actual use, we requested physicians to wear their standard surgical gloves for the product test. They could not open the package. If you can’t test under actual conditions, try to simulate them as closely as possible.

A prepared dessert - Principle #4

After being tested in standard laboratory test kitchens with consumers, the dessert was tested in the real world for in-home use. We had the dessert delivered to peoples’ homes and asked them to serve it to the family as dessert for that evening’s meal. The housewives then participated in a telephone focus group immediately afterward. While they agreed that the dessert tasted wonderful and contained no preservatives, it broke down in consistency under household conditions, making it runny and watery. Mothers are not going to take the care and effort in serving desserts to a busy family that they do operating in test kitchens. Concepts which require demonstration should be demonstrated under actual usage conditions.

ATM’s (Automated teller machines) - Principles #5 & 6

Described previously. Illustrates need to not make it a pass/fail test, but a development process. Also to allow enough groups to develop the concept from negative to positive.

A new kind of antibiotic - Principle #7 & 8

In this case we went to typical physicians. They liked the concept, but “damned it by faint praise” When pressed, they really saw little need for it. By creating the proper atmosphere in the group, we were able to encourage them to speculate on other uses of the drug. They mentioned their frustrations in treating infections in a particular organ system, which was not an anticipated first indication for the drug. It turned out that the drug was extremely effective for that infection. In subsequent groups, they said that they would use it for the infection in question. It turned out that it was this new use that drove much of the initial interest in the drug. The lesson here is that you must listen for and build on what is not said, and what is said in passing (or as a joke), as much as listening for the main content of the session.

A cold preparation - Principle #9

We once worked with a cold preparation which had a different form than the usual tablet, lozenge or liquid. However, my client, a pharmaceutical firm, was not in the consumer end of the business. Because we planned enough groups, we were not only able to see that the concept was a winner, but that the marketing would need a highly specialized packaged goods approach, involving extensive positioning work and unusual advertising, which was not their primary expertise. We recommended that they sell this outstanding product opportunity to an OTC pharmaceutical company, who would be better able to capitalize on it. A winning product can be a loser in the wrong company.

Holter Monitor - Principle #10

The Holter Monitor is a portable electrocardiogram machine which is worn for 24 hours, providing a continuous record. The resulting output can be analyzed to detect cardiac arrhythmias (irregular heartbeats). When we brought together the country’s leading cardiologists to first hear about the ideas in several telephone focus groups, they went wild (the “Aha response”). They told us exactly how to design the unit, what pitfalls to avoid and what clinical studies should be provided. The lesson here, again, is to go to people who will drive the acceptance, and even more importantly, to go to them early enough to be able to do the right homework -- clinical studies in this case.

VCR’s - Principle #8 & 10

We were asked if we could help circumvent the initial unsuccessful introduction of the VCR’s. This may seem unbelievable now. We conducted telephone focus groups of buyers and top salespeople from high-volume retailers to understand why this revolutionary technical breakthrough was meeting such resistance. The manufacturer was focusing more on the technology than the application. Talking to the salespeople directly provided us with a consumer perspective. These people were uninterested in our marvelous technology, but pointed out that we had to help them sell it to their customers, the consumer. TV programs are usually not a valuable asset to be preserved. VCR’s were originally positioned to allow people to record television shows and play them over and over. Not very exciting. In the words of one salesperson,“Who wants to save garbage?”

Most people watch television and accept whatever is on. This “take what you get” constraint of TV turned out to be the opportunity. When I asked the buyers and salespeople, how they would sell a video tape recorder to me, they said that they would sell it as something which would allow me to watch whatever I wanted when I wanted to, then reuse the tape, not save it. The concept of viewing-time option was born, together with greater selectivity. Selectavision was the brilliant name conceived of later. Incidentally, the salespeople joked that it would help if X-rated movies were also available. As you know, it was these two forces that drove the tremendous acceptance of the VCR: viewing-time option and X-rated movies.The lessons to be learned here are: you must go to the people who are likely to know how to turn the idea into a successful product. Also, many a true word is said in jest.

Generic drugs - Principle #10

Some concepts are already on the market when they are reevaluated as concepts. In the early seventies, generic drugs had been around for awhile. However, focus groups of key state legislators, pharmacists and physicians revealed that it was an “idea whose time had come.” Physicians were still very much against the idea, but it was clear that circumstances were such that an overwhelming consumer demand would soon be created. The lesson here is that the prediction of the success of an idea also depends on timing. In your files, you may find many ideas whose time had just not come ï then.

OVERVIEW - KEY POINTS

The following is what should come out of most concept development projects:

• Analysis of current usage patterns.
• Who the real target audience is.
• Under what conditions people will buy, prescribe and/or use the product.
• The psychological state people will have to be in to use the product.
• A description of the product which turns people on (the refined concept statement).
• The “hot buttons” for the product.
• The strategic positioning - its fundamental identity, how it will be described, what it purports to be.  I.e., what are you selling ï a bank card, teller replacement, 24 hour service, fast money access, etc.
• What kinds of substantiation, information and other material will people need in order to try it and use it.
• Who will try it first, next, and so on.
• The type of trial.

I am sure you find new products and new product concepts to be extremely exciting, challenging and rewarding. The proper application of these principles are designed to assist you in translating these new product visions from dreams to reality.