How to Get More Out of Your Focus Groups
Remove some of the pitfalls that are hurting your focus group research
Table of contents
No matter how good you think your focus groups are, you almost certainly can get much more useful information by avoiding some of the pitfalls and removing some of the constraints that reduce their effectiveness. You can also greatly improve what you are getting by holding moderators to a higher standard of professionalism. This article will show you how. Most people are satisfied with the focus groups they are now getting. As far as I can tell, the amount of qualitative research used by companies has been growing steadily. Whole industries, such as the pharmaceutical industry, have been doing dramatically more qualitative research. Companies that never “believed in qualitative” (as if it’s a religion), have now became true believers. Respondents who were never before researched are now being reached by telephone and online focus groups. The main cause of the incredible growth of qualitative research, put simply, is that it is usually better than quantitative research, even though most people believe the opposite. That debate is beyond the scope of this article and will be the subject of a [[separate article]]. The point I want to make here is that there is an increasing appreciation of what focus groups can contribute to marketing success. But as positive as people are to focus groups, many people have a vague feeling that they might be able to get even more from them. What they don’t know is that they would be ecstatic if they experienced what focus groups can really do.
The pitfalls clients fall into
I have identified several pitfalls that clients fall into that lessens the effectiveness of their qualitative research and keeps them from ever hearing truly great focus groups.
Pitfall #1: Running focus groups because you have no idea what else to do.
Now, there’s nothing wrong with this per se. Focus groups are the research to do when you don’t know what to do. But the pitfall is in coming at focus groups by default, as the absence of something. When you’re in this uncertain situation, recognize that you are in a perfectly natural and normal state. People who are always certain are rarely in doubt, but often in error. Turn the uncertainty into a positive. Make a list (right now) of ideas in the following form: If I knew —–, I could ——. For example, “If I knew physicians’ hidden fears, I could develop learning materials that would reassure them about using my product.” If you go to focus groups from strength rather than default, you will get more out of them because you will plan them differently. Instead of focusing on how vague and uncertain your objectives are, focus on all of the things you can get from focus groups. Many people allow themselves to be intimidated by objectives like the following:
- A general exploration to see if the concerns or practices of the people in the marketplace have changed.
- The creative stimulation of hearing real people struggle with a decision about your product.
- A better “feel” for the marketplace.
- A reality check on proposed new directions.
- Getting to know your customers and prospects more intimately, more deeply so that you can respond to them better.
The above are usually considered to be “vague objectives” and are often not approved by marketing vice presidents. I suspect that many product managers and marketing research people are uncomfortable initiating such requests because they think they will look foolish. But many great products and marketing campaigns have come out of just such groups. Procter & Gamble, I’m told, knows more about how people feel about soap, washing, and cleanliness than most pharmaceutical companies know about how physicians feel about medications, sampling, patients, and disease. So, don’t “settle” for focus groups. Raise your expectations, and everyone else’s (including your supplier’s), by stressing the positive values that you hope to get out of the group. Also, don’t refrain from doing groups just because your objectives are not narrowly focused.
Pitfall #2: Not being ambitious enough in your objectives.
Similar to doing focus groups out of default is not having ambitious enough objectives. You can get a lot more from focus groups than top-of-mind beliefs, knowledge, attitudes and practices of respondents. Focus groups are a laboratory in which you can get to much deeper feelings, implicit beliefs, hidden attitudes and secret practices. But more importantly, focus groups are a laboratory in which you can experiment with going beyond the present to what can be, beyond the is to the can and ought to be: You can discover how to change beliefs and behavior, how to persuade, how to teach, how to communicate. Focus groups are persuasion design laboratories in which you can develop and test new approaches.
Pitfall #3: Too few groups
Recently, another research company wanted me to find out the aspirations, hopes, wishes and fears of women professionals, such as lawyers, physicians, accountants, etc. They wanted to know their present life styles, how they spend their leisure time, where they will be shifting their priorities and what types of products and promotional themes would appeal to them in the coming years. Wow! A “Meaning of Life” study, as another qualitative researcher has called it. Just the kind of project that qualitative researchers dream about. “How many groups did the client have in mind?” I asked. They answered, “One group.” Needless to say, such an ambitious project could not be accomplished in a dozen sessions. [I ended up recommending a telephone focus group of experts in women’s issues, as a first step and general orientation.] Now, this is an admittedly egregious example of “one-groupitis.” I’m reminded of the old Texaco ad with Jack Benny finally becoming convinced to try their gasoline. He says, “OK, I’ll try a gallon.” But lesser examples are the norm rather than the exception. Clients continually want to do one group in a “cell,” that is, of a particular kind of participant, or mix groups of very different people. They don’t understand that the first group will usually go for breadth, and the next ones will zero in on what is found and get into increasing depth. Unfortunately, many clients have never heard depth in focus groups, so they don’t know what they are missing. In a pair of groups, usually 80% of the value is in the second group. In three identical groups, usually about 60% of the total value is in the third group.
Pitfall #4: Too much attempted in each group.
Similar to the previous pitfall. The largest number of questions I have ever had submitted to me to cover in a single group was 72. Other moderators tell me they have had more submitted. While this is extreme, I would say that about 90% of the time the quality of the information is hurt by trying to do too much in each group. For example, five minutes before the end of a particularly jam-packed-with-issues group of oncologists, a client sent me a note asking me to ask how they decided between anti-cancer agents in general. Anyone who knows the field knows that there are many different kinds and stages of cancers, that agents are not given alone, and that decisions are not made about single agents but about regimens and protocols, which are highly complex. To answer the question, an entire group would have to be devoted to it, and replicated several times. I was so flabbergasted that I actually asked the question and got rather coherent answers, the kind of answers that a television interviewer would have gotten. The client was satisfied, and thought that they got a useful answer. Because I have explored this question in depth in many other groups, I knew that the answers were superficial, misleading, and extremely incomplete; they would certainly not prove too useful. It was very difficult to convince the client that there were many more fundamental answers, and that the issue was far more complex than it seemed. The maximum number of issues that can be addressed in a group is one to three, with sub-issues under each. Typical guides contain 6-10. There must be a compromise in quality when you try for this much in a group. I realize that the one to three figure may be shocking to most clients, and that there are budget and time constraints, and I’m certainly willing to work out the trade-offs. But these decisions should be made consciously, not by trying to cram as many topics into each group as possible in a misguided attempt to increase the value of the sessions. Mies Van Der Rohe could have been talking about focus groups when he said, “Less is more.”
Pitfall #5: Groups too large.
The ideal group size for both face to face and telephone groups is about 7 or 8. Any more than that and people start relating to each other collectively instead of individually. People with divergent thoughts can hide. In a misguided attempt at getting a more representative sample, hearing from more people and just getting the numbers up, clients often want 10 or more in sessions. This causes higher interaction, especially for less experienced moderators. But the group tends to be more superficial, unless the moderator knows how to get into greater depth with the higher numbers.
Pitfall #6: Not enough flexibility, latitude allowed.
There is an implicit view among many clients that the moderator’s job is to walk participants through the guide sequentially, and as expeditiously as possible. There is not enough time left for the kind of serendipitous information that is so valuable in focus groups. Nor is there enough time for getting into more depth and detail, finding deeper motivations, verifying the information that you do get, etc. These things are the richness that focus groups offer, but there is seldom enough time. Again, most clients have never heard real depth, so they do not know what they are missing. I wish that I had the guts, and that I had a client who also had the guts, to let me go into some groups with only one question, without a guide, and let me see what I could get. I suspect that we would be amazed. I know that in the times when I do have some flexibility, the groups are much more productive, even though there is more pressure on me. It might interest you to know that Ted Koppel does not go onto Nightline with a preset guide, not even a first question. The main key to his skill is that he knows how to listen better than any other broadcast interviewer.
Pitfall #7: Failure to inform supplier of purpose of project.
This one, fortunately, is relatively rare. Some clients labor under the mistaken belief that the moderator will be more objective if he or she is not informed of the purpose of the sessions. I have one client who routinely does not even tell its own market research department what use it plans to make of the information. There are so many directions to go at every point in a focus group that I can’t imagine how this client expects to get the information it wants. They claim they do, and I find it hard to argue with a client that claims that its needs are being met, but I am sure that I could take the value of their research to another level if I only knew what I was doing for them and why I was doing it. Also, if you can imagine how difficult it is to conduct the groups, imagine how difficult it is to do the analysis and write the report! I don’t know if I am supposed to be uncovering hidden needs, assessing present practices, determining the level of knowledge of participants, generating new product ideas, developing promotional themes, or just giving them a feel for the terminology of the field. I’m sure it’s different in each case. This is another extreme example to make the following point clear: work with your researchers to give them as much information as possible. Yes, it will influence what is stressed in the sessions, but that is what you want. A good moderator will go into groups with hypotheses, which he will not allow to turn into biases. He will check out findings from many different directions, and has no vested interest in pushing any particular point of view.
Pitfall #8: Using amateur moderators
Focus group moderating looks easy and is highly lucrative. It is, therefore, a magnet for people who have attended many focus groups and who fancy themselves good with people. It’s a logical thing to move into when a research director or product manager gets fired, and some are even good at it. The truth is that it is pretty easy to run a group discussion: to get people started, to keep them on the subject, to keep the discussion moving, to bring out the people who are not participating, to inhibit the dominators, to bring people back to the subject when they stray, and to move them along to the next subject when they run out of steam. Although I have sometimes seen these things done pretty ineptly, they really are not very hard to do. But the mechanics of running a discussion is only a small part of what a qualitative researcher does. It is a necessary condition – background against which the real work gets done. I have prepared a detailed article outlining in depth what a good qualitative researcher does, so I won’t try to cover the same ground here. But I would like to emphasize that the qualitative researcher has to use the group dynamics to help people to get to deeper levels of meaning, to verify in different ways that he/she is getting the true story, must keep track of different motivations, and much much more.
The competencies of a focus group moderator are somewhat akin to that of a psychotherapist. I have been both, so I can speak from experience. Many people can function as pretty good amateur therapists: they can listen sympathetically, and get people to discuss their problems. But that does not mean that they will be able to get to the levels that will produce change, or that they will know what to do when they get there. There are a lot of ineffective therapists with fancy degrees. Patients can be taken in for a long time before they realize that they are feeling some temporary relief, but they are not getting better in any fundamental way.
Similarly, focus group moderators are paid handsomely to know how to sort out what is important, to understand implications, to decode symbolism, to unravel complex situations, to interpret ambiguous behavior, to develop strategies, to generate and develop new ideas, to design persuasion, to predict behavior. They not only have to be superb psychologists, sociologists, anthropologists (disciplines from which most of the best moderators come), but they must be superb marketing strategists. You have to go through a few hundred product launches and another few hundred new marketing campaigns to see what really moves people.
The fact is that for routine focus groups, any moderator will do. The problem is that there are virtually no routine focus groups.
Almost every group that seems routine turns out to have something in it that takes major skill to sort out. Groups typically come up with gold, sometimes buried in verbiage: advertising themes, hot buttons, new phrases, and new insights that tend to go right by most moderators and clients. Also, like a surgical operation, an airplane flight, a trial, or any other complex endeavor, things can get complicated, or go wrong. I’ve heard it said that an airline pilot’s flights are long stretches of boredom, punctuated by moments of stark terror. I have also heard that airline pilots earn their pay during about 17 critical seconds a year. But do you want to trust your life to a surgeon, pilot or lawyer who is less than the best, just in case? Admittedly, focus groups are not as dramatic as the above examples, and a focus group crash does not involve loss of life. But getting to people’s motivations, which is what most focus group work is all about, requires a great deal of skill, for which astute companies are willing to pay fees comparable to top surgeons because they know the value they get and how rare the skills are. An informal survey shows that top qualitative researchers charge only about 10-25% more than middle level moderators, who also charge about 10-25% more than novices. I think it’s a bargan, considering what’s at stake. Just a few words about companies doing their own focus groups with their own moderators. Aside from the obvious problems with objectivity and internal politics, it isn’t always a bad idea. (That’s not what you thought I was going to say, was it?) People on the inside have a feel for the product area, company strategy, and history of the product that outsiders can rarely equal. However, unless they have the requisite professional training and practice (not just a few courses) in one or more of the social sciences (psychology, sociology, and/or anthropology), and unless they have a lot of experience, and unless they do focus groups constantly and practice their skills, they are unlikely to attain the level of professional skill that is necessary to do a first rate job. It is unlikely that they will ever go though enough product launches, product positionings and re-positionings, concept tests, and other kinds of groups that will give them the necessary baseline of experience, comparative judgment and perspective to be able to interpret the data and make predictions. Those who do become first-rate focus group moderators tend to then leave and go to suppliers, where they can often make more money than the marketing VP of the company they left.
Here is a quick review, and some additional tips for improving the quality of your qualitative research.
Tip #1: Include your qualitative research consultant in the early planning stages of projects.
Tip #2: Call several researchers and see who has the best grasp of your problem.
Tip #3: Stay away from researchers who run formal groups.
Tip #4: Ask your researchers how they know that they are getting to the truth. Make sure that the groups get to the authentic and core levels, rather than the superficial, games and defensive layers.
Tip #5: Get product managers to do more general exploratory sessions, rather than only focusing on a specific, narrow problem.
Tip #6: Don’t treat your consultants like suppliers. Make them a member of the team and treat them as such. Often, they have more experience than the whole team put together. On the other hand, if your consultant acts like a supplier, always trying to please you instead of pushing back on your preconceptions and assumptions, get another consultant.
Tip #7: Use focus groups not only to explore, and to ask specific questions, but to develop new ways of approaching the marketplace. Be ambitious. Make the moderator work. Make them develop solutions to problems, rather than just uncovering problems.
Tip #8: Run enough groups.
Tip #9: Get the important things in depth, rather than trying to cover everything you always wanted to know about the subject.
Tip #10: Conduct true freewheeling discussions, rather than lock-step question and answer sessions. Tip #11: Don’t believe everything you hear. Check out everything, not just by replication. Use indirect approaches, such as projective techniques.
Tip #12: Hire professionals. Hold them to high standards. Listen to them.