Everything in Moderation
Readers of my series of marketing research articles in The Market Navigator (Update: No longer publishing. Using the blog instead.) have sent in more requests for me to write this article than any other. I have wanted to write it for many years.
But I have a problem: How do I write it without it seeming to be brazenly self serving? There is really no impartial and dispassionate way for me to write about what makes a first rate focus group moderator. I have to come from my own experience and some strong positions of advocacy.
First let me say that in the last few years, I’ve seen and listened to many face-to-face, online and telephone groups that other moderators have conducted, supplied by clients so that I could conduct a series of sessions in the next phase of the research. I think that the sample is somewhat biased, since the client was sometimes coming to me because they didn’t get what they wanted in the original series of sessions. I’m uncomfortable about saying the following, because it’s considered unprofessional to criticize colleagues. But I am so appalled by the level of professionalism that I see that I have to speak out.
I see too many amateurs. I often see people who lack credibility, who don’t establish rapport, who don’t establish an informal and candid atmosphere, who don’t manage the group dynamics. These moderators take verbalizations at face value, without checking, without probing deeply, without challenging, without having alternate methods to get past people’s defenses.
I have seen groups in which the participants attacked the concept and the moderator, where there was nothing wrong with the concept. The moderator caused the problems, and it was evident from the first minute of the moderator’s opening remarks that the moderator was poisoning the well.
I have seen situation after situation where the moderator shut down the interaction by a series of subtle behaviors which anyone trained in group dynamics and psychological interviewing would spot instantly.
Worse yet are the groups in which the moderator and the client get suckered because the moderator does not get into depth and does not verify that they are getting the truth instead of defenses (See the articles on Getting to the Right Psychological Level in Groups and Getting Beneath the Surface in Focus Groups).
I remember one series of groups in which a highly experienced, highly regarded moderator established excellent rapport, and sounded extremely competent. He dug out of a group of physicians that the “reason” that they do not prescribe a particular drug is because of a particular side effect. The moderator probed, and got detailed descriptions of the physician’s experience. I’m sure that if you had listened to the group, you would have believed them, as I initially did. But the moderator committed a cardinal sin. He believed what people told him without checking. The side effect was there all right. But it was an excuse, a smokescreen for other reasons for not prescribing the drug. When I removed the side effect, by getting groups to discuss what they thought of a hypothetical drug that was exactly the same, without the side effect, they said that they still would not use it. We eventually uncovered several professionally embarrassing issues that would have to be faced for some people to use the drug. The company was spending a lot of its time helping physicians control the side effect when the issue was actually irrelevant.
I happen to spend a lot of time with some friends who are fellow members of the QRCA (The Qualitative Research Consultants Association). We go out after meetings and bemoan the fact that many clients don’t appreciate, or sometimes even recognize, the difference between amateurs and professionals in qualitative research. Many clients send in people to do their focus groups, either from their own staff, or from the outside, who have little experience, or who have even decades of experience but are just not doing the job. They wouldn’t dream of taking a new employee and sending them in to do a conjoint analysis or other statistical technique. But anyone can get and keep a group discussion going, so they think that that person is able to conduct serious research.
I say, and many of my colleagues agree, that this situation is our own fault. As qualitative researchers, we have never laid out what excellent moderating is and what it takes. We hope that clients recognize excellence, and many of them do, but in the absence of standards it looks like magic, like the moderator’s personality, or good recruiting. As a magician, let me assure you that there is no magic. Arthur Clarke, the science fiction writer, once said that any technology, if sufficiently advanced, looks to the uninitiated like magic. Magic, let me assure you, is the performance of the seemingly impossible thorough hidden means. It’s about time we exposed some of the skills and methods to clients so that they can appreciate what we do, and so that everyone can be held to the highest possible professional standards.
I’d like to lay out some of the characteristics I have observed are most important in a moderator, tell you why I think they are important and identify where other moderators and I have gotten these skills. Then I’d like to present a checklist for rating moderators, both before and after you hire them.
What does a moderator do?
Now there’s a really dumb question. Or is it? Doesn’t a moderator just get people talking, ask probing questions and write up a summary of what people said?
Doesn’t a tennis player or golfer just hit the ball? Doesn’t a windsurfer just hang onto the sail? Try any of these sports, including moderating, and you will find that they’re not as easy as they look.
But here the analogy breaks down. With golf and tennis, you see where the ball is going, so there are objective measures of your success. Likewise, one false move and the windsurfer ends up in the drink. A focus group, on the other hand, may look wonderful, with everyone participating in a lively manner, interacting with each other, with the moderator sounding totally professional and probing just what you want (following every word of the guide), getting clean, clear-cut answers. But you may be witnessing a disaster in the making. You may be getting rationalizations and many other defensive communications.
In my article, “Getting beneath the Surface in Focus Groups,” I describe many ways to make sure that you are getting to the truth in focus groups and to confirm what you are hearing. It’s not appropriate to go into them here. But what is relevant is to make the points that: (1) there is a lot more to moderating than there appears to be, and (2) that it is not always easy to tell if a moderator is doing a good job just by how he/she sounds or how a group seems to be going.
Let’s do what is commonly called a task analysis and break down what a moderator does.
Already we get into trouble, because there are two kinds of people who moderate focus groups (leaving aside people who shouldn’t run groups!): (1) Qualitative Research Consultants and (2) Moderators. Now you may think that a qualitative research consultant is to a moderator what a sanitary engineer is to a garbageperson. But the term “qualitative research consultant” is no glorification and no euphemism.
Basically, a qualitative research consultant is someone who does the whole job, and a moderator is someone who moderates groups. I mean nothing negative by the latter term. Some people, particularly in advertising agencies, client companies and some of the larger research companies, are called in just to take the guide and run the groups. There is nothing inherently wrong with this, especially for projects which are relatively circumscribed, simple and straightforward.
Moderating is a subset of the “whole job,” but what is the “whole job”?
What does a qualitative research consultant do?
Brings years (often decades) of experience to the project
The experience gained from conducting 100-200 groups a year, writing 50+ reports a year, and conducting 50-100 formal and informal presentations a year for 20-30 years.
Hundreds to thousands of:
- new product/service ideas
- new product introductions
- different kinds of respondents
- different research problems
- subtle differences in participant responses
- different client objectives
- different marketing strategies
- different advertising, promotional, and sales executions
- analyses, reports and presentations
- predictions of what will work and not work
Based on this experience,
- the ability to tell you how groups compare with a vast experience base of similar situations
- the expertise to interpret what has not happened
- what should have happened but didn’t
- what did happen but is unusual
- or what might happen that can be planned for.
Clarifies the research problem
- Understands the background leading up to the research
- the presenting problem
- the real problem
- technical market and product details
- strategic marketing issues
- internal client political issues
- understands what product characteristics are technically feasible.
Lays out the research objectives in precise and operational terms.
- Helps the client prioritize the objectives.
Designs the research
Advises the client about methodology:
- broad methods
- specific details.
- Actively and persuasively discourages the client from doing what is not in the client’s best interest.
Writes and supervises:
- Inviting script
- Confirming letter
- Materials sent before the session, if any.
- Script for confirming phone call
- Moderator guide
Coordinates the project
- Helps the client understand and anticipate difficulties in executing the research.
- Manages all resources in the project, including the clients’ role.
- Makes life easier for the client, by telling him/her exactly what needs to be provided, by when.
- Manages the recruiting, facilities, etc.
- Makes all things happen that need to happen.
Conducts the research
- Makes sure all client people are briefed.
- What has transpired previously
- What to expect
- How they can help the research
- What to watch for
- Determines how people will be greeted, and all aspects of how they are “handled” before the group.
- Moderates the sessions.
- Creates atmosphere of psychological safety
- the purpose of the group
- the ground rules and how to participate
- Sets atmosphere of informality, openness, interaction.
- Conducts a discussion, not just a serial interview.
- Handles all of the different types of people, and different behavior in the group in such a way as to make sure that the group is productive.
- Performs verification and confirmation procedures to make sure he/she is getting the truth.
- Modifies the guide, even in the middle of groups, to pursue new ideas, follow hunches, etc.
- When possible and appropriate, takes the research beyond the original objectives.
- Creatively develops solutions to problems, helps the group generate new ideas, actively generates hypotheses, pursues hunches and uncovers hidden qualms.
Conducts post-session debriefing
- Identifies what is already established
- Interprets what has been heard
- Makes hypotheses and hunches clear
- Identifies patterns within and between groups
- Cautions about what conclusions should not yet be drawn
- Helps client identify clearly how the guide should be changed, or what should be emphasized in the next groups.
Analyzes and interprets the results
- Helps the client draw the appropriate interpretations and conclusions.
- Makes strategic and tactical recommendations, based on the research conclusions and on how they fit into years of similar experience.
- Supports the client in presenting and explaining the results to higher level management.
What are the skills necessary to perform these tasks?
If you study the above task analysis carefully, you’ll realize why there are so few excellent moderators and why the people who are the best are so highly prized and highly paid. What kind of a person, with what kind of training, can perform such diverse tasks and still work on several projects at a time?
A few general observations, before we get into specifics:
Analytical vs. creative
As you study the above list, notice that it requires an extremely rare blend of analytical and creative skills. People who are analytical are rarely creative enough to make the imaginative leaps necessary to think strategically. Those who are extremely creative are rarely analytical and cerebral enough to look at something objectively without jumping to the conclusions their creativity is generating.
People vs. things orientation
It also requires a rare blend of people skills and interest in “things.”
But many people who are interested in people are right-brained and shun technology. They are “technologically impaired.” They suffer from a gadget deficiency. They don’t like The Sharper Image catalogues and stores. They don’t even have a cherry pitter. Their videotape recorders are blinking 12:00 AND THEY DON’T CARE!
Most technologically adept people are left-brained and awkward with people. They wear different color socks (as comedian Steve Wright says, he doesn’t care, he goes by thickness). They are frustrated by and impatient with people: with their subtleties, ambiguities, contradictions, inconsistencies, feelings and dreams.
Unfortunately, in most people who are equally right and left brained the two sides cancel each other out, producing a colorless dullness.
In order to be a qualitative research consultant, a person must be able to constantly switch between the analytical and creative modes and be superbly adept at both.
Also, the person must delight in figuring out what people like about pizza, or why they choose word processors or like being ophthalmologists.
The simplifier vs. the complexifier
It also takes a rare blend of the simplifier and the “complexifier.” Simplifiers take complicated situations and cut to their essence [like that]. Complexifiers (to coin a phrase) look for all the multifaceted aspects of seemingly unambiguous circumstances in an effort to translate the deceptively self-evident into the incisively penetrating and richly embellished [like that]. Both an ability to cut to the essence and an ability to appreciate the subtleties are needed (even though verbal complexity is not). Again, they are rare in the same individual.
A person who loves business
It takes a person who loves business in general and marketing in particular. Too many psychologists and sociologists, particularly those with an academic background, think there is something dirty about business. They cynically view marketers as exploiters and con men.
The effective researcher must be an entrepreneur at heart. He or she must appreciate that good business is a win-win situation between customer and company. That business is basically trade, where each person exchanges what he has for what he wants.
How to pick a moderator or consultant
How do you find and select someone who can do all of the things described above? As the old joke goes, “Very carefully.” However, it is no joke that the job is all too often left to beginning market research analysts. While I have a great deal of sympathy for new analysts, and enjoy teaching them about qualitative research, and am delighted to be picked by them to conduct studies, I always find it strange when the qualitative research consultant selection decision is left up to someone who has never heard a focus group, or someone who has only heard a few.
Some practical tips for choosing a moderator or QRC (Qualitative Research Consultant)
Check with other people in your company, sister companies, or other people in your industry.
Ask those people specifically why they think the person they are recommending is a good moderator. Some moderators are excellent for same types of focus groups and not for others. For instance, an extremely natural and informal manner is needed for salespeople and for experts, for opposite reasons: an imposing manner will make salespeople think of the moderator as a representative of the home office, and will make experts fall into their formal, teaching mode, cutting off their more creative, speculative side. On the other hand, top corporate presidents tend to feel more comfortable starting out more formally, and gradually loosen up. Creative idea generation groups require a different style than probing deep attitudes about sensitive personal subjects.
The point here is that you should be looking for a moderator who has demonstrated competence with the type of participants and the type of objectives that you are pursuing.
Next, call the moderators who are likely candidates.
Since the respondents’ first impression of a moderator has a major effect on the outcome of the group, tune into your first impressions. This is the only time you will have a first impression of that moderator.
Is he/she easily approachable, easy to talk to, open, casual, purposeful, articulate and direct? Are you asked penetrating questions? Do his/her questions put you on the spot and make you uncomfortable, or challenge you in a pleasant way to explore new areas and subtleties? Do you find yourself wanting to open up or get off the phone? Does the person sound like a “regular guy/gal” with a keen intelligence, or a refugee from a succession of jobs that necessitated going into business for himself or herself? Is there a “weirdness factor” If you’re not sure, don’t give anyone the benefit of your own self doubt. Think of how the participants will feel.
Trust your instincts here. Does the person sound like the sort of person you would like running a group you were in?
While it is not appropriate to judge all business conversations by how pleasant they are, your conversations, particularly your first, with a prospective moderator should be energizing, provocative and stimulating.
Assess their approach to your marketing problem, grasp of marketing
Aside from being approachable and stimulating, a prospective moderator should be able to listen and to hear. Does the person “get it” or need unnecessary explanations? On the other hand, does the person get it too fast: making assumptions, or jumping to conclusions?
Does the person have the ability to grasp the situation, the marketing problem and the research objectives? How the moderator interviews you about your marketing problem is probably the single most important thing you can look at in picking a qualitative research consultant.
Ability to advise you on methodology
Can the person make you aware of options for designing the project and quickly help you sort them out?
Or is the person pushing a particular methodology without being able to justify it, or trying to expand the scope of the research (size of sample, number of groups, etc.) in order to make a bigger sale?
How does the proposal or presentation rank in terms of:
- Grasp of problem
- Clarity of Objectives
- Did you deal with the Moderator or a salesperson?
Background and Training:
Ask for a “Statement of Qualifications,” or similar document. Here is what to look for.
|Psychology||Thorough training. Not only formal academic degrees, but experience.||First and foremost, you are trying to get to people’s motivations and change their minds. Psychology is the most relevant discipline to this process.|
|Personality theory||Motivation, personality organization, attitudes, etc.||Here is where you get an appreciation for what really motivates people, the pervasive defenses they use to distort the truth, and the superficial the games they play. Also, a working knowledge different personality types and how to interpret their answers.|
|Treating patients||This is where you practice working with people, establishing rapport, drawing people out, getting around defenses. Here, you get a first-hand feel for what it takes for people to change even the simplest behaviors.|
|Intelligence, ability, achievement and aptitude testing||This is where you learn how people process information and skills and learn to appreciate different cognitive and emotional styles.|
|The use of ambiguous stimuli to elicit deeper thoughts, feelings, attitudes, values and styles.||Where the consultant is trained in how to probe for deeper material than can be gotten with usual questioning techniques. How to probe repeatedly without introducing bias. Here is also where you learn how to interpret ambiguous material, verify hypotheses, and communicate clearly.|
|Psychopathology||Knowledge and appreciation of neurotic and psychotic styles and character disorders.||Crucial to interpreting what people tell you in groups.|
|Educational psychology (or other communications training)||How to communicate new information in a way that sticks.||An appreciation for how to communicate with different types of people in a variety of circumstances in a ways that really get through. Methods for changing behavior.|
|Formal psychological research||Research design and analysis. A healthy distrust of statistics. An appreciation for the many ways in which bias can creep into research.|
|Group Dynamics, organizational development courses, seminars, practica and workshops.||The management of groups and large organizations.I have spent years, including a one-year psychological internship, in intensive group dynamics training, both as a participant and a leader. Have led many encounter groups, T-groups, Sensitivity training groups, therapy groups, conflict resolution groups, etc. before running my first focus group in 1969.||How people act in groups, and how to facilitate open and productive groups. Feedback on how you are perceived as a leader and the effects you have on groups. Practice in handling difficult situations in groups. Group facilitation methods, exercises, formats and styles.|
Problem Solving Training
|Formal programs in creative and analytical problem solving.||How people solve problems and how to facilitate problem solving. How to draw out new ideas and encourage people to improve on them without rejecting them. Important for all groups, but particularly important for idea generation, concept testing groups and salesperson sessions.|
|First hand experience developing and selling products. E.g., I worked in my father’s drug stores since the age of 9. He was highly promotion oriented, so I got to see what brings people “into the store.” I was a professional photographer through college and graduate school. I started three successful businesses. I have sold everything from low ticket items to 6 and 7 figure programs. The largest single sale I have had was $750,000. The largest program I have sold (multiple successive sales) was $5,000,000.||A qualitative consultant must have an entrepreneurial attitude: able to see opportunities, creative in devising ways to accomplish objectives. A fundamental understanding of business is necessary to make practical recommendations. Sales experience is necessary because the central job of marketing research is about finding ways to persuade people. You have to have faced people in many selling situations in order to understand the dynamics of persuasion. You have to understand how your marketing recommendations fit into a marketing plan, so you must have experience in strategic planning, advertising, sales promotion, sales training, PR, trade shows, direct mail, distribution, and all other aspects of marketing. If you’re going to advise people on how to launch products, it helps to have launched a few yourself.|
|Presentation training||How to give presentations.||So much about qualitative research consulting is about communicating effectively in groups (client meetings, focus groups and presentation of findings), that it is worthwhile getting training in how to make presentations.|
|Specific training in conducting Focus Groups||Hands on experience in conducting groups under the guidance of an experienced mentor.||Feedback about specific skills in running groups.|
|Independent experience in conducting qualitative research||This is the “school of hard knocks” where experience is the best teacher.||There is nothing like a few thousand groups, over a few decades, on a wide variety of subjects, with enough time to follow what happened in the marketplace to allow someone to become fully professional.|
Also look at:
They should have past business and personal experience that lets them be a well rounded individual.
Kinds of products
It is not important that a consultant worked with your product in the past. Sometimes it’s an advantage, sometimes a disadvantage. It is important that they have worked with similar products and that they be a marketing expert.
How long moderating
It takes a decade or two to become an accomplished qualitative consultant. Some moderators have 20 years’ experience, others have one year’s experience 20 times.
Ask around. Experienced people know who the best moderators are.
Can be important for a less established consultant, less important for a more established one. Ask the references specific questions about the strengths and weaknesses of the consultant.
Professional attainments, contribution to field
Articles, workshops, courses, books, speaking engagements, etc.
Membership in professional associations
While membership in the QRCA (Qualitative Research Consultants Association) does not guarantee competence, I must say that I find it hard to understand why some qualitative consultants do not have the time or inclination to participate in improving their skills and the professionalism of their field, when other qualitative consultants are willing to travel across the country to participate in workshops, seminars, conferences and meetings sponsored by this organization.
What to watch for during the sessions
Mistakes in moderating sessions
Failing to establish rapport
The excellent moderator establishes a working, candid relationship with the participants. They want to give him/her information and express their feelings.
- Being too formal, unnatural
Addressing physicians as Dr. Jones instead of Jim, failing to set up an informal, relaxed atmosphere.
- Setting up too much structure
- Some moderators actually call on people rather than encouraging informal discussion
- Setting up the purpose of the groups too abstractly or vaguely or not at all
- Setting up as a serial interview
- Letting each participant respond only to the moderator, instead of encouraging a discussion among the participants.
- Asking compound Q’s
- Asking several questions at one time.
- NOT LISTENING
You would be surprised at how often moderators are thinking of their next question, instead of really listening. Do not confuse attentiveness with listening.
Use of jargon Asking people to explain their behavior
Asking people why they do things, instead of asking what they do, how do they feel about it, how they would feel if they couldn’t do it, and the myriad other probes that get to their deeper motivations.
- Asking inappropriately vague questions, or improperly precise questions that lead the respondents.
- Arguing with the participants, or otherwise losing objectivity, rather than playing Devil’s Advocate
- Making participants wrong
- Praising specific comments of participants
Telling people “that’s good,” or “that’s interesting,” etc. Shuts down conversation because it is subtly evaluative and tells people that their comments that did not receive praise were not good. Also biases people toward making comments congruent with the praised comments. There are other ways to get people to expand on a point without revealing your interests, expectations and desires, or without giving gratuitous praise.
Ignoring silent participants
They are either very smart or very dull. Find out.
Giving information which biases the group
Obvious, but you’d be surprised at how much it happens.
Not making it interesting
Keeping the interest level high among participants keeps up their motivation.
Letting someone speak for the whole group
Letting the group take it away
Polling can be appropriate to get people to take a stand, or express things more clearly. But they are too often used inappropriately to get a projectable vote.
Jumping to conclusions, making unwarranted assumptions
Sometimes moderators tend to jump way beyond what people are actually saying, or they assume that what is said is what is meant. Everything has to be checked out and verified.
Refusing to generate hypotheses
On the other hand, most moderators are afraid to make legitimate inferences about what they are hearing, and then they don’t get a chance to check it out in the group.
What to watch for in the analysis
- Were the objectives accomplished?
- Is the report short and to the point, or is it padded with verbatims in order to impress?
- Were the findings stated clearly?
- Is it clear which are findings and which are conclusions or implications?
- Are the implications clear?
- Does the report contain practical recommendations or hedged, ambiguous, or “safe” nonsense?
- Is it well organized, so that you can see the relationships of the various parts?
- Is the report clear enough, short enough and interesting enough to motivate the decision makers to read it?
- Is it credible to the decision makers?
A few further thoughts
By picking an average moderator, you will avoid rocking the boat. You will get some new information and safe findings. By picking an excellent qualitative research consultant, you will get new information, fresh insights, and, often, breakthroughs. These often rock the boat. So, in any given case, you have to decide whether to play it safe or take a little more risk that some people’s pet beliefs are challenged. But keep in mind that one marketing insight, one actionable recommendation can often make the difference between success or failure of a product, or can make a two or threefold difference in sales. We are talking about tens or hundreds of millions of dollars per product. You are playing for big stakes, both in your career advancement and in the success of your products. Isn’t it worth getting the best qualitative consultant available, even if it costs a thousand dollars more per session?
The better qualitative consultants bring enormous competence to bear on marketing questions. I’m amazed at the versatility, attainments, and sheer competence they display. They truly are professionals in every sense of the word. When that professionalism is not appreciated, when clients hire people who are less expensive, who are not skilled and then like the job they did, it is a source of pain to each of us, but as I mentioned before, it is our own fault for not educating people in what we do. After all, when they hire the better consultants, all that they see is an effortless grace, a natural flow that looks as if the recruiters got a great bunch of people together, but certainly does not look like the moderator is working very hard.
Qualitative Research is frequently criticized because it is dependent on the particular individual doing the research. True. I believe that this fact is both its strength and its weakness. That’s why it’s “everything in moderation.”