The learning curve of qualitative studies

Matthew Ryan Nielson

The learning curve that governs qualitative marketing research is very important if you want to avoid ending up with mountains of data. What’s more, the data is soft, difficult to interpret, and likely to cause costs to soar. The pragmatic method based on the experience curve is a good starting point for making the most of these studies. What’s more, it’s an approach that promotes efficiency.

The learning curve of qualitative studies

After 10 to 12 interviews, the lessons – insights – are often repetitive, so the qualitative marketing research experience curve is reached

This method is simple. I owe it to my teacher, Pierre Louis Dubois, a reference in the field of marketing and research and the author of “Foundations of Marketing.” It’s a rule of “efficiency“*, not effectiveness.

Qualitative marketing research and the effectuation principle

In the case of qualitative marketing research, as in business in general, the principle of effectuation must reign, do just enough to get the right result, neither too much nor too little.

Restoring common sense in qualitative marketing research with the learning curve rule

First of all, it must be admitted that the literature in this field is fairly confusing. Some authors recommend sampling strategies very similar to quantitative sampling strategies (Frisch, 1999), others aren’t quite sure and will tell you “it depends” (Quinn Patton, 2002, Qualitative research evaluation & methods), and still others avoid answering the question altogether (Giannelloni and Vernette 2017 are otherwise very mathematical).

It would seem that there is a space here for some common-sense instructions for professionals wishing to do things right without giving in to any methodological madness.

Because of the learning curve, we always aim for 12 interviews (10 when that’s not possible), dividing up the audience for our qualitative research in order to maximize our results … without exhausting ourselves unnecessarily by interviewing hundreds of people, as some authors suggest

The first level in qualitative surveys

The first thing that strikes me about the work of students (but also of some professionals) is that most of the time, they start from a very blurred picture of their subject of study and they haven’t understood that there is a first-level of investigation that can be useful to them… interviewing experts (internal or external, or both).

In this case, there’s no need to go into sample details, the objective is completely different. You’re trying to untangle the situation, most of the time very unclear.

Qualitative marketing research: experience curve in Quali
A good starting point for qualitative research is to start with the experts, and wo better than those who have written books on the subject you’re studying, for example?

You’re not an expert in the field – since you’re studying it – and you need to understand at least a minimum of it in order to be able to recommend something sensible to your client, your associates, or yourself. Because of this, you should resort to interviewing a few experts. The more varied they are, the more important their insights will be to your research.

Where to look and not to look

They’ll tell you where to look and where not to look, and you can ask them about the hypotheses you’ve formulated for your investigation. Too many people bypass this phase, when in fact it’s one of the most rewarding. It saves a lot of time and gives you a better understanding of how your study will turn out and will help you formulate more solid hypotheses.

Finally, this phase will put you in touch with VIPs and specialists in various fields. You never know, these are people you could very well be happy to have in your contact database in the future.

The first approach to qualitative research is through a non-directive study. You’ll start with a few hypotheses and a list of questions that you’ll refine as you go along. This is not an in-depth questionnaire as such.

The in-depth qualitative survey

In fact, what I learned from my time in business school and during my initial training was I was able to test later in the field as far as qualitative research is concerned. Studies always follow more or less the same pattern. It’s not scientific analysis, but feedback based on experience and pragmatism.

When you’re interviewing people in depth, i.e. after the initial stage when you’ve interviewed a small number of experts, you can go out and visit larger populations to ask them questions. It’s not necessary to interview more than 10 to 12 people, as most of them will repeat themselves after a while. There’s no obligation to stick to this rule but be warned that going beyond this will take time and bring few insights.

The learning curve in qualitative research

This is known as the “learning curve”. I have to admit that I haven’t found any scientific evidence for it, apart from the lectures I received when I was at school, but I’ve been able to confirm these figures almost every time I’ve had to carry out an in-depth interview guide in the field, It always works. Whatever the subject. The learning curve always peaks after 10 interviews, sometimes 11, other times 12, and most of the time after the 12th interview. Beyond that, there’s no point in continuing.

This is provided that your interview guide is well constructed and that your active listening approach is good, of course. That is, you interview your participants with the same interview guide, there is no bias in this guide, you have filtered this guide during the first phase with the experts, which we described above, and your questions are consistent throughout your interviews.

At the end of this process, you should also note that you’ll need a deeper understanding of the subject through quantitative surveys in order to go further.

How to distribute interviewees within the experience curve in qualitative research

A good idea regarding sampling within the experience curve of these in-depth qualitative studies is to divide the number of people surveyed (12) into as many subsets as possible in order to draw conclusions for each small subset, and then quantify the subsets and conclusions using the quantitative survey. You don’t need to make your subsets representative. But you do need enough of them for homogeneous points of view to be expressed.

Once again, this is not scientific, but based on experience. If you interview the same people 12 times, it doesn’t make sense, and if your sample is unbalanced, so will the insights you draw from it.

Bring in as many diverse viewpoints as possible

What you want to do in the in-depth interview phase is to bring as many points of view as possible from the 12 you’ve collected, to bring different perspectives into your research: for example (in B2B) 3 employees for the internal view, 6 buyers from 2 different sub-groups for the external view and 3 resellers.

[note that the same applies to B2C, replace “buyers” with “consumers” and resellers with distributors, for example].

Enlarging the sample size for qualitative marketing research

If you need more subsets, this may be an opportunity to bring in others and expand your sample beyond the limit of 12. That is if you think you’ll have enough time to conduct these interviews, make the transcriptions, prepare the analysis, and of course, if there’s something in it for you. But in any case, don’t multiply the interviews if the insights are repetitive, it would be a waste of time and money.

Finally, remember that qualitative surveys only provide insights to be verified and quantified and by no means definitive certainties. In fact, they are often used to lay the foundations for your study hypotheses for the quantitative phase.

How can this method be adapted for customer interviews?

In the context of a customer interview, the question of whether the qualitative questionnaire will take place before or after the quantitative doesn’t arise. In this case, in fact, we’re essentially looking for very precise, in-depth customer insights on a subject that isn’t necessarily quantifiable.

Also, in complex sales, most of the time, the number of customers will be too small to be able to conduct a quantitative interview. In this case, the statistical view is of no interest. The qualitative mode of questioning is good and truly the right one for this exercise.

Qualitative marketing research: the Quali experience curve
Qualitative research can also be used to interview customers

For the rest, the methodology is pretty much the same. The content of the questionnaire, however, will be very different. It’s all about the relationship between the customer and the salesperson. Each case is unique, it’s not possible to generalize this type of questionnaire.

The purpose and use

First of all, you need to define the purpose and use of the questionnaire: will it be used to measure product quality or suitability, relationship quality, understanding of customer issues, or ability to anticipate changing market needs. Or any other specific point, such as the launch of a new offer, product, or service?

Knowing how to decode bias in a qualitative study

For the rest, as we’ve just seen, the method is the same, but you’ll need to know how to decode the biases, particularly in the case where the interviewer is the salesman facing his customer.

In this case, it is very likely that the customer will not reveal himself in the same way, because when faced with the salesperson he remains in a commercial relationship, and will therefore tend to use information (received or given) to negotiate with his customer.

Understanding fake beards

So you’ll also need to decode customers’ words to understand false and true arguments, which requires a certain tact (and the fact that the interviewer is well briefed before starting the investigation). Cross-checking information will be necessary.

In other words, it’s quite difficult to carry out this type of survey if you’re on the vendor’s side, to maintain enough neutrality and objectivity to hear the criticisms, which are often the richest elements to be gleaned from this type of survey.

Latest posts by Matthew Ryan Nielson (see all)