| |
And there is nothing more disheartening for a marketing Consultant
than these so-called "Strategic" decisions based on hearsay, whereas
the very profile of existing customers is not even known. One
of my recent engagements has put me in the presence of managers
in charge of a 14 million customer-base, losing 1 million customers
a year, who could not describe the latter even with some vague
precision. Moreover, each manager in the organisation had a different
view of those customers? Some of them described them as the average
gutter-press reader, whereas others would describe them as representative
of the UK population, without being able to back these assumptions
by even the slightest evidence. Needless to say that any money
spent on "Strategy" in those conditions is not very a very good
investment.
The "vision" that can be carried across the organisation is also,
in this case, reduced to a minimum. Managers, therefore rely on
assumptions which are mostly inaccurate, and their future strategic
choices will be impaired by them. Let us take a closer look
at the Banking Sector in Europe. In an environment where the increase
in competition (BANKING BANANA SKINS - CSFI, Curzon Street London,
1997) is perceived by Banks themselves as a critical factor, most
financial institutions give the impression of drifting along,
by lack of clear strategies, seemingly unable to voice clearly
their objectives.
However, in an industry where product differentiation is non existent,
customer perception and customer satisfaction should indeed be
placed at the centre. On the contrary, we are running the risk
of seeing more and more institutions drift before they are subject
to mergers and other hostile take-overs.
As a consequence, two main differentiation factors emerge as
crucial from those observations: Customer Surveying and Customer
Database Management. Yet it does not suffice to mention
the significance of these two factors. Here, more than anywhere
else, rigour of approach and methodology are key to the success
of field Marketing. Unfortunately, just as what happened in the
area of Direct Marketing, the absence of methodology and understanding
of the complex area of Market surveys, has led to severe abuses,
and sometimes, to the rejection of this powerful tool.
Building a Questionnaire requires that one follow the rules,
in order to avoid the usual biases arising from a bad methodology.
I want to provide my reader with a simple check-list, which is
a useful guideline for the conduct of Market surveys. Most of
the time, these simple, down-to-earth principles are not followed.
Indeed, questionnaires also require strict and precise wording,
and general/personal qualities that cannot fit onto a check-list.
For each of these rules, there is a number of biases that one
has to avoid (See our method for Building
Questionnaires in 12 Steps). But it is somewhat difficult
to discuss quality control within the Market survey process, without
looking at some real-life examples. Last but not least I
think it is necessary to admit the inherent imperfection of Market
surveys and take the necessary precautions against bad analysis
and conclusions, rather than believe that scientifically produced
and administered Questionnaires may even exist.
However, if Market surveys have become "evil" in the eyes of
many, it is mostly because these steps that we have just described
have not been followed, therefore generating generalised suspicion
towards this tool. It is certain, that given the uncertain
economic environment that is prevailing, market survey methodology
should be updated; mainly, questionnaires should be made smaller
and more frequent, but there is no evidence of that on the field.
On the contrary, we have observed monster surveys on the field
such as even the most courageous of interviewees could not face
without a yawn. However, the main principles that lead to
the conduct of market surveys are still valid. Once gain, this
is a case of avoiding the confusion between reactive, flexible
marketing and suicidal non/mis-management (Badot/Cova '92).
The
Dark Side of IT
There is another pitfall that many Managers have fallen (and
continue to fall) into, I mean the dark side of IT. Failing
to grasp the importance of the human factor within IT has
led to market research being perceived as immensely powerful (and
even potentially dangerous). Reality has often been less
promising; but most of the time, it has nothing to do with IT
itself. The belief that IT has enabled managers to "massage" huge
volumes of data has led to the mirage of all-singing, all-dancing
customer databases which were meant to increase revenue or competitive
advantages as if by miracle. However, a customer database -however
vital- is of no use if it is not cleaned up and maintained regularly.
It does not serve any purpose if there is no strategy behind it.
It cannot replace strategy. At best, it can be a powerful tool
for feeding your strategy with meaningful numbers; that is to
say when it is used with vision. Otherwise, it is just another
way of producing data rather than information more quickly and
in bigger volumes (SAMLI '95).
As usual with IT (and with marketing alike), there is this well-spread
tendency to go from one excess to the other, i.e. either believe
that technology can replace human tasks satisfactorily (and it
fortunately can't) or otherwise that it is useless and disappointing.
These two points of view are excessive. Our ambition is
to help Managers become more reasonable in their expectations,
so that they can withdraw more both from Marketing and from its
new servant, Information Technology.
The
Design of Questionnaires in 12 Steps
| Step
1 - Objective : |
-
Describe the Objective of your survey in a few bullet
points. Avoid accumulating objectives. 2 or
3 main points are a limit to a good survey. Although
it may sound restrictive, it is very unlikely that your
interviewees will be able to stomach more than this (with
the exception of highly specialized business to business
surveys focussed on professionals)
-
Failing to describe your objective will make you lose
sight of the overall purpose of the survey and will have
serious consequences on the length, the accuracy and the
sequencing of your questions.
|
| Step 2 - Population : |
-
Define the total population concerned with your survey
-
Match this definition with the objective stated in Step
1, i.e. neither too wide or too narrow
|
| Step 3 - Pre-study : |
-
A Pre-study (caution, a pre-study is NOT a pre-test) should
never be overlooked
-
Carry out the study on a few individuals concerned with
your study
-
Avoid prejudices (yours or more likely, your customer's
!) which could make you jump to conclusions even before
the beginning of the survey
|
| Step 4 - Assumptions : |
- State
clearly the points (in accordance with Step 1), that you
wish to clarify or verify, or even contradict
-
Do not filter these assumptions according to your own
prejudices, i.e. avoid attitudes such as "I don't need
to check that point because I feel that ... " unless
you already have evidence of the phenomenon in the first
place.
|
| Step 5 - Writing
the Questions : |
-
Check the wording of your questions so that they are ...
-
Understandable
-
Unbiased
-
Non suggestive
-
Not built in a way that interviewees react defensively
-
Not long-winded and repetitive so as to avoid weariness
factor
|
| Step 6 - Contact
Methods : |
-
Pre test your questionnaire on a representative sample
of the given population
-
If cards have to be shown to the interviewees, then add
to pre test
-
Choose Contact mode
-
Mail Questionnaire
-
Personal interviewing (arranged or "intercept")
-
Phone interviewing
(New
methods may include fax, Email, or even Html forms, but
one may wish to stress the difficulty in using these new
media. In a sense, they bear a strong resemblance
with the phone in so far as the interviewee is invisible
and difficult to identify. These media can only be
used for short questions aimed at checking precise assumptions.
As for the population involved, it goes without saying that
Web usage is not pervasive enough - even in the USA - in
order to use this medium to survey wide populations of interviewees.
At the moment, the web is best suited for Internet survey,
but things might change in the near future)
-
Modify questions in accordance with Pre test results
-
Modify sequencing of questions in accordance with Pre
test results
-
Suppress redundant questions (this is often a sore point)
-
Insert missing questions
-
If cards have to be shown to the interviewees, ensure
that contents match questionnaire.
|
| Step 7 - Sampling : |
-
Define the representative sample
-
Give a sufficient size to the sample (methods of appraisal
of error margins can be used)
|
| Step 8 - Administration : |
-
Administer Questionnaire in a neutral fashion (i.e. preventing
interviewers from introducing personal biases while asking
the questions. This may imply that these interviewers
must be either trained or skilled professionals, and that
a clear, thorough and comprehensive debriefing session
takes place).
-
If this is a Mail Questionnaire, reply envelope (free
post or s.a.e. if small organisation involved) must be
included
|
| Step 9 - Non
responses : |
-
Response rate is an important factor. Do not just overlook
them. They are a good indicator the quality of your
questionnaire (amongst other things, namely if a gift
is sent to all respondents)
|
| Step 10 - Interpretation : |
-
Interpretation should be distinct from the opinion of
the collector of the data
|
| Step 11 - Analysis : |
-
Analyse ALL the responses (avoid partial analysis)
-
Do not extend results that are valid for the given sample
to the entire population without taking the necessary
precautions
Note: A famous example of a bad survey analysis that is
given by the focus groups that led Coca Cola to change their
flagship product at the end of the 1980's. The analysts
of the focus groups were adamant that traditional Coke had
to be changed, but the assumption was wrong, the methodology
too, and eventually, the decision that arose from these
groups almost led the Atlanta giant to a disaster.
|
| Step 12 - Report : |
-
Write a report that will describe the results comprehensively
-
Avoid biased conclusions
-
Minimise prejudices
-
Avoid "politically correct" conclusions
-
Mention numbers and percentages (both are necessary)
|
|
|