The Big Data Problem That Market Research Must Fix

Today’s businesses see market data as a commodity. Readily accessible information about consumer activity and preferences allows market researchers to develop large data sets to mine for consumer insights. And indeed, a look through recent market research industry publications shows that discussions in the field have been dominated by a focus on data analysis.

But more often than not, insight into what customers really care about is hampered by the quality of the data being collected. Some market researchers conflate the idea of data quality with sample size, with the belief that reliability, validity, and other characteristics of “good measurement” derive solely from the amount of data collected. This is certainly not the case.

A heavy emphasis on data collection and analysis is irrelevant if it omits the first and most important step of market research — the design of the metrics. In the psychometric tradition, survey development and the construction of specific survey questions has been emphasized as the most important step in the research process. Unfortunately, this step is getting short shrift by most market researchers today.

Failing to assess the measures that are the foundation of business decisions poses a colossal risk. Making data-driven decisions based on poor measures can be infinitely worse than making decisions without data at all.

To help organizations think more critically about the measures they use to collect information about consumers, we’ve outlined four common misconceptions held by many market researchers and provide suggestions for how to break away from these mistaken beliefs.

Belief in Measurement (Without Thinking About What We’re Measuring)

Historically, when market researchers wanted to measure a construct, such as how consumers feel about a particular brand (for example, “brand love”), they would ask respondents to rate questions that directly describe the construct, such as “How much do you love this brand?”

This kind of “measurement by describing” has its share of problems. For instance, many constructs are too abstract for regular consumers to report on in concrete terms. Think about how you’d reply if you were asked how much brand love you have for Tide laundry detergent. Most people couldn’t get more specific than reporting general approximations such as “a lot” or “a little.”

Researchers have begun to move toward methods that use self-reported data in better ways.