Social Listening Is Revolutionizing New Product Development

Every day, billions of people talk on social media about where they’ve been, what they’ve bought, and their feelings and opinions about products and services. This information is a gold mine for consumer-facing industries, including retail, consumer goods, retail banking, insurance, and health care. But few companies have acted on this valuable data.

Companies have traditionally relied on surveys, focus groups, and research reports to assess what consumers think of their products or services, but these traditional approaches have several shortcomings. Sample sizes are limited and subject to bias. Studies take time to organize, and results quickly become dated. Moreover, what people say often differs from what they do, like complaining about discount airlines but using them all the same.

Leaders who employ social listening — analyzing what consumers say on social media — can gain a competitive advantage by getting better insights that they can act on quickly, without incurring the higher cost of traditional approaches. This new approach, fueled by social platforms, increasingly informs the new product development, marketing, operations, and international expansion of consumer product companies.

Put another way, social listening can serve as the map for a treasure hunt. It is already overturning how consumer product companies develop, market, and package their products — and we are only beginning to discover the scope of possibilities that machine learning advances will catalyze.

How Machine Learning Is Refining Social Listening

Until now, social listening has mostly been limited to public opinion monitoring, such as counting the number of times a brand is mentioned (buzz) and whether the content is positive or negative (sentiment). But this data tends to feed into standardized or aggregated metrics, which are too generalized to use for making business decisions.

Recent advances in machine learning have enabled smart analysis of natural language content, as well as the monitoring of pictures and videos. This progress allows companies to investigate a wider range of customers’ feelings and opinions and to identify specific trigger points — such as the color of a product or location of a service — for those feelings. By analyzing customer data on social platforms, companies can now map and update customer preferences and monitor how they connect with and influence one another.

Machine learning can also help with the problem of unclean data. Companies can buy social media data from vendors, but few of the source posts come from consumers; instead, many have been written by the manufacturer’s marketing agency, e-commerce sellers, or robots. Through research for clients across many product categories, we discovered that just 10% of posts provided by a data vendor were written by an actual consumer. By filtering out this fake content, machine learning can help companies avoid gaining a distorted or false impression of their brand health.

Deeper Consumer Insights for Product Development

New-product development cycles last for months, if not years. A product concept that is right for the market now might be outdated by the time it has been developed and approved. Manufacturers need to be able to spot early signals and trends to create products that will fit the market by their future launch dates.

Social media offers many clues about the changing preferences that can pinpoint consumer trends. By understanding a trend’s broader context, companies can better assess whether it is likely to sustain and support their product goals.

Analyzing social media data requires a combination of natural language processing and consumer insights from other sources. These techniques are already paying off in the beauty and apparel sectors. One skin care product manufacturer we worked with doubled its product success rate thanks to better consumer information, halving its costs by developing fewer product flops.

By bringing manufacturers closer to consumers, social listening can facilitate the development of customized products for specific groups, such as allergen-free formula for babies. Social listening can also help determine how trends in one market influence those in another — and if a product might be successfully exported internationally.

More Effective Tribal Marketing

Analysis of social media interactions also makes tribal marketing easier and more precise by locating groups with tastes and needs in common on, say, Twitter or Weibo, its Chinese equivalent. Once tribes with a strong affinity for a product category have been identified, a brand can build connections with leaders — for example, by inviting them to events or sending them exclusive product previews and samples. Influential leaders’ posts about the latest products or services may generate interest in the brand among community members. Such marketing techniques complement mass advertising campaigns with paid influencers like celebrities and key opinion leaders. In the apparel industry, we have seen tribal marketing combined with social listening generate a return on marketing spend significantly higher — by 20% to 50% — than traditional advertising.

By tracking and measuring the connections among a group of new mothers, for example, marketers can map their tribe and identify the leaders who are best connected with the other mothers. By engaging with group leaders, a manufacturer can better understand their lifestyles and tastes, perhaps sparking their interest by sharing other products and services, such as gym classes for new mothers.

Improving Operations

Many consumers post on social media about their interactions and experiences with brands, products, and services — how bad the packaging is or how poor the delivery was. It can be tricky, however, for a manufacturer to act on such comments. Simple aggregations of keywords, such as packaging, are not useful, since they don’t indicate a specific complaint or establish whether an issue is recurring or confined to a particular batch. Without such details, it is impossible to act on improving operations.

But techniques such as natural language processing, machine learning, and image recognition are now helping product makers deduce more precisely what consumers are saying, leading them to effective decisions in response. For example, social media posts complaining that a bottle is hard to carry because it is large and made of glass could prompt a manufacturer to consider smaller containers made of different materials. If other posts praise foundation makeup for its long-lasting wear and effective pore coverage, these qualities might be explored as themes for future advertisements.

The Future of Social Listening

In practice, social listening use cases can be combined. A retail bank launching a new and independent digital unit, for instance, can use social listening to detect and quantify customer pain points, segment potential customers, and inform product design. After launch, marketers can target influencers and track brand perception to optimize products and services continuously.

Very few companies are currently making the most of social listening techniques. That’s not because of a lack of material to work with: Social media data — all posted with explicit user consent — is publicly available from social media platforms or via third-party data vendors. Instead, the main barrier to social listening has been poor-quality data that is too aggregated or includes too many fake posts to provide significant consumer insights.

Obtaining clean data that can inform business decisions is not easy. Data gathering can require natural language processing in different languages and advanced techniques to analyze social media accounts and content automatically and effectively. Soon, these techniques will be common in consumer-facing industries — and companies that neglect social listening will find it hard to catch up. In contrast, product manufacturers that learn how to understand consumers in actionable ways will have a considerable advantage.