From Niches to Riches: Anatomy of the Long Tail

Eric Clemons, a professor at the Wharton School, is an aficionado of Dogfish Head World Wide Imperial Stout beer at $160 per case. How did Dogfish Head find a customer like him? They didn’t. He found them. Clemons was not always a connoisseur of rare beers, but after trying Victory Hop Devil beer, the top-ranked India pale ale at the time, he learned of Dogfish Head, as well as Victory’s Storm King Imperial Stout and other niche beers, through the Internet. Clemons notes that he “would never have bought the Dogfish Head without the reviews on ratebeer.com and without the chain of experiences with ever-more interesting beers along the way.”

For most of the past century, companies of all types strove to introduce products and services that were blockbuster hits and could capture the mass market. Bigger was better. But now dozens of markets, from beer to books, music to movies, and software to services of all types are in the early stages of a revolution as the Internet and related technologies vastly expand the variety of products that can be produced, promoted and purchased. Though based on a simple set of economic and technological drivers, the implications of this are far-reaching for managers, consumers and the economy as a whole.

Early discussions of Internet markets focused on how “frictionless commerce” would lead to fierce price competition online. However, while consumers certainly do benefit from lower prices online, our research indicates that they derive far more value from another important characteristic of Internet markets: the ability of online merchants to help consumers locate, evaluate and purchase a far wider variety of products than they can via traditional brick-and-mortar channels.1 (See “About the Research.”)

Consider the market for books. Amazon.com and other Internet retailers sell nearly all of the more than 3 million books in print. A typical brick-and-mortar store, however, stocks only between 40,000 and 100,000 unique titles. One might argue that consumers don’t really care about the remaining 2.9 million book titles. However, the data paint a different picture. We analyzed Amazon’s sales patterns and found that 30% to 40% of sales are in books that wouldn’t normally be found in a brick-and-mortar store. (See “Share of Amazon Sales Above Rank 100,000.”) Moreover, we found that the consumer surplus created by providing access to these relatively obscure book titles exceeds $1 billion annually.2

And it’s not just with books that consumers have been able to indulge their taste for variety online. We found a similar pattern in the other markets we examined. (See “Product Variety Comparison for Internet and Brick-and-Mortar Channels.”) Wired editor Chris Anderson has dubbed this phenomenon “The Long Tail,” and his analysis has identified many other IT-enabled markets where consumers’ preferences have greater depth than what one could find in a typical brick-and-mortar storefront. Examples include consumers’ preferences for music at Rhap-sody.com, movies at Netflix and custom news and information through various blogs and online communities.3

While the emergence of the Long Tail is now indisputable, two key questions remain: What factors are driving this change, and what are its implications for the structure of markets? (See “Anatomy of the Long Tail,”)

Supply-Side Drivers

For all brick-and-mortar businesses, stocking decisions are driven by the same basic constraints: how many products can be provided in a limited amount of shelf space, and how many consumers in the local geographic area are willing to pay for these products. The characteristics of IT-enabled markets change both the cost and benefit side of this equation. From the perspective of costs, brick-and-mortar retailers must allocate costly shelf space for each product in each of their locations. On the Internet, the cost of stocking an additional product is much lower, involving space in a centralized warehouse that is often located on more inexpensive real estate. For products that can be drop-shipped from distributors or digital products (such as e-books, MP3 music and database products) that can be sent over the Internet, almost all it takes to stock an additional product is an additional line in a product database, which can then be used to dynamically generate the relevant Web pages on demand.

On the benefit side, brick-and-mortar retailers sell to consumers in their local geographic region. Consumers with mainstream tastes will be served before consumers with one-in-a-million tastes. Internet retailers, on the other hand, can aggregate demand on a national or even global scale. With the potential Internet market approaching a billion consumers, even if you have one-in-a million tastes, there are still over a thousand like-minded consumers who share your niche tastes.

Similarly, IT systems can change production costs for products directed at a niche audience. Print-on-demand systems for books are a good example. Books printed using traditional offset-printing technologies are profitable only in volumes of 1,000 or more, and not all books have a sufficient readership to justify such a print run. Large print runs also involve the risk associated with the initial printing costs for a title with unknown demand. Using print-on-demand technologies, authors and small publishers can print individual titles for around $3 per copy. Further, because they can be printed one at a time, authors and publishers don’t need to incur upfront costs from large initial print runs and the associated risks from uncertain market demand.

IT systems can lower production costs in other ways for products in the Long Tail. For example, when author Sharon Deubreau needed an illustrator for a children’s book, she posted her project to Guru.com, and within four days had 46 responses from experienced illustrators all over the world. This reduced her search costs associated with obtaining bids on her project and provided a larger set of bidders — and thus potentially lower prices and a better match to her skill needs. Not surprisingly, Guru.com has become a popular site for small publishing houses and self-published authors in search of illustrators. Guru.com currently has more than 1,800 professionals from 71 different countries who list “Child Book Illustration” as a primary skill.

Similarly, music, movies, games, news and journals are becoming all-digital. This lowers the production, distribution and promotion costs, opening up niche markets. On the production side, when CNN covers stories in remote locations it used to send both a camera person and a reporter and wait for the story to be rushed to a satellite uplink station. Now CNN can send one reporter equipped with a digital camera, a laptop and a satellite phone. The reporter can record, edit and transmit the story in a fraction of the time, at a fraction of the cost. This allows CNN to cover stories that otherwise would not have been cost-effective.

On the distribution and promotion side, up-and-coming recording artists are using Web sites and online networking sites such as MySpace.com to connect with fans and distribute promotional copies of their songs — bypassing radio stations, and brick-and-mortar retailers that previously would have served as gatekeepers to promotional and distribution channels.

Demand-Side Causes

The more products retailers make available, however, the harder it is for consumers to locate the product in which they are interested. In fact, consumers can become overwhelmed when choices are poorly organized, and they may actually reduce their purchases as a result. Thus, the Long Tail makes it critically important that retailers provide tools to facilitate the discovery of products through both active and passive search.

Active search tools allow consumers to locate products easily in which they know they are interested. Sampling tools, such as Amazon.com’s samples of book pages and CD tracks, allow consumers to learn more about products in which they might be interested. Active search tools can also help consumers identify products of which they weren’t previously aware. For example, while searching for his last name on Google.com, Bernie Robichau came across a reference to his great-uncle Earl Robichau in a book indexed by Google’s Book Search feature. According to Mr. Robichau, this discovery “prompted me to buy two copies of a book that I never would have known about.” Mr. Robichau is not alone — comScore Networks Inc. estimates that Americans made 6.6 billion searches online in April 2006, and Page Zero Media estimates that paid search advertising will be a $15 billion business in 2006.

Passive tools, such as most recommender systems, use the consumer’s revealed preferences from past purchases or even page views to identify new, interesting products. Consumer search is also facilitated by tools combining both active and passive search, such as customer product reviews, online communities or the readership of product-focused blogs. Such product-focused blogs as the-gadgeteer.com and Giz-modo.com have become important outlets for both established companies and small inventors to promote new products. For example, John Patterson, the founder of Bathys Hawaii Watch Co., says, “Our sales and Web site hits went through the roof ” after the company’s watch was discussed on the Gizmodo blog — with Web site hits increasing from 60 to 1,800 per day and sales increasing 300%.

Our research suggests that search tools can also be very effective in allowing consumers to discover and purchase products they otherwise would not have considered, resulting in changes in sales distribution among a company’s products.4 We analyzed consumer purchase data collected from a retailing company that has both an Internet channel and a print catalog. The company sells the same product selection in both channels, and they have the same pricing and shipping policies. However, because of search, browse and recommendation tools that are unique to the Internet channel, product sales are significantly more evenly distributed on the Internet than through the catalog. For the catalog channel, the top 20% of products generate just over 80% of the company’s sales, nearly mirroring the widely used Pareto principle or so-called 80/20 rule, which generally holds that 20% of anything produces 80% of the results. However, at the company’s Internet channel, the top 20% of products generate barely 70% of sales. Since the same products are available in both channels at this retailer, this shows that the demand-side drivers of the Long Tail phenomenon can operate independently of the supply-side drivers, such as virtual shelf space.

Second-Order Effects of the Long Tail

Powerful as these effects are, we predict that they will be amplified over time because of second-order, or positive feedback, effects for both consumers and producers. For producers, the Long Tail will change the kinds of products that are profitable. In a world where only products with mass-market appeal make it to store shelves, producers have strong incentives to focus on mass-market tastes, to the exclusion of niche audiences. Expanding the types of products that can be profitably sold by retailers may provide artists and other producers with incentives to create products to serve more varied tastes.

We may already be seeing examples of this phenomenon. As noted above, print-on-demand and other IT systems have significantly lowered the costs to produce books targeted at niche audiences. It is possible that these changes are driving an increase in the number of new books produced each year. R.R. Bowker, the official U.S. agency for assigning International Standard Book Numbers, has reported that the number of new titles has increased by 72% since 19955 — with most of the increase coming from small publishing houses and the self-publishing model. Sites like Lulu.com and CafePress.com have sprung up to serve this market by offering on-demand printing and promotional services for books produced by niche authors. Lulu.com says that it publishes 2,300 new books a month, mostly from authors who wouldn’t otherwise have access to major publishing houses.

The music industry has seen a similar effect owing to changes in the cost to produce, market and distribute music. Niche bands can reportedly turn a profit with sales of 25,000 albums, compared to break-even points of 500,000 through major labels.6 Other examples of “new products” include news and commentary from any of the more than 40 million blogs tracked by Tech-norati.com, or audio and video Podcasts catering to niche audiences, such as Paula Berinstein’s “The Writing Show” for aspiring authors or Don McCallister’s “Screencasts Online,” which offers video training on Macintosh software.

Managers should understand that the underlying economic principles are not new. More than 200 years ago, Adam Smith observed that “the division of labor is limited by the scope of the market” because of the need to amortize fixed costs. What has changed is the technology and thus, both the size of the addressable market and the relevant fixed costs of production and distribution. Today, an author, singer or film producer can cost-effectively reach billions of potential consumers via the Internet and similar technologies. This makes it profitable to invest time and effort to create products that might be of interest to even just a small fraction of them.

The Long Tail can also facilitate second-order changes in consumer tastes. As markets with increased product variety, and increased information about these niche products, allow consumers to discover and purchase products that otherwise would be unavailable, consumers are led even further down the Long Tail. They begin to cultivate deeper tastes for these niche products, just as Professor Clemons refined his tastes for India pale ales.

Our research found that consumer tastes are far more varied than one would expect from seeing the limited choices of products in brick-and-mortar stores. Moreover, consumers receive a great deal of value from having access to these products.

Finally, IT-enabled tools can enhance the effect of Long Tail markets by allowing consumers actively and passively to discover products that they otherwise would not have considered. Millions of these consumers have chosen to become content producers themselves, highlighted by the emergence of Web 2.0 tools that facilitate information sharing and social networking. For example, at a typical brick-and-mortar retailer almost all product information comes from salespeople or the manufacturers themselves. In contrast, at Amazon.com the majority of product information is created directly by customers in the form of reviews, personalized profiles, product lists and ProductWiki entries, and this information can strongly influence consumer purchase decisions.7 Similarly, while social networking sites like MySpace.com can help musicians with supply-side promotion and distribution, they can also help consumers discover new artists by checking who is linked to their favorite artist’s “friend space” section. The ability of consumers to openly share product information creates numerous opportunities for artists, entrepreneurs, marketers and IT developers to shape and benefit from these new markets for information.

Niches don’t translate to riches for everyone, however. Clearly, the accelerated Darwinism created by improved search can erode the market for blockbuster hits if they simply aim for the least common denominator. Ironically, even niche artists could be hurt in some scenarios — if record labels invest in new acts mainly in hopes of developing a superstar, then a reduction in platinum hits could mean lower incentives for investing in new talent. There can also be societal and political implications if consumer tastes become Balkanized and common experiences become, well, less common.8 Will democracy and social cohesion suffer if each voting group reads its own custom news feeds and commentary while experiencing only carefully tailored movies, music and videos? The changes in the underlying technology and economics of production, distribution and consumption are compelling. As these drivers grow in importance, the only clear winners will be business strategies that take the Long Tail seriously.