Why Detailed Data Is As Important As Big Data

Sam Ransbotham, Assistant Professor at Boston College

Big data gets all the press these days, but as important — and perhaps even more important — is detailed data.

That’s according to Sam Ransbotham, an assistant professor at Boston College in the Information Systems department. He’s been at BC for four years, and before that he was at the Georgia Institute of Technology, where he got his PhD in IT management and his BA in chemical engineering.

Detailed data gives companies “the opportunity to try to figure out the ways that items, such as customers, differ,” he says. And that’s not just demographically, but in their behavior. “By observing detailed, transactional data, we can actually find much more interesting things than we can by lumping them into demographic groups.”

Ransbotham’s initial research interests were in security and risk, but that led him to analytics. “What really sparked my interest is how do you make sense of that much data, since detection system logs have data in huge numbers, the billions of records magnitude. How do you spot trends and how do you figure out what’s going on in those trends?” His research also now encompasses what he calls “more positive areas” of customer service and customer reviews and how those things are being used in marketing.

In a conversation with David Kiron, executive editor of Innovation Hubs at MIT Sloan Management Review, Ransbotham explains why detailed data can tell companies not just why someone did something but why they didn’t do something else, how hospitals don’t seem to face heightened malpractice risks when they install electronic medical record systems and what companies should and should not be worried about when their customers fire off real-time comments on Twitter.

You see all the talk about what big data is doing to the competitive landscape. What do you see as “big data”? What does that mean to you?

It’s easy to talk about the number of records, just the total volume, and there’s no question that that’s increasing and is huge. But more than just the size, is also the types of data. It’s transaction-level, detail-level data — such as clickstream data versus summary data. And that’s really the more interesting of the trends. The details are what give companies the opportunity to do so much more.