Data and analytics are making it harder and harder to keep anything secret.
Secrets may be an unexpected casualty of increasing analytical prowess. Just ask Volkswagen. Their data seems to have been as dirty as exhaust from their cars, which were recently found to be exceeding U.S. emissions standards. The Environmental Protection Agency asserts that the carmaker manipulated software so that performance differed during testing. Analysis of data from real driving conditions later indicated something might be amiss. While the secret lasted a while, it didn’t last forever.
Volkswagen is not alone. Secrets in general don’t seem to have much shelf life now. Examples are everywhere. Credit card infrastructure that requires keeping card numbers secret seems to fail again (e.g., PF Chang’s) and again (e.g., Home Depot) and again (e.g., Neiman Marcus) and again (e.g., Dairy Queen) and again (e.g., Jimmy John’s). In the security context, certainly in the aftermath of Edward Snowden’s disclosures, there is a realization of the “declining half life of secrets” — secrets that were once useful for intelligence purposes for 25 to 50 years are revealed much, much sooner now.
But it isn’t just the simple, though potentially devastating, loss of data. Yes, users’ secrets were revealed at Ashley Madison, taking executives’ jobs with them as they left. But the secrets of the users were not the only secrets lost. A frenzy of subsequent analysis of the data indicated more than just the private information of the users. Analytics on data can reveal business practices, some less than flattering — such as possible corporate espionage and potential misrepresentation at Ashley Madison.
And though secrets frequently have negative connotations, it certainly doesn’t have to be the case. For example, revelations can divulge corporate strategy that is not nefarious. Despite secrecy at Apple, such innocuous bits of information as restrictions on vacation scheduling or supplier preparations can point to future product announcements. Or in marketing, individuals may have preferred to be anonymous, but analytics identifies individuals and organizations — the “Identity Paradox” of big data — in order to personalize offers or provide relevant information. Or semi-private data can be analyzed at scale, particularly when reporting requirements combine with weak anonymization. For example, New York City officials released detailed records of 173 million taxi trips that, while encrypted, did not withstand modern analysis techniques for long. While not particularly despicable, people involved likely would have preferred that the details like these remain private.
Three recent changes have led to the difficulty in keeping secrets. First, increasing digitization of processes generates unprecedented data “exhaust.” Second, advances in systems, tools, and techniques make storing, processing, and analyzing the data easier and cheaper than ever before. Third, every discovery adds to the bank of knowledge about the potential uses (and, yes, abuses). The same changes that enabled the application of analytics to discover business insights also enable the application of analytics to discover secrets — the unknown information is agnostic and doesn’t know if it is an insight waiting to add business value or a secret waiting to harm.
Furthermore, managing in the aftermath of undesired disclosure is hard, if not impossible. Organizations have few tools at their disposal other than trying to bury information they don’t want seen with information that they would prefer. But once information is known, forgetting is far easier to require than to effectuate. What’s worse, if organizations aren’t careful, attempts to quell dissemination draw even more attention to the information — a phenomenon known as the Streisand Effect.
From this perspective, analytics has truly opened Pandora’s box. But the same analytical tools that expose secrets offer hope to at least mitigate reduce their effect — by making them a much less attractive alternative.
As exposure become increasingly inevitable, organizations then have incentives to act in ways that can stand scrutiny, both now and in the future. The Ashley Madison and Volkswagen examples may cause organizations to rethink activities they would prefer not see the light of day. There are two reasons. First, the monitoring that has come with data is a form of surveillance, and people change their behavior when they know they are being watched. Second, even for organizations that do not adjust their behavior, as the half-life of these secrets diminishes, so too does the time period during which organizations can profit from them.
As a result, getting a good return-on-investment for secret behavior should be harder. Organizations can adapt their secret activities by increasing their egregiousness (to increase their benefit) or by decreasing them (to reduce their eventual costs). It remains to be seen how our understanding of secrecy will evolve.