Secrets in the Age of Data

Secrets may be an unexpected casualty of increasing analytical prowess. Just ask Volkswagen. Their data seems to have been as dirty as exhaust from their cars, which were recently found to be exceeding U.S. emissions standards. The Environmental Protection Agency asserts that the carmaker manipulated software so that performance differed during testing. Analysis of data from real driving conditions later indicated something might be amiss. While the secret lasted a while, it didn’t last forever.

Volkswagen is not alone. Secrets in general don’t seem to have much shelf life now. Examples are everywhere. Credit card infrastructure that requires keeping card numbers secret seems to fail again (e.g., PF Chang’s) and again (e.g., Home Depot) and again (e.g., Neiman Marcus) and again (e.g., Dairy Queen) and again (e.g., Jimmy John’s). In the security context, certainly in the aftermath of Edward Snowden’s disclosures, there is a realization of the “declining half life of secrets” — secrets that were once useful for intelligence purposes for 25 to 50 years are revealed much, much sooner now.

But it isn’t just the simple, though potentially devastating, loss of data. Yes, users’ secrets were revealed at Ashley Madison, taking executives’ jobs with them as they left. But the secrets of the users were not the only secrets lost. A frenzy of subsequent analysis of the data indicated more than just the private information of the users. Analytics on data can reveal business practices, some less than flattering — such as possible corporate espionage and potential misrepresentation at Ashley Madison.

And though secrets frequently have negative connotations, it certainly doesn’t have to be the case. For example, revelations can divulge corporate strategy that is not nefarious. Despite secrecy at Apple, such innocuous bits of information as restrictions on vacation scheduling or supplier preparations can point to future product announcements. Or in marketing, individuals may have preferred to be anonymous, but analytics identifies individuals and organizations — the “Identity Paradox” of big data — in order to personalize offers or provide relevant information. Or semi-private data can be analyzed at scale, particularly when reporting requirements combine with weak anonymization. For example, New York City officials