We inhabit a world of blinding technological change. New devices, new programs and new infrastructure rise up, dominate discourse and pass away before we even have time to comprehend their intent. But for all the change we've experienced, the the most profound transformation of the digital era is really just getting started. Welcome to the era of Big Data.
For the sake of clarity, let's recall what Big Data means. At the intersection of the Internet, digital recording technologies and a profusion of small-scale wireless sensors comes an exponential increase in stored data. We're recording information of every kind: tweets, engine temperatures, Facebook photos, stock trades, grocery store purchases. These data points pile up, billions upon billions, trillions upon trillions, recording our lives and the life of the world we inhabit. They are very, very valuable to people with the ability to see patterns in, and extract insight from, the blizzard of information blowing out of the cloud.
Ultimately, the promise of Big Data is the ability to understand (and control) a seemingly chaotic world on levels never before imagined. The dangers of Big Data stem from that very same promise. Its impact on society will be akin to the transformative effect of past technological revolutions.
Consider thermodynamics, the all-embracing theory of energy and change that led us to ideas like entropy and the arrow of time. Thermodynamics traces its roots to the steam-power revolution of the 19th century. It was the rapidly expanding industries of power generation, of railroads and steamships, that gave birth to the new worlds of human behavior that emerged from the industrial revolution. In the tumble and tumult of all that change, scientists recognized new possibilities for describing and understanding the world and the science of thermodynamics was born. Soon that science grew beyond the realm of steam power to touch everything from cosmology to biology.
As strange as it may seem, Big Data may be the steam engine of our time.
Not everyone, however, is enchanted by the ever-burgeoning sets of numbers at our disposal. Emanuel Derman, one of the pioneers of Wall Street financial modeling, says this in an excellent piece for Frankfurter Allgemeine Zeitung:
Some talking heads like to say that computer-aided analysis of patterns will soon replace our traditional methods of discovering the truth, in medicine and the social sciences as well as in physics.
Derman then warns against our penchant to become "bewitched" by data, to believe the data can speak for itself. He argues that human intuition is a crucial ingredient in our quest to understand the world.
Big Data is useful, but is not a replacement for the classic ways of understanding the world. Data has no voice. There is no "raw" data. Choosing what data to collect takes insight; making good sense of it requires the classic methods: you still need a model, a theory, or intuition to find a cause.
While this point is well taken, I believe there is something real and powerful happening in the Big Data revolution. It's more than just a fad. It's the next link in the long chain connecting culture and technology to human history.
Now Big Data — seen and unseen — is hitting us in all corners of our lives, from the price of things to traffic patterns to who our social networks think we befriend.
Through new fields like data science and network theory, Big Data will not only change the world we move through as individuals, it will change the world we imagine through science.
Like it or not, Big Data will only get bigger (and bigger).
You can keep up with more of what Adam Frank is thinking on Facebook and on Twitter: @AdamFrank4
Copyright 2020 NPR. To see more, visit https://www.npr.org. 9(MDAwMTM1NDgzMDEyMzg2MDcwMzJjODJiYQ004))