Humans are predictable sorts. While we’re all slightly different, our biology and similarity of experience on the planet conspire to make us more like each other than not. The 2010 French movie Babies showed us that infants from the Mongolian Steppes, African bush, Tokyo and San Francisco all progress in nearly the same way, at the same rates. Does that surprise you? It surprised us.
Predicting
Professionals are aware of this. But until recent times, precision in predicting how people will behave was a challenge. There wasn’t enough data or enough technology to give data the inspection it needed. Instead, scientists conducted studies and marketers set up focus groups. It was all about getting the right ‘sample’ size and makeup.
Shooting fish in a barrel is a great metaphor for what’s changed. Big Data gives us the ability to process increasingly large volumes of preference and behavioral data, not from studies, but from real people. The more data, the more patterns of similarity emerge. The more refined the patterns, the greater the likelihood that a person’s likes, dislikes, and future actions can be predicted.
We can hit the target every time.
Is it too easy?
So when we agree to give people our information, we’re giving them the power to predict exactly how we’ll respond. We’re giving up, in a real, scientifically proven way, our free will. How many people realize this when they allow a cookie on their computer, or turn on geo location on their phone? How many people are targeted because of extraneous detail that makes them an ‘easy shot’?
When our preference for movies demonstrates our politics, Netflix is the new campaign bumper sticker.
Want to see what others have said? Head to Quora to see the discussion.
Solve for Interesting
Seth Godin Blog
Jeremiah Owyang
IT Redux
HBR Blog
Clay Shirky Blog
SocialMedia Today
Chris Dixon
Brian Solis
Chris Brogan
The TIBCO Blog
Venture Beat
Engadget
Wired
Lifehacker
Mashable
The Verge
GigaOM
TechCrunch
Certainly, for more than a century, life insurance companies have based their success on the predictability permitted by a great enough N over a great enough period of time. Put more generally, obviously we could not do a great many things we do without an adequate capability to predict the behavior of large populations. So I don’t doubt your assertion that ‘big Data” means we will get better and better in this respect, but with two caveats.
First, none of this means we will get any better at predicting an individual’s behavior, any more than physics can predict exactly which subatomic particle will decay. All we can say is that in a given population,we can predict that X individuals will behave a certain way with Y accuracy within a given time term.
Second, if a chaotic or random-like mechanism is driving the behavior we wish to predict, then we will not know when a particular behavior will occur, only that, if given enough time, it will occur with X accuracy. For example, many experts on North Korea expect that regime will collapse in some way over the next 20 years. But, since the timing and path of such events are very hard to predict, we are likely to be surprised when it does happen.
I can imagine how Big Data might address these two problems, but I think progress will be much slower than expanding our ability to predict more stable behaviors of populations, like buying behavior and the like.
Sincerely,
Kenneth Watman