What We Can Learn About Predictive Analytics From The Blizzard That Wasn't

We are drowning in data. According to Nate Silver we now produce every day, three times per second, the equivalent of the amount of data that the Library of Congress has in its entire print collection. 

That’s a lot of data!

And it’s only going to get worse. Each of us is in some part responsible for the deluge. Every move we make on our Internet browsers and on each social media network we engage in is being tracked. Our comings and goings, our likes and dislikes are all helping to make Big Data bigger.  

One of the hottest new jobs is that of data scientist, a person whose job it is to analyze these enormous piles of data for insights that can predict. Predict where we want to travel to, what we want to eat for dinner, what shoes we want to buy, who we want to work for, who wants us to work for them, who we want to date.

Sometimes it works quite well.

Right around Christmas I was looking for new snow boots. I looked in a few stores, but for the most part my research was being done on line. I knew what I wanted. Comfortable with a good tread. After last winter I wasn’t as concerned with the fashion statement I was making as I was with not slipping on the ice. But I  still wanted something that said I had some sense of style. 

Boots started to follow me everywhere I went. On my iPhone, my iPad and my desktop. 

And then the result of all my research appeared right in the middle of my Facebook feed on the day after Christmas. 

It was an ad from Amazon for a pair of Merrell boots that was on sale. 

I had never bought a pair of Merrell’s before, nor had I ever bought an article of clothing on Amazon before. But the price was right so I clicked through. I read about the boots. I went to the Merrell website. I read the reviews.  

I decided to go for it.  I took advantage of my Amazon Prime membership and free shipping and hit the buy button. 

The boots fit perfectly. They are comfortable and keep my feet dry. All those cookies that had been tracking me served me up what I wanted. I am a happy customer.

But it doesn’t always work so well. 

Sometimes we get it right. Sometimes we don’t. 

Last week’s blizzard was is a case in point. 

We have more data than ever before to predict the weather. We want to believe all those insights are going to give us the right answer. But as we saw - especially if you lived as I do in New York City - there was no “historic snowstorm” as the data was indicating and all the meteorologists and politicians were standing behind. 

Instead of backpedaling afterwards, all one needs to do is understand the truth of the data going in. 

It informs. 
It garners valuable insight. 
It can direct decisions. 
It can assess risk or potential. 

But not every behavior or weather pattern can be predicted. 

Things will happen. Things that we never accounted for. 

Like that weather band that decided to skirt New York City last week.