We are drowning in data. According to Nate Silver we now produce every day, three times per second, the equivalent of the amount of data that the Library of Congress has in its entire print collection.
That’s a lot of data!
And it’s only going to get worse. Each of us is in some part responsible for the deluge. Every move we make on our Internet browsers and on each social media network we engage in is being tracked. Our comings and goings, our likes and dislikes are all helping to make Big Data bigger.
One of the hottest new jobs is that of data scientist, a person whose job it is to analyze these enormous piles of data for insights that can predict. Predict where we want to travel to, what we want to eat for dinner, what shoes we want to buy, who we want to work for, who wants us to work for them, who we want to date.
Sometimes it works quite well.
Right around Christmas I was looking for new snow boots. I looked in a few stores, but for the most part my research was being done on line. I knew what I wanted. Comfortable with a good tread. After last winter I wasn’t as concerned with the fashion statement I was making as I was with not slipping on the ice. But I still wanted something that said I had some sense of style.
Boots started to follow me everywhere I went. On my iPhone, my iPad and my desktop.
And then the result of all my research appeared right in the middle of my Facebook feed on the day after Christmas.
It was an ad from Amazon for a pair of Merrell boots that was on sale.
I had never bought a pair of Merrell’s before, nor had I ever bought an article of clothing on Amazon before. But the price was right so I clicked through. I read about the boots. I went to the Merrell website. I read the reviews.
I decided to go for it. I took advantage of my Amazon Prime membership and free shipping and hit the buy button.
The boots fit perfectly. They are comfortable and keep my feet dry. All those cookies that had been tracking me served me up what I wanted. I am a happy customer.
But it doesn’t always work so well.
Sometimes we get it right. Sometimes we don’t.
Last week’s blizzard was is a case in point.
We have more data than ever before to predict the weather. We want to believe all those insights are going to give us the right answer. But as we saw - especially if you lived as I do in New York City - there was no “historic snowstorm” as the data was indicating and all the meteorologists and politicians were standing behind.
Instead of backpedaling afterwards, all one needs to do is understand the truth of the data going in.
It informs.
It garners valuable insight.
It can direct decisions.
It can assess risk or potential.
But not every behavior or weather pattern can be predicted.
Things will happen. Things that we never accounted for.
Like that weather band that decided to skirt New York City last week.
I told myself when I started this experiment that it was for me first. That I wasn't writing any of these posts with my readers in mind. Whatever I was inspired to write, I would write.
But then I caught myself the other day checking my Google Analytics. Trying to make some sense out of which posts are getting the most reads and which ones fall flat. What's getting a retweet and what is a lonely soldier. And then my head started spinning with what I should write that will get read more so I could get the stats up instead of what I wanted and needed to write.
The next thing you know I couldn't write at all.
Except I had to. I was committed to this 30 day experiment. And almost half way through I am not about to bail. So instead of staring at a blank page on my computer I stared at my Stats Overview. And then my inspiration clicked in.
I've been here before. I saw myself fall into it when I self-published my first novel. Checking Amazon every day for numbers of books sold and where my darling was ranked. It was that same obsession with numbers that had been drilled into me during my corporate career. That above all - above the quality of the work, above the process, what is most important at the end of the day are the numbers. Did we meet our KPIs? Are we pacing to meet our goals? How much money will this make us?
It's a failed premise. Yes, the metrics are important and in the age of Big Data we are now entering, there is no doubt there will be even more emphasis on numbers.
But an obsession with how may Facebook shares you garnered gets in the way of delivering good product. A fixation on how much revenue is hitting the bottom line gets in the way of creativity and productivity. It gets in the way of the work. And sometimes it gets in the way of writing a blog post.
note: This is Day #13 of a 30 Day Experiment. Here are the details on how it all started.
