• Big Data

When Big Data goes Wrong

Wart Fransen
Wart Fransen
May 30, 2017
3 Min read
When Big Data goes Wrong

Because Web Analytics is part of our core solution we have spent a lot of time pondering over and perfecting our use of Big Data. LeadBoxer has been in the industry for more than 15 years (both directly and indirectly) and we can say with fair certainty that we have learnt several things during our cumulative “service”. But we will be lying to ourselves if we don’t admit that sometimes Big Data can fail you. With that in mind, we wanted to write this piece on how to make sure that your company is using it the right way and it doesn’t lose touch with the individuals that make-up the data sets. In the spirit of honesty, I have to give credit to Martin Lindstrom’s book “Small Data”, because it is what gave me the inspiration for this article. In it, he makes a point that nowadays the corporate world has become blinded by Big Data and this happens because it is incredibly hard to describe emotions using data.

Data does not reflect emotions.

Martin starts his book with the famous example of LEGO. In 2002-2003 the company was going bankrupt so they used Big Data to determine that Millennials have short attention spans and get easily bored. This led LEGO to make their small, iconic bricks into huge, simplistic building blocks. This change only accelerated their decline so out of desperation, the company decided to go into consumers’ homes to try and reconnect with their once loyal customers. I am not going to reveal much from the book, but in a few words, after meeting with an 11-year-old German boy, they discovered that for children, playing and showing mastery in something was more valuable than receiving instant gratification. This made LEGO pivot again and after their successful movie in 2014 they surpassed Mattel to become the world’s largest toy maker. Now, of course there is much more to the story that is hidden in the background, but the bottom line is that patterns and trends can sometimes be misleading.

Correlation is not causation.

Big Data is all about finding correlation and patterns in human behaviour, but that is not always the whole truth. Take as an example Google; back in 2008 they believed they could use Big Data to predict flu outbreaks and launched a program called Google Flu Trends (GFT). The algorithm analysed search queries with the word “flu” to follow patterns but it did not take into account unrelated searches. And then in 2013 it failed dramatically by missing the flu season peak of 140 percent. This is not necessarily a failure of Big Data as much as it is a failure of people using it. Call it hubris or simple error of judgement. But the fact of the matter is that correlation is not always causation and sometimes Big Data is simply just too big and not all relevant.

Not all data is useful.

The problem with Big Data is that you are looking at the data you have collected and not necessarily the data that you need. Sometimes the numbers you have are not interesting or insightful. Or even worse, they are just a vanity metric. Like thinking that a lot of Twitter followers translates into actual real-life influence or that a lot of website traffic leads to high conversion. You only need to take a look at the Big Data Industry Report for 2016 by Prompt Cloud to see that the biggest issue for companies is how to get value out of the data they have collected. leadboxer-big-data

What is the take-home?

Data has become so rooted in our work that we sometimes use it almost exclusively. But the examples that I mentioned and all of the others that didn’t make the cut to be included in the article, show that it takes something more than data. Organisations need to keep in mind that data does not reflect emotions. Correlation does not always equate causation. And just because you have a lot of information it does not mean that it is useful or actionable.