How to work in a truly data driven way

Why data is dangerous

These days most companies preach the gospel of working in a data driven way. Most ambitious managers say that it’s important and perhaps even maybe have an anecdote about a coworker that actually created a data driven insight that was actually used in a significant way.

However it’s mostly lip service and to be frank, smoke and mirrors. Most companies can not work in a data driven way. It’s just not in their DNA or paradigm.

The great news is that individuals can actually be the Martin Luther to the Catholic church and use new insights to break up old structures and in the long run be a part of upending a unhealthy monopoly of power and knowledge of the clergy that is the political class of today’s management.

In the real world you don’t have to go as far to upend the current world order but to be truly working in a data driven way actually has this kind of weight and potential impact.

1. How to work in a data driven way

Why guess what your customers want when you can test it, over and over again, and get real answers on what they want? In today’s digital world, there is an infinite amount of data that companies can collect and analyze. Running online experiments to find out what works best for your customers can have a great impact on your business’ success. But in order for it to work, you need to be willing to be confronted every day by how wrong you are, and you must be willing to let your customers decide for you.

A few companies that have discovered the power of using data and online experimenting are Google, Facebook, Microsoft, Amazon, Expedia and Booking.com. These companies run hundreds, some even thousands, of online tests each year, which has helped them to improve their marketing and innovation. Although it has proven successful for these companies, there are still loads of companies that run only a few, or no, tests per year.

What can companies do to work in a truly data driven way?

2. Follow a scientific approach

You need to work with scientific methods, meaning that you avoid the common pitfalls of ordinary companies, like survivor bias and vanity metrics.

Survivor bias, or survivorship bias, is the logical error of focusing on the experiments that make it past some selection process, while forgetting about the ones that did not pass. This consequently leads to false conclusions and overly optimistic beliefs as previous failures have been ignored.

A vanity metric is something that you can measure that does not actually matter, and which can be easily changed or manipulated. These metrics do not represent business success, but are still used because many people do not understand that they do not tell much about the business, and since it looks positive and therefore inaugurates false hope. This is problematic as people make decisions based on data that does not say much, which leads businesses in the wrong direction.

To avoid common pitfalls, make sure to follow a process. Establish a data governance strategy where you have detailed instructions for how to add and modify existing data, how to analyze and interpret experiments, and what to do with experiments that do not make it past a selection process. For example, when you decide to collect new data, take a step back and think of what questions you want to answer. Then, think of what visitor actions that must be tracked and analyzed in order to answer those questions. This will guide which metrics that could be of interest, and will hopefully rule out vanity metrics.

Follwing a process, a scientific method, would fit under the “measure and learn” part in the “Build, Measure, Learn- Framework”, which is one of the central principles of Lean Startup.

3. Cultivate the right culture

When you have made a decision to take on a data-driven approach, you need to stick to it. Strong opinions should not win over empirical results of an online experiment, no matter who’s opinion it is. This is often more difficult than it sounds, as we tend to accept results that are in line with what we want, and discard results that are against our opinions. Set a vision before you introduce the new data-driven approach, and present rational arguments for how this will improve the business and facilitate decision-making.

To motivate employees to use a data-driven approach, set data-based goals. Data should be used in a way which supports the business, and thus its business goals.