Every Marathon Runner Has a Story – And So Does Your Data

At least once a year I sit down to watch one of the many televised marathons that happen in cities all around the world. Recently, as the helicopter camera panned away to show thousands of runners at the start line, it occurred to me that this marathon was not unlike the data within our business.

I looked these many thousands of runners and realized that being in the same place at the same moment did not mean all of the runners were equal. Some were young, some were old, some were fast, some were slow. Some were from isolated backgrounds, some had seen the world change a hundred times. Some were there for the first time, some would not return next year.

Now, I don’t watch the race to see who wins. That’s irrelevant. What I want is the stories. I want to see the interviews with the charity runners and hear the story of how they got here. And of course, every story is different. This richness of information helps me to understand the importance of the race, and what encourages millions of people every year to push their bodies past the point of expectation over 26.2 miles.

The data within a business is like this too. When you look at a report, a table, a dashboard, the data that makes up what you can see has come from many different places and is of varying quality. Some you will see again; some you are seeing for the last time.

Why does this matter? Well, to be truly confident of the insights you are deriving from your data you should be supremely confident about the source of the information, the story behind the data. By knowing this, you gain the understanding of why the insights are important.  

Within your business, you can’t interview a piece of data, but you can understand its story. You’ll know this as data lineage. Understanding where the data is from, who has touched it, what has happened to it and, ideally, where it is going and which other data and applications it will affect. Data lineage should never be an afterthought. It should be the first question before introducing any new dataset into your business or into your data workflows.

Being able to record and track data lineage is critical to having confidence in your business reporting and subsequently the decisions your business is choosing to make.

To learn more read our eBook A Data Integrator’s Guide to Successful Big Data Projects.  This eBook will guide through the ins and outs of building successful big data projects on a solid foundation of data integration.