(US & Canada) | Data Lineage Is a Much Less Solved Market Problem — Bigeye Co-founder and CEO

Kyle Kirwan, Co-founder and CEO of Bigeye, speaks with Derek Strauss, Gavroshe Chairman, and CDO Magazine Editorial Board Member in a video interview about his learnings around data management from his time as a data scientist at Uber, and how the space has changed over the last five years.

Bigeye is a data observability platform that helps data engineering and science teams ensure their data is accurate and reliable.

Kirwan spent five years at Uber where he was deeply involved in areas like data science, data visualization, and product management. From a data management perspective, he says that one of the things that Uber did, like many digital native companies, is invest in hiring data talent early in its life cycle.

This enabled the company to build data-driven and competitive products and services and make decisions based on data for various uses like allocating marketing spends, managing refunds, dynamic pricing, etc. 

Sharing a key learning from working at Uber, Kirwan says that putting data products directly into the core parts of your business helps create a flywheel around data. The more that a business gets value out of data products, the more one can justify investment into the data team, infrastructure, management, and science.

From his experience of working with enterprises, Kirwan maintains that this “flywheel” is not always in place in many businesses, and in some cases, the data is not trusted or reliable. This, in turn, affected the confidence of using data for business.

Kirwan then goes on to share two major takeaways around data management from running Bigeye for the last five years:

1. The problem was more acute with a larger data team than it was with the smaller data teams. With a variety of disparate data sources, larger teams had even more justification to solve the data reliability challenges.

2. Data lineage is a much less solved market problem.

He explains how Uber treats its toolchain with the Unix philosophy - each tool with a specific function can interoperate with others and be chained together to solve different types of problems.

Accordingly, Bigeye’s Lineage solution acts as a dedicated microservice that can be plugged into other tools.

CDO Magazine appreciates Kyle Kirwan for sharing his invaluable data insights with our global community.