You enter the budget review expecting to see the new dashboard your team was building over the past two months. The CFO opens a spreadsheet instead. Burned once by a number that turned out to be stale, the CFO isn’t about to be burned twice in front of the board. When the CEO asks for the numbers they actually trust, neither one is what your team built.

I think the question that changed things for me wasn’t “What do you need from us?” Instead, it was the question the room never asked: “What specific business choice breaks without this tool, and is keeping it alive worth sacrificing the engineering time needed for our next priority?”

I’ve been the person walking into that room more than once, including at a company where I served as fractional CDO and inherited hundreds of dashboards. Each time that I asked analysts which ones we could retire, the answer was always the same: the business needed them for Black Friday, for the quarterly review, or for annual planning. Nothing could come off the list. When I asked the engineers the same question about the pipelines and data sources behind those dashboards, the answer simply moved one link down the chain: the analysts need the data. Nobody owned the decision to stop – everyone was just enabling the person next in line. 

And whenever someone actually opened one of those dashboards to make a real decision, it usually triggered an incident. The data was wrong, stale, or quietly broken somewhere upstream. We used monitoring tools that were genuinely good at their job. They caught errors about twenty-four hours after they happened. They never once prevented the next one, because they were solved and forgotten.

In practice, that meant that the team spent almost all its time responding to what had just broken, not building what came next. I had to repeatedly explain why a budget that looked expensive belonged to people who were entirely competent and still couldn’t keep up. The moment anyone shipped something new, it joined the same rotation and produced its own incident within a few weeks. 

The problem was never that the team lacked skill or the org lacked tools. The problem was that nothing was ever allowed to actually end, so everything eventually needed defending at once, leaving little time to do the thing a data leader is actually hired to do.

Evaluating the Business Value of Data Products 

Starting with what we keep rather than what we need changes what the team is actually optimizing for. Many tools get built to answer a question someone asked once, without anyone going back to check whether the question still matters. The rare tool that’s still worth keeping isn’t the one that answered a question three years ago and has since been running out of habit. It’s the one that changed how a decision actually gets made, and you keep it for that reason, not because deleting it feels risky. 

What was missing back then wasn’t a way to say no. It was a language for what any given data product was actually worth, and how much time we were spending to keep it running instead of working on something else. Every dashboard had a defender. None of them had a value. 

Deciding Which Data Products Are Worth Keeping 

What actually changes a meeting is not a better kill list or more documentation, but a standing habit. Every quarter, before the next roadmap review, someone puts a name and a business owner next to each live product. That owner then produces one sentence – not a slide – answering one question: “What specific business choice breaks without this tool, and is keeping it alive worth sacrificing the engineering time needed for our next priority?” 

“It gets a few hundred views a month” doesn’t count as an answer. A better answer is: “Without this, finance can’t close the books on time, and that’s worth the two hours a week it costs us.” If no one can give that kind of answer for a tool that’s been running for a year, it’s a candidate for retirement before you’re defending it in the next budget meeting. 

Sadly, out of all the dashboards we had, very few were still worth keeping. The review exposed a deep mistrust in the data: having more but using less, and spending hours trying to find what we needed. 

The quarterly review habit is harder than it sounds, and I’m not sure the problem is ever fully solved. Portfolios accumulate faster than anyone wants to review them. Every data leader I’ve talked to about this issue agrees that it’s easier to describe than to do consistently, quarter after quarter. But data leaders who are consistent, even if imperfect, about conducting quarterly reviews walk into the budget meeting with something the CFO can trust. Not a pitch for the team’s work, but an honest accounting of it, completed before anyone asks.  

So if you want to know what in your own portfolio is still worth keeping, don’t start with the list of what to cut. Start with the last product your team shipped. Could the product owner answer one question today, in one line: What specific business choice breaks without this tool, and is keeping it alive worth sacrificing the engineering time needed for the team’s next priority? If the answer is longer than a sentence, have a conversation before the next roadmap review, instead of when you’re standing in front of a CFO with a spreadsheet already open.