When the CEO asks for engineering metrics, the first mistake most CTOs make is thinking it’s a single question with a simple answer. It’s not. It’s four very different questions wrapped into one, and answering the wrong one wastes time and erodes trust.
Will Larson’s framework breaks engineering measurement into four categories. First, measure to plan: are we working on the right things? Show the board how engineering time maps to business impact—features that move revenue, infrastructure that prevents outages. That’s what the CEO and board really want to know. Second, measure to operate: is the system healthy? Incidents, downtime, latency, cost ratios—these explain why a roadmap might slip and help prioritize firefighting over feature work. Third, measure to optimize: are we getting faster? DORA metrics like deployment frequency and lead time live here, but these are for engineering leadership, not the board. Without technical context, they’re meaningless to most non-engineers. Fourth, measure to inspire: what’s the story of engineering’s impact? This is where you share narratives that turn engineering from a cost center into a strategic advantage—how a platform rebuild cut feature delivery from six weeks to two days, for example. It’s the hardest category to build and the easiest to skip, but it’s what wins you headcount and support.
And it gets worse. Even if you pick the right category, dashboards often destroy trust in predictable ways. Goodhart’s Law warns us that when a metric becomes a target, it stops being a good measure. Developers are smart; they’ll game any metric tied to performance reviews. AI only makes this worse—code volume becomes meaningless when AI can churn out pull requests in bulk, inflating output without adding value.
Another trap is measuring individuals instead of teams. Software is a team sport. Trying to isolate individual contributions is like measuring a piston’s output in an engine—it makes no sense and poisons team dynamics. McKinsey’s disastrous attempt to measure individual developer productivity proved this painfully clear.
There’s also the measurement loop: stakeholders ask for more dashboards, more metrics, and nothing satisfies them. This is not a metrics problem; it’s a trust problem disguised as data. No dashboard fixes a broken relationship between engineering and the business.
Plus, optimization metrics like cycle time or deployment frequency belong in engineering leadership meetings, not in front of the board. Presenting them to CEOs without context leads to misinterpretation and dangerous targets that drive the wrong behaviors. CEOs and boards want planning and operations metrics, period.
Finally, perfection paralysis kills progress. Some CTOs wait for the perfect framework, the perfect tools, and never start measuring at all. Start with what you have, measure the easy stuff first to build trust, then iterate.
Now AI adds a new layer of complexity. According to the 2025 DORA report, increased AI adoption correlates with a drop in delivery throughput and stability. Cortex’s 2026 data shows PRs per author up 20%, but incidents per PR up 23.5%, and change failure rates up 30%. AI speeds up coding but clogs the pipeline with more broken code that takes longer to review and fix. Deployment frequency alone no longer signals progress; you have to pair speed with quality metrics like rework rate, which DORA added in 2025 to capture unplanned follow-up work caused by production issues.
So where do you start? Focus on a minimum viable dashboard: delivery predictability—did you ship what you promised? System reliability—incidents and recovery time. Investment allocation—where engineering effort went. And team health—attrition, hiring, engagement. Report metrics you’re already tracking, show trends not snapshots, and never show speed metrics alone. Deployment frequency without change failure rate is a lie by omission.
Metrics are like a thermometer. They reflect the health of your engineering practices but aren’t goals themselves. If your dashboard creates theater instead of insight, you’re confusing the map for the territory.
When the CEO asks for engineering metrics, ask yourself: which of the four categories are they really after? Would your current metrics survive scrutiny under Goodhart’s Law? What story is your dashboard telling—one your engineers would own, or one they’ve learned to perform? If you stripped away activity metrics and kept only outcome metrics, what remains?
Peter Drucker didn’t say “what gets measured gets managed.” He said knowledge work can’t be measured like manual work. Tom DeMarco, who famously claimed you can’t control what you can’t measure, later retracted that. Measurement isn’t the goal—understanding is. Use metrics to make better decisions, not to create theater.
You can read the full article—with all the data and sources—on ThePragmaticCTO Substack.
Read the full article — with all the data and sources — on ThePragmaticCTO.
