The four DORA metrics — deployment frequency, lead time for changes, change failure rate, and mean time to recovery — have become the standard language for measuring engineering performance. They are well-researched, widely adopted, and genuinely useful for what they measure: the health of the delivery pipeline. How quickly can a team push code to production? How often does that code break? How fast can they recover when it does?
These are important questions. They are also, for the VP of Engineering or VP of Product accountable for a strategic outcome, the wrong questions to stop at.
DORA metrics tell you how fast the team is moving. They do not tell you whether the team is moving toward the goal. And in a scaling SaaS organization — where roadmap drift is gradual, misalignment is structural, and execution risk compounds silently — the gap between velocity and direction is where the most consequential problems live.
What DORA measures well
Credit where it belongs. DORA metrics brought rigor to a domain that was previously governed by intuition and anecdote. Before DORA, engineering performance was assessed through a mix of story points, lines of code, and gut feel — metrics that were either gameable, misleading, or both.
The DORA framework solved this by focusing on outcomes at the pipeline level. Deployment frequency measures how often a team delivers value to production. Lead time measures the speed from commit to deploy. Change failure rate measures the reliability of the delivery process. Mean time to recovery measures resilience. Together, they give a structured picture of delivery pipeline health — and the research behind them demonstrates a clear correlation between strong DORA performance and organizational outcomes.
Engineering analytics platforms have built entire product categories around these metrics and their extensions: cycle time breakdowns, pull request throughput, work-in-progress distribution, bottleneck identification. The dashboards are sophisticated, the benchmarking data is extensive, and the measurement infrastructure has matured significantly.
If the question is "how healthy is our delivery pipeline," the tools exist and they work.
The question DORA doesn't ask
But the VP sitting across from the board, or the product leader preparing the quarterly executive review, is not being asked about the delivery pipeline. They are being asked: did we ship what we planned? Is the roadmap on track? Are we converging on the strategic outcome we committed to?
These are direction questions, not velocity questions. And DORA metrics are structurally silent on direction.
A team can have excellent DORA numbers — deploying frequently, with low lead time, minimal change failure rate, and fast recovery — and still be drifting from the roadmap. Because DORA measures the efficiency of the pipeline without reference to what's in the pipeline. If the team has been pulled toward urgent customer escalations, or if a dependency shifted priorities mid-sprint, or if the work being delivered is technically sound but strategically misaligned, the DORA dashboard won't show it. The pipeline is healthy. The trajectory is off. The numbers are green. The quarter still surprises.
This is not a flaw in DORA. It is a scope boundary. DORA was designed to measure delivery performance, and it does so effectively. It was not designed to answer whether the organization's actual execution pattern is converging on its stated goals. That is a different measurement entirely — one that requires synthesizing signals across the organization, not just within the delivery pipeline.
Why the dashboard doesn't interpret
Engineering analytics platforms — the category built around and beyond DORA — share a common architecture: they instrument the delivery pipeline, extract metrics, and display them on dashboards. Some add trend analysis. Some add team-level comparisons. Some add predictive delivery dates based on historical throughput. All of them present data for the leader to interpret.
This is the architecture of a dashboard: data in, display out. The leader provides the interpretation. The leader decides what the 12% drop in deployment frequency means. The leader connects the rising work-in-progress count to the cross-functional coordination problem they heard about in a 1:1. The leader is the interpretive layer.
For many use cases, this works. An engineering manager reviewing their team's cycle time distribution can interpret the data because they have direct, first-hand context. They know who is working on what. They know what changed last sprint. They can connect the metric to the cause.
But the VP of Engineering overseeing six teams does not have first-hand context for each one. The VP of Product managing a roadmap that spans engineering, design, and cross-functional dependencies cannot interpret a delivery metric in isolation — because the delivery metric doesn't contain the organizational signals that would make it meaningful. A velocity dip on Team A might be caused by a blocked dependency from Team B, a coordination change driven by Team C's re-prioritization, and a capacity shift that originated in a hiring decision two months ago. The DORA dashboard for Team A shows the dip. It does not show the system.
This is where the gap between engineering analytics and execution intelligence becomes visible. Engineering analytics measure the pipeline. Execution intelligence interprets the organization. The first tells you what the delivery system produced. The second tells you whether what the organization is actually doing is consistent with what it planned to do — and if not, where the divergence started and what's driving it.
The missing layer
What the VP of Engineering needs — and what no engineering analytics platform provides — is the interpreted synthesis that connects delivery metrics to organizational behavior to strategic direction.
That synthesis requires reading signals beyond the delivery pipeline. It requires behavioral metadata: the patterns of communication, coordination, and task movement across the organization's tools. Not just how fast tickets move, but which tickets are moving — and whether the pattern of movement reflects the priorities in the roadmap or a different set of priorities that emerged without anyone consciously choosing them.
Continuous roadmap monitoring operates at this level. It reads behavioral metadata across the full surface of an organization's collaboration, project management, and development tools — and interprets the pattern against the stated plan. It produces two measures. Momentum tracks whether the organization is converging on its goals — not output volume, but directional progress. Confidence tracks whether the underlying signal is strong enough to trust the reading.
This is not a replacement for DORA metrics. It is a different layer. DORA tells you whether the delivery machine is healthy. Monitoring tells you whether the delivery machine is pointed at the right target. Both are necessary. Neither is sufficient alone.
The distinction matters because the failure mode they address is different. Poor DORA metrics indicate a pipeline problem: slow deployments, high failure rates, long recovery times. Poor Momentum indicates an alignment problem: the organization is executing efficiently but diverging from the plan. The dashboard that flags one will not flag the other. And for the VP accountable to a strategic outcome — not a pipeline metric — the alignment problem is the one that determines whether the quarter delivers what was promised.
The question that follows "how fast"
Engineering analytics earned their place by making delivery performance visible and measurable. That contribution is real. But the evolution from measurement to intelligence requires a shift in the question being asked.
"How fast are we shipping?" is a pipeline question. The tooling for it is mature.
"Are we shipping the right things?" is an execution question. It requires reading organizational behavior — coordination patterns, cross-team communication, task trajectory, capacity allocation — and interpreting whether the real pattern of work matches the planned one.
"What does this mean for the roadmap?" is an intelligence question. It requires synthesizing delivery data, behavioral signals, and strategic context into a plain-language picture that a VP can act on without assembling the interpretation themselves.
Each question builds on the one before it. The first is well-served. The second is underserved. The third is almost entirely unaddressed.
The VP who stares at the DORA dashboard and feels like something is missing is not imagining things. The dashboard is answering a question. It's just not the question the VP is actually asking. The pipeline is healthy. The trajectory is unknown. The distance between those two statements is the distance between engineering analytics and execution intelligence.
DORA tells you how fast. It doesn't tell you where.
