Cost of Delay (CoD)

Cost of Delay (CoD) is a popular factor for work prioritization, applied in queue-based or iterative work scheduling. It can be powerful for rough (order of magnitude) comparisons, but also misleading if applied naively. This article explains the key ideas behind CoD, how to calculate it, applicability and limitations in the context of product work, and a brief history of the method.

As a prioritization factor, Cost of Delay originated in operating systems and queueing theory research in the 1950s and 1960s, and it’s now applied in many different industries, including healthcare, defense and manufacturing. Cost of Delay was first used in product development in the early 1980s, and became more widely popular through Kanban and the Scaled Agile Framework (SAFe) in the 2010s (it’s worth noting that SAFe defines CoD in a very specific and potentially problematic way).

This article deals specifically with using Cost of Delay in IT product development. The formulas, applicability and limitations are presented in that context.

What is the Cost of Delay?

Cost of Delay (CoD) is an estimate of value that would be lost if some work were intentionally postponed in order to deliver something else, or if it were delivered late. Usually, it is expressed as money per unit of time, for example dollars per month. Since cost of delay puts a price on waiting, in theory it makes it possible to compare items that address different types of work, such as investing in marketing, completing compliance tasks or reducing operational costs. Reinertsen suggests that “without a cost of delay, queues appear to be free and therefore, unworthy of attention”.

The major advantage of using cost of delay, according to Reinertsen, is that it helps to express changes in a single unit of measure (he argues that life-cycle profit impact is the ultimate measure of product development success), making them comparable. This builds a model of value and helps everyone focus on it. Joshua Arnold succinctly explains the benefits as “in the absence of information about Value, of course the system optimises for other things”.

Donald Reinertsen and Preston Smith outlined a definition of the Cost of Delay in a 1991 article in the Journal of Business Strategy:

Construct a life cycle profit and loss statement for the product, assuming it is delivered on schedule. Then, create another profit and loss statement estimating what would happen if the product were delayed by six to twelve months. Calculate the difference between these two scenarios (the cost of delay) and express this difference in dollars per month.

– Donald G. Reinertsen and Preston G. Smith, The Strategist’s Role in Shortening Product Development

In the 1997 book Managing the Design Factory, Reinertsen suggests that this method can be applied at three levels: a sub-process, a project, or the entire business. In more modern contexts with continuous product development, Cost of Delay is often applied to larger product ideas (which would map to Reinertsen’s sub-process).

Joshua Arnold and Özlem Yüce define the metric in terms of expected value for a specific feature:

The Cost of Delay of a feature is the value that could be generated over a given period of time, if it was available immediately.

– Joshua Arnold and Özlem Yüce, Black Swan Farming Using Cost of Delay: Discover, Nurture and Speed Up Delivery of Value

Because Cost of Delay is an economic measure, it often gets compared to other economic measures such as return on investment, internal rate of return, net present value or opportunity cost. James Goljan, Jonathan Ritschel, Scott Drylie and Edward White even equate it with opportunity cost.

Cost-of-delay is defined as the economic impact resulting from a delay in product delivery or, said another way, opportunity cost. In principle, CoD assesses the negative impacts resulting from changes to the priority of a project.

– James Goljan, Jonathan Ritschel, Scott Drylie and Edward White, Cost-of-Delay

Practically, the difference between Cost of Delay and more popular economic measures is in granularity and frequency of use. Measures such as ROI are good options for a single large decision (such as “should we fund this project?”). Cost of delay helps to create decision rules that teams apply during work without escalating to management.

Cost of delay also provides context for other types of process improvement. For example, Reinertsen explains that many development organizations measure cycle time and aim to reduce it, but when asked “how much life-cycle profit will decrease due to a week of delay, they don’t know”. In organizations that do not calculate the Cost of Delay, Reinertsen argues that “people working on the same project will give answers that vary by 50 to 1.” (Note that this statistic is presented with no sample, research method or reference.)

How to apply Cost of Delay for prioritizing product work

Cost of Delay itself is just a number (or more precisely a rate), and although it can be used to directly compare proposed plan items, it’s mostly combined with other factors. Several prioritization methods use CoD as a factor, including Weighted Shortest Job First (WSJF) and Cost of Delay Divided by Duration (CD3). In The Principles of Product Development Flow, Reinertsen suggests four sequencing principles based on the nature of work:

Reinertsen's sequencing principles F15 to F17 and F19, from chapter 7 of The Principles of Product Development Flow (2009).
PrincipleConditionRule
F15delay costs are homogeneousshortest job first
F16job durations are homogeneoushighest cost of delay first
F17neither is homogeneousweighted shortest job first (cost of delay divided by duration)
F19job duration is unknowntime-share capacity

Calculating the cost of delay depends on estimating value, which itself is a notoriously problematic issue in IT product development. It’s dangerous and misleading to treat CoD as a precise measurement. However, it can be a powerful tool for rough (order of magnitude) estimation, and can provide clarity for stakeholder decisions to compare many classes of work.

Goljan argues that applying cost of delay analysis upfront “provides an immediate and empirical way to demonstrate a potentially cost efficient list of priorities”, and that quantifying the dollar cost savings gives some idea of what features are the most important. However, he still warns that some features that provide significant improvements may be “challenging to quantify and justify with a simple dollar estimate”, so he suggests that “CoD assessments are best utilized as a supplemental tool to help prioritize features, but should not be considered a final optimal solution”.

How to calculate Cost of Delay

There are several methods for calculating the cost of delay in product development. The original Greenberger formulas are often used as the inspiration for calculation methods, but they are not directly applicable in product work because the underlying factors (such as expected business value and work duration) cannot be estimated with reasonable accuracy.

Riesener and colleagues at RWTH Aachen University created a working model for Cost-of-Delay calculation for projects that involves 15 mathematical formulas over 20 factors, including late delivery fees, schedule variance, projected costs and sales volumes and many others. Their work might be an interesting reference for people trying to create their own models, but it is presented as an example, without any proof of actual benefits or that it helps to prioritize projects, and it’s too complex for rough comparisons.

Reinertsen’s profit-and-loss variations

Reinertsen and Smith recommended calculating the difference of hypothetical profit-and-loss statements in four different variations: an introduction delay, a development budget overrun, a product cost overrun, and a performance shortfall. As an example based on representative data, Reinertsen, McNutt, Greiner and Hutchison present a case study of changing engines on an air-refueling tanker with a baseline value of $4,069 million, modeled in a spreadsheet and converted into decision rules.

Decision rules from Reinertsen, McNutt, Greiner and Hutchison’s 2002 tanker re-engining case study. The underlying cost and benefit data are representative rather than actual.

Deviation from planValue
1-month schedule slip$29.7M
1% development cost overrun$2.0M
1% production cost overrun$2.0M
1% performance shortfall$0.7M

Arnold’s expected value and urgency

Joshua Arnold defines Cost of Delay as “the partial derivative of the total expected value with respect to time”. To calculate the value, Arnold and Yüce suggest four “benefit types”, which primarily consider the likely impact on revenue and costs that a feature or project might lead to:

  1. Increase Revenue
  2. Protect Revenue
  3. Reduce Costs
  4. Avoid Costs

The numbers assigned to a feature or an idea from those buckets are then converted into money per week using “urgency profiles”, which represent how benefits materialize and decline over time. Arnold and Yüce provide examples for four types of urgency profiles:

Arnold's four urgency profiles, from Black Swan Farming Using Cost of Delay: Discover, Nurture and Speed Up Delivery of Value.
ProfileWhat happens when you are late
Short life-cycle, peak affected by delaylateness clips the peak and it never recovers
Long life-cycle, peak affected by delaya first mover holds the position; latecomers cannot catch up
Long life-cycle, peak unaffected by delaybenefits ramp to a plateau, so only the units of time lost matter
Impact of an external deadlinenothing is lost until the last responsible moment, then the cost steps up

Goljan’s public sector savings formula

Goljan and colleagues proposed a modified model for calculating the cost of delay for public-sector contexts, where revenue is not relevant. Starting from Arnold’s categories, they excluded “Increase Revenue” and “Protect Revenue” from possible benefits, and excluded the “long life-cycle peak affected by delay” and “short life-cycle peak affected by delay” urgency profiles. The resulting formula is hours saved per week multiplied by the hourly cost rates of the people saving them (taken from the Air Force’s cost-rate table for active-duty personnel). This number is then divided by development duration in weeks to calculate the CoD score for comparing features.

SAFe “relative” Cost of Delay

The Scaled Agile Framework uses a relative estimate of the Cost of Delay. Instead of economic value, teams compare backlog items relative to each other using a modified Fibonacci sequence and combine three proxy components to create a relative cost of delay.

Cost of Delay = User-Business Value + Time Criticality + Risk Reduction/Opportunity Enablement

Although the relative variant avoids the danger of trying to precisely estimate an economic value for future work, it suffers from Mathiness by adding incomparable items such as time and risk, and effectively giving all three components equal weight in the addition (for example, an item with very low value and high criticality scores the same as an item of very high value and low criticality).

Qualitative Cost of Delay

Arnold criticizes the additive formula, saying that Cost of Delay combines value and urgency, and “if either of these two parameters are zero then the Cost of Delay is zero”. Instead of adding three parameters, Arnold argues that value and urgency are not interchangeable, and suggests combining value and urgency in a three-by-three decision matrix, similar in form to the Eisenhower Matrix.

Arnold's qualitative Cost of Delay matrix, from Qualitative Cost of Delay. Value bands are Killer, Bonus and Meh; urgency bands are ASAP, Soon and Whenever.
Value \ UrgencyWheneverSoonASAP
KillerMediumHighVery High
BonusLowMediumHigh
MehVery LowLowMedium

Arnold suggests that the qualitative form “would lack many of the benefits of a quantitative approach”, so it is best treated as a way to start using the method.

The problem with naive economic models

All the calculation formulas simplify the economics of a product to a few estimates, so their results should not be taken as precise numbers. Goljan’s thesis admits that its proof of concept only “illustrates point estimates for many of the time savings”, and suggests that more advanced models should incorporate risk and uncertainty. None of the proposed calculation formulas has any measure of potential variance or error, or present confidence intervals.

Most practical calculations also assume the flat rate urgency profile. Arnold notes that “In most established organisations this is the most common urgency profile you will find”, and Goljan’s Air Force test used it for every feature. It is the easiest profile to compute, so it ends up being the default, but choosing the wrong profile increases errors.

Simple models also leave out costs that are hard to see. Merle Crawford’s critique of Reinertsen’s 1983 delay model pointed out that it was built with “one particular accounting”, and that when calculating the cost of shipping on time, “researchers excluded the costs” sunk in other initiatives that had to be dropped or delayed while the main project was rushed to market.

Origins of Cost of Delay

Martin Greenberger chose the term as the common measure for priority decisions in the MAC-TR-22 technical report in November 1965, during the early days of modeling process schedulers for computer systems. Greenberger notes that he had first used the measure in a 1960 study of message priority in a naval communication system.

We choose an inverse measure of performance: the cost of delay. Depending on the context, it may be thought of as a disutility, penalty cost, loss of goodwill, opportunity cost, postponement of revenue, customer dissatisfaction, storage cost, poorness of service, or some equivalent.

– Martin Greenberger, The Priority Problem

In the same paper, Greenberger presents a precursor to urgency profiles, with cost/rate curves (“These curves plot rate-of-accrual of cost versus time of waiting.”). Greenberger’s model proposes that “Each message (or customer) type was considered to have a separate cost rate curve”, and gives examples of messages with linear (“cost accruing at a constant rate”), quadratic (“the longer a message waits, the greater its marginal cost”) and exponentially decaying (“quick transmission is important, but timeliness decays exponentially”, such as a weather forecast).

Mathematically, the cost of delay for a specific message is the area under its curve: “the integral of the curve over the duration of wait equals the total cost attributable to the message”. Greenberger then defines the fitness formula for system prioritization as minimizing the total cost accrued by all customers:

Assignment of cost rate curves to customers makes the conflicting objectives commensurable by reducing them to a common measure of system performance. The importance of a customer is reflected by the height of his curve. His aversion or intolerance to delay is represented by the shape of his curve. Minimizing the total cost accrued by all customers becomes the single system objective.

– Martin Greenberger, The Priority Problem

Greenberger’s curves provide two factors (height is importance, shape is the intolerance to delay) which directly correspond to the dimensions that Arnold later re-introduces (value and urgency). Many modern prioritization techniques based on Cost of Delay have direct equivalents in the sequencing rules presented by Greenberger.

S. R. Clark and T. A. Rourke took the mathematical models and ran them through simulations in 1970, modeling three priority classes with different cost curves. They concluded that adding the cost of delay to the system cost per job creates an optimal configuration of a computer system:

When the mean cost of delay per Job is added to the mean system cost per Job well formed minima appear indicating an optimum CPU utility, store size and CPU speed for the balance of these two costs.

– Clark and Rourke, A Simulation Study of Cost of Delays in Computer Systems

The concept of Cost of Delay was applied to product development by Donald Reinertsen, who published the idea in the 1983 article “Whodunit? The Search for the New-Product Killers” (Electronic Business in July 1983). There are no electronic copies of that article available, but there is substantial secondary literature referring to it, including Reinertsen’s later books.

With iterative development becoming mainstream in the early 2000s, and the introduction of Lean methods such as work-in-progress limits and the application of queueing systems to work item planning, the concept of cost of delay became more popular. The book The Principles of Product Development Flow in 2009 made cost of delay one of the central concepts for flow management.

If you only quantify one thing, quantify the cost of delay

– Donald Reinertsen, The Principles of Product Development Flow

David Anderson’s work on Kanban leans heavily on the concept of cost of delay and lists “Work scheduled by (opportunity) Cost of Delay” as the third of nine key emergent behaviours to expect when companies adopt Kanban. Anderson ties the concept of classes of service in Kanban directly to the cost of delay. Fixed-delivery-date class items tend to have a cost of delay that starts in the future, standard class items have an immediate cost of delay (“If we had this function today, benefit would be derived tomorrow!”) and intangible items have “no cost of delay within the timeframe that it might take to deliver the item”.

Applicability and limitations

The importance of cost of delay for optimal prioritization has strong mathematical backing in queueing theory and operating system development, where both the effort and the cost of delay can be precisely described. Whether the same benefits apply to product development is less certain. The academic research on this topic is thin, and mostly about products outside IT, but there are some interesting conclusions to consider.

Delay might not matter as much for many classes of features or products

The basic premise of the cost of delay model is that delays have a critical financial impact. Robert Cooper and Elko Kleinschmidt factor-analyzed seven performance measures across 103 major new industrial products in the chemical industry, and found that timeliness was loosely connected to financial performance, not as close as commonly claimed:

The conclusion is that, although timeliness does have some positive connection with financial performance of a new product project, its impact is not nearly as strong as one might have expected, especially given all the “hype.” Timeliness emerges as an independent or stand-alone performance dimension (in the factor analysis). Further, the correlation matrix reveals only a handful of significant correlations between the measures of time (and the timeliness construct) and the various financial performance measures, but certainly far less than the direct or almost one-to-one links the “hype” seems to imply (note that the correlation of 0.42 found between timeliness and profitability explains only 16% of the variation in this possible casual [sic] relationship; that is, 84% is unexplained).

– Robert Cooper and Elko Kleinschmidt, Determinants of Timeliness in Product Development

In a follow-up paper, Cooper argued that the most important factor of profitability is developing a unique and superior product with real value for the customer, and that such products involve no time penalty. Cohen, Eliashberg and Ho conclude that “faster is not necessarily better” if the market potential for a new product is large and the existing product being replaced has a high margin, and that when a firm faces an intermediate level of rivalry, it is better to take time to develop a superior product.

Merle Crawford argued that lateness causing significant financial penalties applies only to “high-growth markets and short product life-cycles”, and that in the same model, in slower-growth markets, delays matter much less than product cost overruns.

In Managing the Design Factory Reinertsen argues that only queues on the critical path of a program actually cost anything, and that a queue outside of the critical path is “free, and even valuable” because it helps load a resource and keep it efficient, so cost-of-delay calculations might not be valuable for items outside the critical path.

Delays that matter may not be economic

Even when delays matter for a specific product class, the delays that matter may not be economic.

Late-stage products often focus on the business part of the Value Exchange Loop, so defining savings or revenue metrics might be easier than for early-stage products which often focus on value delivery to customers.

Products in the “Stickiness” phase of the Five Stages of Growth are better off optimizing customer retention and frequency of usage than focusing on economic value. Cost of delay focused on economic parameters might lead to the wrong schedule for such phases.

As a workaround in contexts like that, it might be worth using the Five Stages of Growth model to decide which metrics are actually important to optimize and which to control, and creating a different model of value similar to the Overall Evaluation Criteria, then applying the cost-of-delay analysis to it.

Cost of delay requires inputs that may not be obtainable or valid

Daniel Vacanti and Prateek Singh simulated CD3 performance against alternatives, and concluded that if both the value and duration estimates match actual values, and nothing happens during the execution that changes the economics, CD3 delivers about 68% more value than random ordering. With value and duration re-assessed after delivery, prioritizing by shortest duration produced 18% more value than CD3. Once items above five weeks were also split into smaller pieces, shortest-duration-first was 51% more effective than CD3.

Vacanti concludes that “the weakness of CD3 is not one of stochastic inputs”, but “it is one of timing”. Teams prioritize before work starts, when they have the least amount of information about the actual value and expected timing, and the most uncertainty. Unfortunately, most real-world product development suffers from inaccurate estimates and changing business opportunities.

Vincent Mabert, John Muth and Roger Schmenner found across six development projects that marketing was often viewed as reluctant “to provide estimates of potential volume” for products at given prices, and that only one of the six firms had a satisfactory marketing effort. Michael Riesener and colleagues rate expected profit, urgency and sales forecast as the hardest of their twenty parameters to obtain. Even when the estimates are provided, they are often wrong. Ron Kohavi and colleagues, reporting from Microsoft’s experimentation platform, report that only one third of tested ideas improved the metric they were designed to improve.

Reinertsen notes that the accuracy of an economic model is limited by the accuracy of its input data, and that there is no input “more likely to be in error than the sales forecast”, suggesting that sales volumes can in some cases be off by a factor of ten.

History and dependencies are ignored

There are two more contextual issues with prioritization based on the cost of delay. The first is that previous investment (sunk cost) is not taken into account. The second is that dependencies or interactions between items are ignored. Greenberger’s paper also deals with those issues in theoretical queueing performance, and defends the usage for the linear cost-of-delay class only.

Past history is not relevant to the priority decision in the linear case; nor is there interaction among units. Unfortunately, these simplifications do not apply to the general nonlinear case.

– Martin Greenberger, The Priority Problem

For more complex classes of ideas, with dependencies, focusing on cost of delay might be wrong.

Reinertsen and colleagues warn that cost of delay analysis produces tactical-level decisions, and that “these tactical decision rules should not be followed blindly”, because other factors including political influences might apply, and that budget constraints can sometimes force a different decision from one based on cost-of-delay analysis.

Using Cost of Delay in practice

Cost of Delay is best used for large order-of-magnitude comparisons, to help improve decision making. It should not be used as the only method for prioritization, but as a supplemental tool.

Reinertsen and Smith recommended keeping models simple, because the true benefit of modeling “comes from its ability to influence decisions”, not from the accuracy of the calculations. It’s best to treat cost-of-delay estimates as a starting point to expose and argue about assumptions, not as a final measurement of value.

Because the estimates can vary widely, it might be useful to discount the inputs based on the confidence of the method used to obtain them. Data from a completed production prototype matters a lot more than someone’s PowerPoint slide, even though they are both in the same unit. Multipliers such as the Confidence Meter score or scales from the Levels of Evidence can be good counterparts to rough estimates, and methods from Impact Estimation Tables can potentially be used where a more systematic approach is needed.

Instead of focusing on more clever sequencing, reducing the queue size tends to provide better results. Reinertsen suggests that sequencing matters most when queues are large, and that “by operating with small queues, we minimize the payoff of sequencing and, thus, the need to do it”. Vacanti arrives at a similar conclusion, that sizing work items “has a more powerful impact on value delivered than changing prioritization schemes”, and recommends limiting work in progress.

Get notified about new articles & major updates
Share