The 5 Longest Lead Times in Software Delivery - Part 1
March 08, 2017

Mark Levy
Micro Focus

Rapid business change, fueled by software innovation, is transforming how software delivery organizations define, develop, test, and release business applications. For these software organizations to keep their competitive advantage in today's complex and volatile digital marketplace, they must become more agile, adaptive, and integrated into the business. They must get fully on board with the digital transformation of business practices. However, most current software delivery practices can't keep pace with the demands of the business.

Long software delivery cycles are a significant impediment to business technology innovation. Agile development teams have shortened development cycles, but Agile by itself is insufficient as it does not remove the cultural and technical barriers between development and operations. DevOps principles and practices developed in response to this problem, facilitates cooperation and coordination amongst teams to deliver software faster and with better quality.

Every enterprise IT organization currently has software delivery processes in place. This is known as its deployment pipeline. In fact, large enterprises have many deployment pipelines since they typically have hundreds of applications spread across many types of application infrastructures that are usually managed by different, geographically dispersed teams.

The whole goal of scaling "DevOps" for the enterprise is to prioritize and optimize these existing deployment pipelines, seek efficiencies, and remove waste to deliver better business outcomes. Creating new and optimizing existing deployment pipelines in large IT organizations is key to improving their efficiency and effectiveness in delivering software at the speed that the business requires.

So where do you start? Every enterprise IT organization is unique in that it will have different bottlenecks and constraints in its deployment pipelines. I recommend that you prioritize your deployment pipelines and conduct a value stream mapping exercise to identify specific problem areas. Starting and Scaling DevOps in the Enterprise, by Gary Gruver is a great book and provides a good framework for getting started. With that being said, there are some common problem areas that typically produce the longest lead times in your software delivery process.

First, let's make sure we are on the same page with what lead time is. Lead time is one of two metrics commonly used to evaluate performance in an IT value stream or deployment pipeline. The other metric is task time. Whereas the lead time clock starts when the customer request is made and ends when it is fulfilled, the task time clock starts only when you begin work on the customer request; specifically it omits the time in the queue waiting to be processed. Requests are fulfilled for both internal and external customers. So, if the Dev team requests a new test environment, it may take only one day to actually create the environment, but the request might wait in the queue for four days. Therefore, the actual lead time would be five days and the processing time would be one day.

Here are the most common areas that generate the longest lead times:

1. Organization Bottlenecks

DevOps culture strives to break down the organizational silos and transition more to product teams. This is because the current silo'd organizational structure provides headwinds to the objective of short lead times and continuous flow. Organizational silos are artifacts of the industrial era designed specifically for "Batch and Queue" processing which drives up lead times with handoffs from one team or organization to another.

Handoffs drive lead times up by requiring additional communication and usually using different tools. Each handoff is potentially a queue in itself. Resolving ambiguities require additional communication between teams and can result in significant delays, high costs, and failed releases. Unfortunately, only so much can be successfully conveyed with words and diagrams. There is a great deal of tacit knowledge, which is difficult to convey by just writing it down or verbalizing it. You also get work bounce back or work bounced around between teams. Handoffs can become the "Hot Potato" that nobody wants. You need to strive to reduce the number of handoffs by either automating a significant portion of the work or re-organizing into product teams that are not dependent on other teams to deliver value to the customer.

Shared resources across projects and organizational silos is another bottleneck. Lead times can typically exceed 2x greater than task times if waiting on a shared resource while they are completing another task for another project or team. The more you can encapsulate the team to be enabled to continuously work on creating customer value, the faster the flow, the better the quality, and the lower the lead time.

2. Approval Processes

The approval process is another area that can generate long lead times and waste. The approval process within most large enterprises is slow and complex. It is often comprised of a set of manual stovepipe processes that use email and Microsoft office tools to track, manage, and, more often than not, wait on people for approval of a software change. Insufficient data leads to hasty or faulty approvals or bounce backs, further frustrating software delivery teams, reducing quality, and impeding deployments.

Change approval processes originally were developed to mitigate risk and provide oversight to ensure adherence to auditable standards for moving changes into production. Over time this process tends to get bureaucratic as board members struggle to understand the changes the teams are implementing. External change approval boards can increase lead times significantly, with negligible impact on stability.

Peer reviews, along with the DevOps practice of infrastructure as code and automating the deployment pipeline, enable better change control, a more rigorous approval process, and a dramatic improvement in speed. All the software artifacts and automated scripts are kept under version control in one SCM tool. With everything in version control, the SCM system contains all the documented changes and can provide a complete auditable record. Toolchain integration with the change management system will provide visibility of the state of the change, while the approval process is the automated test that validates whether that particular change passes the criteria to be deployed. Releasing into production might need approval from the business, but everything up to that point could be automated with dramatically reduced lead times.

Read The 5 Longest Lead Times in Software Delivery - Part 2, outlining the other three common areas that generate the longest lead times.

Mark Levy is Director of Strategy at Micro Focus

The Latest

September 18, 2018

To celebrate IT Professionals Day 2018 (this year on September 18), the SolarWinds IT Pro Day 2018: A World Powered by Tech Pros survey explores a "Tech PROactive" world where technology professionals have the time, resources, and ability to use their technology prowess to do absolutely anything ...

September 17, 2018

The role of DevOps in capitalizing on the benefits of hybrid cloud has become increasingly important, with developers and IT operations now working together closer than ever to continuously plan, develop, deliver, integrate, test, and deploy new applications and services in the hybrid cloud ...

September 13, 2018

"Our research provides compelling evidence that smart investments in technology, process, and culture drive profit, quality, and customer outcomes that are important for organizations to stay competitive and relevant -- both today and as we look to the future," said Dr. Nicole Forsgren, co-founder and CEO of DevOps Research and Assessment (DORA), referring to the organization's latest report Accelerate: State of DevOps 2018: Strategies for a New Economy ...

September 12, 2018

This next blog examines the security component of step four of the Twelve-Factor methodology — backing services. Here follows some actionable advice from the WhiteHat Security Addendum Checklist, which developers and ops engineers can follow during the SaaS build and operations stages ...

September 10, 2018

When thinking about security automation, a common concern from security teams is that they don't have the coding capabilities needed to create, implement, and maintain it. So, what are teams to do when internal resources are tight and there isn't budget to hire an outside consultant or "unicorn?" ...

September 06, 2018

In evaluating 316 million incidents, it is clear that attacks against the application are growing in volume and sophistication, and as such, continue to be a major threat to business, according to Security Report for Web Applications (Q2 2018) from tCell ...

September 04, 2018

There's a welcome insight in the 2018 Accelerate State of DevOps Report from DORA, because for the first time it calls out database development as a key technical practice which can drive high performance in DevOps ...

August 29, 2018

While everyone is convinced about the benefits of containers, to really know if you're making progress, you need to measure container performance using KPIs.These KPIs should shed light on how a DevOps team is faring in terms of important parameters like speed, quality, availability, and efficiency. Let's look at the specific KPIs to track for each of these broad categories ...

August 27, 2018

Protego Labs recently discovered that 98 percent of functions in serverless applications are at risk, with 16 percent considered "serious" ...

August 23, 2018

After another record year of breaches, The 2018 DevSecOps Community Survey found that 3 in 10 respondents suspected or verified breaches stemming from vulnerabilities in open source components — a 55% increase over 2017, and 121% increase since 2014 ...

Share this