Uncovering the $126 Million Software Knowledge Gap. Is Your Organization Missing Out?
September 20, 2021

Michael Stahnke
CircleCI

Technology executives are often hard-pressed to explain how investing in developer headcount and tooling results in returns for their bottom line. As a business leader, it can be difficult to measure the true value of hiring more developers or investing in software delivery tools.

In the retail industry, for example, ROI can be measured by sales per square foot. Retail execs can easily understand their velocity of sales by space for a new versus existing
store, how long it would take to pay off, and their overall sales volume.

But in the world of software, it's far more complicated.

To accurately make this type of ROI assessment, it's essential to track meaningful engineering output metrics. But after surveying more than 2,000 business leaders across the US and UK, a recent report revealed a significant gap in how these metrics are being measured:

Tech leaders are relying on ineffective metrics. While software is a driver of business value, the knowledge gap between leaders and their developer teams is holding back growth.

Lack of innovation comes with a cost. Relying on ineffective software metrics, like lines of code and story points, are costing companies up to $126 million in lost revenue per year.

Companies are on different digital transformation journeys. Despite this knowledge gap, many businesses are in a good position to reach their potential by focusing on and adopting modern DevOps practices.

There's a lot at risk but there's good news too — there are several strategies engineering teams can employ to bridge this gap and ensure organizations are reaching their maximum potential in software delivery. Let's explore how to put them into practice.

Business Success Relies On Software and Software Engineers

97% of leaders believe at least some of the success of their business within the next year relies on the ability of their software teams, with 62% of those saying that it "matters a lot" or is "critical." More than 60% of these leaders ‘strongly agree' that they understand how to measure their teams' performance.

Accurate measurement plays a vital part in aligning the software engineering process and output with business goals. But it doesn't happen overnight. It happens as you start tracking a few metrics, drill deeper, and add more meaningful metrics to your
dashboard over time.

The Software Knowledge Gap

One of the key findings of this study — and one that may surprise business leaders — is the gap between the ability leaders think they have in measuring the output of their software teams, and the way they actually measure that output.

89% of survey respondents believe they have a good understanding of how to measure the performance of their engineering teams. Yet many are still measuring delivery effectiveness through outdated metrics. In total, 40% of respondents said they are measuring by lines of code written, and 37% are using story points.

Based on the best estimates of respondents, improving how software delivery is measured could be worth a mean revenue increase of as much as 40.8%. Using the reported sales of these businesses, that would equate to a staggering $126 million per company per year left on the table.

Through observing day-to-day practices across the software industry, Rachel Stephens, Senior Industry Analyst at Redmonk, came to this conclusion as well. "High-performing software delivery teams can be a key differentiator for business success, but many companies are still grappling with how to define and measure their teams' performance," Stephens said. "Poorly chosen metrics can negatively impact developer productivity and can hold organizations back from success."

Looking Forward: How Businesses Can Reach Their Maximum Potential

So, what can businesses do to ensure they're measuring software delivery output in the most effective way and maximizing earning potential? First, the knowledge gap must be addressed.

A knowledge gap occurs when different parts of an organization stop communicating and collaborating effectively. Developers and engineering managers and their business counterparts should work together to review and revise company and team goals, taking time to check shared definitions and understandings. The same should be done with budgets, tools, and team and manager skills audits.

Defining Good Outcome Measurements

One possible way to measure software delivery outcomes is to align on service level objectives (SLOs) for engineering teams to maintain. SLOs are simple, numerical measures that can bring business context to a particular part of your output, and commonly include things like error rates, uptime, and time to recovery.

Investing in continuous integration and continuous delivery is an important method for improving critical metrics like Duration, Success Rate, Mean Time to Recovery, and Throughput. Teams that commit to the practice of CI/CD have been shown to rank as elite in these four metric categories.

How to Link Business and Technology Goals

1. Get everyone on the same page. Introduce goals, whether from the overall business or within a technology team, so everyone knows what they're working towards. These goals should link back to the company's mission.

2. Link goals to important metrics and explain them. Leaders must be clear about what they need from technologists and what the expected product must achieve. Developers should set their team goals and KPIs in ways the business can understand and support them.

3. Start small. Changes should be incremental. If they work, keep doing them.

4. Don't rely on common sense. Consistently review metrics. If you review goals, KPIs, and results and find a disconnect, bring all the stakeholders together to review and get back on track, without judgment.

5. Look at software delivery as a revenue accelerant. If software is a critical aspect of your organization, it is not a cost center. Innovators need the right tools to grow and improve, and to deliver on the company mission.

Digital Transformation is a Journey

Business leaders need to accurately measure the productivity and business impact of engineering work, but digital transformation in and of itself has no endpoint. It's important that both engineering and business leaders don't exclusively compare themselves to others. Run your own race, compare your teams to a baseline, and improve from there.

Michael Stahnke is VP of Platform at CircleCI
Share this

Industry News

October 21, 2021

Splunk announced the latest enhancements to the Splunk observability portfolio including advanced product innovations for Splunk Application Performance Monitoring (APM), Splunk Real User Monitoring (RUM), Splunk Synthetic Monitoring, Splunk Log Observer, Splunk Infrastructure Monitoring and Splunk IT Service Intelligence.

October 21, 2021

Platform9 announced a number of new enterprise features that greatly eliminate operational complexities in managing multi-cluster and multicloud Kubernetes deployments, available in the new release, Platform9 5.4 Release.

October 21, 2021

Graylog is announcing Graylog Security.

Designed to overcome legacy Security Information & Event Management (SIEM) challenges, Graylog’s scalable, flexible cybersecurity platform makes security analysts’ jobs easier and faster. With SIEM, Anomaly Detection, and User Entity Behavior Analytics (UEBA) capabilities, Graylog’s security solution will provide security teams with even greater confidence, productivity, and expertise to mitigate risks caused by Insider Threats, credential-based attacks, and other cyber threats.

October 21, 2021

Prodly announced a Series A investment of $10 million led by Leta Capital.

October 20, 2021

SonarSource added over 5,000 customers in the last 12 months, reaching the 15,000 commercial customers milestone in record time.

October 20, 2021

Actian announced the general availability of its newly released DataConnect 12 integration platform, demonstrating a continued focus on ease of use for complex data integration and data quality.

October 20, 2021

Salt Security announced new capabilities in its next-generation Salt Security API Protection Platform to secure GraphQL APIs.

October 20, 2021

vFunction announces the availability of the vFunction Application Transformation Engine and the expanded vFunction Modernization Platform, with new, advanced capabilities that enable enterprises to automatically assess, analyze, and manage the full modernization and migration process from start to finish.

October 20, 2021

Mage raised a $6.3 million seed round led by Gradient Ventures.

October 19, 2021

Couchbase announced its Couchbase Capella hosted Database-as-a-Service (DBaaS) offering on Amazon Web Services (AWS).

October 19, 2021

Checkmarx announced the launch of the Checkmarx Application Security Platform to help CISOs, AppSec teams, and developers address the growing and dynamic security challenges they face.

October 19, 2021

Tasktop announced Affinity Modeling for model-based integration in Tasktop Hub, helping Agile and DevOps software delivery teams reduce time to market and develop software faster.

October 19, 2021

Morpheus Data is continuing released version 5.3.3 targeted at enterprises trying to manage a complex mix of VMware, Kubernetes, and Public Cloud services.

October 19, 2021

Okta announced the availability of Okta Workflows as a standalone offering for all customers.

October 18, 2021

Red Hat announced a series of updates in its portfolio of developer tools and programs aimed at delivering greater productivity, security and scale for developers building applications on Red Hat OpenShift.