Understanding DevOps Complexity Offers a Way Forward for Enterprises
October 22, 2020

Jon Collins
Gigaom

Sometimes, my work as an analyst offers opportunities to pause, to reflect. At a risk of making this blog all about me, I have completed the report on Value Stream Management (VSM) that I mentioned in a previous blog, and I can now turn my full attention to DevSecOps. More will come after that, each report peppered with real-world experiences from people like you, good reader (and I welcome hearing about your own). But in the interim, between reports, I've had time to reflect on DevOps, and how I can help further the cause of delivering software-based innovation at scale. 

There's a massive irony here. My first job was as a programmer; I later ran tools and infrastructure for development groups; I went on to advise some pretty big organizations on how to develop software, and how to manage data centers, servers, storage, networking, security and all that. I've written books about it, for heaven's sake — so how come, when I write about it all, I can sometimes feel out of my depth? 

It's an important question to answer, because my own experience of imposter syndrome reflects many of the enterprises I speak to. Some have set up DevOps-type groups as independent bubbles within the organization, leaving those outside feeling very much away from the cutting edge. This phenomenon is not new: back in the Nineties I was working at the forefront of what we might call "the agile boom" — a time in which older, ponderous approaches to software production, with two-year lead times and no guarantees of success — were being reconsidered in the light of the internet. 

The idea was and remains simple: take too long to deliver something, and the world will have moved on. As a Dynamic Systems Development Methodology consultant, my job was to help the cool kids do things fast but do things right. Over time I learned one factor that stood out above all others, that could make or break an agile development practice: complexity. It was in this period that I learned the power of the Pareto principle, or in layperson's terms, "let's separate out the things we absolutely need, from the nice-to-haves, that can come later."

Complexity kills innovation, there, I've said it. Back in the days of Waterfall methodologies, processes would be bogged down in over-specified requirements (so-called analysis paralysis) and exhausting test regimes. No wonder software development gurus looked to return to the source (sic) and adopt the JFDI approach that remains prevalent today. 

Trouble is, complexity never went away: it just moved along the pipeline. In a recent online panel, I likened developers to the Sorcerer's Apprentice — it's one thing to be able to make a broom at will, but how are you going to manage them all? It's as good an analogy as any for how simple it is to create a software-based artifact, and what issues this creates. VSM hasn't come into existence on a whim: it's emerged in response to the challenges caused by its absence. Same with DevSecOps, for that matter. 

Below the surface lies as simple truth, that short-termist approaches miss out on fundamental elements such as planning, setting strategy and so on. And the spin-off result of doing lots of things very fast, is to generate a lot of complexity which then needs to be managed. Even our most darling of cloud-native mega-businesses are now struggling with the complexity of what they have created — good for them for ignoring it, while they established their brand, but you can only put good old-fashioned configuration management off for so long. 

Do I think that software should be delivered more slowly, or favor a return to old-fashioned methodologies? Absolutely not. But it does explain why we're seeing what we might call "a wave of governance" start to envelop the world of software development, as short-term perspectives are reconsidered in favor of getting things right first time. There's buzz-phrases for this, of course, such as "shift-left," which is about thinking about quality and security earlier in the process. 

The challenge of complexity also offers a way forward, for enterprise organizations feeling out of their depth: it's a problem, for sure, but it is one they know how to address. Step aside, imposter syndrome: it's time to bring some of those older wisdoms, such as configuration management, requirements management and risk management, to bear. While enterprises can't suddenly become carefree startups, they can recognize that such enterprise-y practices are actually a good thing, which can be woven into new ways of delivering software. 

This won't be easy, but it is necessary, and it will be supported by tools vendors as they, too, mature. Over coming years, I expect to see simplification and consolidation across the tools and platform space, enabling more policy-driven approaches, better guardrails and improved automation. So that developers can get on and do the thing with minimal encumbrance, even as managers and the business as a whole feels the co-ordination benefit. 

The bottom line is that for DevOps to scale, governance principles need to be baked in. I hesitate before suggesting that the core CALMS notions need to add a G — perhaps the G is silent, but it is no less important. 

Jon Collins is VP of Research for Gigaom
Share this

Industry News

April 14, 2021

SmartBear has integrated TestComplete, its UI test automation tool, with BitBar, its native mobile device cloud.

April 14, 2021

Elastic announced an expanded strategic partnership with Confluent to deliver the best integrated product experience to the Apache Kafka and Elasticsearch community.

April 14, 2021

Threat Stack announced its ability to support AWS Graviton2-based instances through the Threat Stack Cloud Security Platform.

April 13, 2021

Broadcom and Google Cloud announced a strategic collaboration to accelerate innovation and strengthen cloud services integration within the core software franchises of Broadcom.

April 13, 2021

Nylas announced the launch of Components, JavaScript UI/UX solutions that allow developers to bring productivity features to market faster without needing to design front-end elements from scratch.

April 13, 2021

Perforce Software announces its new version control desktop client — Helix Sync — enabling non-coders such as artists and designers to version digital assets, with a simple drag-and-drop UI.

April 12, 2021

ShiftLeft introduced ShiftLeft CORE, a unified code security platform.

April 12, 2021

GrammaTech announced a new version of its CodeSonar SAST (static application security testing) product that helps developers build safer and more secure code without disrupting workflows.

April 12, 2021

Panaya announced a strategic partnership with Being Guided, a Salesforce Consulting Partner, specializing in the CRM and Salesforce ecosystem, to bring Panaya's ForeSight solution to a wider audience.

April 08, 2021

Palo Alto Networks announced the second generation of Checkov, the static analysis tool for infrastructure as code (IaC).

April 08, 2021

Postman now allows any team with up to three members to collaborate in Postman with unlimited shared workspaces and unlimited shared requests at no cost.

April 08, 2021

Taos, an IBM company, has announced 24x5 managed service availability.

April 07, 2021

VMware unveiled expanded cloud workload protection capabilities to deliver security for containers and Kubernetes.

April 07, 2021

Catapult CX is launching the DevOps Institute’s (DOI) Assessment of DevOps Capabilities (ADOC).

April 07, 2021

Equinix announced that Tinkerbell, an all-in-one open source bare metal provisioning platform, has added significant new features since joining the Cloud Native Computing Foundation (CNCF) Sandbox program.