Understanding DevOps Complexity Offers a Way Forward for Enterprises
October 22, 2020

Jon Collins
Gigaom

Sometimes, my work as an analyst offers opportunities to pause, to reflect. At a risk of making this blog all about me, I have completed the report on Value Stream Management (VSM) that I mentioned in a previous blog, and I can now turn my full attention to DevSecOps. More will come after that, each report peppered with real-world experiences from people like you, good reader (and I welcome hearing about your own). But in the interim, between reports, I've had time to reflect on DevOps, and how I can help further the cause of delivering software-based innovation at scale. 

There's a massive irony here. My first job was as a programmer; I later ran tools and infrastructure for development groups; I went on to advise some pretty big organizations on how to develop software, and how to manage data centers, servers, storage, networking, security and all that. I've written books about it, for heaven's sake — so how come, when I write about it all, I can sometimes feel out of my depth? 

It's an important question to answer, because my own experience of imposter syndrome reflects many of the enterprises I speak to. Some have set up DevOps-type groups as independent bubbles within the organization, leaving those outside feeling very much away from the cutting edge. This phenomenon is not new: back in the Nineties I was working at the forefront of what we might call "the agile boom" — a time in which older, ponderous approaches to software production, with two-year lead times and no guarantees of success — were being reconsidered in the light of the internet. 

The idea was and remains simple: take too long to deliver something, and the world will have moved on. As a Dynamic Systems Development Methodology consultant, my job was to help the cool kids do things fast but do things right. Over time I learned one factor that stood out above all others, that could make or break an agile development practice: complexity. It was in this period that I learned the power of the Pareto principle, or in layperson's terms, "let's separate out the things we absolutely need, from the nice-to-haves, that can come later."

Complexity kills innovation, there, I've said it. Back in the days of Waterfall methodologies, processes would be bogged down in over-specified requirements (so-called analysis paralysis) and exhausting test regimes. No wonder software development gurus looked to return to the source (sic) and adopt the JFDI approach that remains prevalent today. 

Trouble is, complexity never went away: it just moved along the pipeline. In a recent online panel, I likened developers to the Sorcerer's Apprentice — it's one thing to be able to make a broom at will, but how are you going to manage them all? It's as good an analogy as any for how simple it is to create a software-based artifact, and what issues this creates. VSM hasn't come into existence on a whim: it's emerged in response to the challenges caused by its absence. Same with DevSecOps, for that matter. 

Below the surface lies as simple truth, that short-termist approaches miss out on fundamental elements such as planning, setting strategy and so on. And the spin-off result of doing lots of things very fast, is to generate a lot of complexity which then needs to be managed. Even our most darling of cloud-native mega-businesses are now struggling with the complexity of what they have created — good for them for ignoring it, while they established their brand, but you can only put good old-fashioned configuration management off for so long. 

Do I think that software should be delivered more slowly, or favor a return to old-fashioned methodologies? Absolutely not. But it does explain why we're seeing what we might call "a wave of governance" start to envelop the world of software development, as short-term perspectives are reconsidered in favor of getting things right first time. There's buzz-phrases for this, of course, such as "shift-left," which is about thinking about quality and security earlier in the process. 

The challenge of complexity also offers a way forward, for enterprise organizations feeling out of their depth: it's a problem, for sure, but it is one they know how to address. Step aside, imposter syndrome: it's time to bring some of those older wisdoms, such as configuration management, requirements management and risk management, to bear. While enterprises can't suddenly become carefree startups, they can recognize that such enterprise-y practices are actually a good thing, which can be woven into new ways of delivering software. 

This won't be easy, but it is necessary, and it will be supported by tools vendors as they, too, mature. Over coming years, I expect to see simplification and consolidation across the tools and platform space, enabling more policy-driven approaches, better guardrails and improved automation. So that developers can get on and do the thing with minimal encumbrance, even as managers and the business as a whole feels the co-ordination benefit. 

The bottom line is that for DevOps to scale, governance principles need to be baked in. I hesitate before suggesting that the core CALMS notions need to add a G — perhaps the G is silent, but it is no less important. 

Jon Collins is VP of Research for Gigaom
Share this

Industry News

May 25, 2022

JFrog introduced Project Pyrsia, an open-source software community initiative that utilizes blockchain technology to secure software packages (A.K.A Binaries) from vulnerabilities and malicious code.

May 25, 2022

Kasm Technologies, in partnership with Docker, has developed Kasm Workspaces as a Containerized Desktop Infrastructure platform for streaming remote workspaces directly to your web browser.

May 25, 2022

Cascadeo announced the integration of Amazon DevOps Guru with cascadeo.io, Cascadeo’s cloud monitoring and management platform that provides users with a single view of multi-cloud or hybrid infrastructure environments.

May 24, 2022

Oracle announced the availability of Java 18, the latest version of the programming language and development platform.

May 24, 2022

Docker announced the acquisition of Tilt, makers of a development environment as code for teams on Kubernetes.

May 24, 2022

F5 announced the release of F5 NGINX for Microsoft Azure, an Azure-native service offering developed in partnership with Microsoft, that helps customers deliver modern applications on Azure with just a few clicks.

May 24, 2022

Pegasystems announced a strategic partnership with Google Cloud that will help enable joint clients to accelerate their digital transformations with Pega’s low-code enterprise software on Google Cloud’s highly scalable cloud services.

May 23, 2022

Sauce Labs announced the release of contract testing with mocking on the Sauce Labs API Testing Platform.

May 23, 2022

Pure Storage announced a series of updates to its Portworx portfolio.

May 23, 2022

StackHawk has secured $20.7 million in capital.

May 19, 2022

Jellyfish announced the launch of Jellyfish Benchmarks, a way to add context around engineering metrics and performance by introducing a method for comparison.

May 19, 2022

Solo.io announced the addition and integration of Cilium networking into its Gloo Mesh platform, providing a complete application-networking solution for companies’ cloud-native digital transformation efforts.

May 19, 2022

Aqua Security announced multiple updates to Aqua Trivy, making it a unified scanner for cloud native security.

May 18, 2022

Red Hat unveiled updates across its portfolio of developer tools designed to help organizations build and deliver applications faster and more consistently across Kubernetes-based hybrid and multicloud environments.

May 18, 2022

Armory announced public early access to their new Continuous Deployment-as-a-Service product.