Understanding DevOps Complexity Offers a Way Forward for Enterprises
October 22, 2020

Jon Collins
Gigaom

Sometimes, my work as an analyst offers opportunities to pause, to reflect. At a risk of making this blog all about me, I have completed the report on Value Stream Management (VSM) that I mentioned in a previous blog, and I can now turn my full attention to DevSecOps. More will come after that, each report peppered with real-world experiences from people like you, good reader (and I welcome hearing about your own). But in the interim, between reports, I've had time to reflect on DevOps, and how I can help further the cause of delivering software-based innovation at scale. 

There's a massive irony here. My first job was as a programmer; I later ran tools and infrastructure for development groups; I went on to advise some pretty big organizations on how to develop software, and how to manage data centers, servers, storage, networking, security and all that. I've written books about it, for heaven's sake — so how come, when I write about it all, I can sometimes feel out of my depth? 

It's an important question to answer, because my own experience of imposter syndrome reflects many of the enterprises I speak to. Some have set up DevOps-type groups as independent bubbles within the organization, leaving those outside feeling very much away from the cutting edge. This phenomenon is not new: back in the Nineties I was working at the forefront of what we might call "the agile boom" — a time in which older, ponderous approaches to software production, with two-year lead times and no guarantees of success — were being reconsidered in the light of the internet. 

The idea was and remains simple: take too long to deliver something, and the world will have moved on. As a Dynamic Systems Development Methodology consultant, my job was to help the cool kids do things fast but do things right. Over time I learned one factor that stood out above all others, that could make or break an agile development practice: complexity. It was in this period that I learned the power of the Pareto principle, or in layperson's terms, "let's separate out the things we absolutely need, from the nice-to-haves, that can come later."

Complexity kills innovation, there, I've said it. Back in the days of Waterfall methodologies, processes would be bogged down in over-specified requirements (so-called analysis paralysis) and exhausting test regimes. No wonder software development gurus looked to return to the source (sic) and adopt the JFDI approach that remains prevalent today. 

Trouble is, complexity never went away: it just moved along the pipeline. In a recent online panel, I likened developers to the Sorcerer's Apprentice — it's one thing to be able to make a broom at will, but how are you going to manage them all? It's as good an analogy as any for how simple it is to create a software-based artifact, and what issues this creates. VSM hasn't come into existence on a whim: it's emerged in response to the challenges caused by its absence. Same with DevSecOps, for that matter. 

Below the surface lies as simple truth, that short-termist approaches miss out on fundamental elements such as planning, setting strategy and so on. And the spin-off result of doing lots of things very fast, is to generate a lot of complexity which then needs to be managed. Even our most darling of cloud-native mega-businesses are now struggling with the complexity of what they have created — good for them for ignoring it, while they established their brand, but you can only put good old-fashioned configuration management off for so long. 

Do I think that software should be delivered more slowly, or favor a return to old-fashioned methodologies? Absolutely not. But it does explain why we're seeing what we might call "a wave of governance" start to envelop the world of software development, as short-term perspectives are reconsidered in favor of getting things right first time. There's buzz-phrases for this, of course, such as "shift-left," which is about thinking about quality and security earlier in the process. 

The challenge of complexity also offers a way forward, for enterprise organizations feeling out of their depth: it's a problem, for sure, but it is one they know how to address. Step aside, imposter syndrome: it's time to bring some of those older wisdoms, such as configuration management, requirements management and risk management, to bear. While enterprises can't suddenly become carefree startups, they can recognize that such enterprise-y practices are actually a good thing, which can be woven into new ways of delivering software. 

This won't be easy, but it is necessary, and it will be supported by tools vendors as they, too, mature. Over coming years, I expect to see simplification and consolidation across the tools and platform space, enabling more policy-driven approaches, better guardrails and improved automation. So that developers can get on and do the thing with minimal encumbrance, even as managers and the business as a whole feels the co-ordination benefit. 

The bottom line is that for DevOps to scale, governance principles need to be baked in. I hesitate before suggesting that the core CALMS notions need to add a G — perhaps the G is silent, but it is no less important. 

Jon Collins is VP of Research for Gigaom
Share this

Industry News

January 27, 2021

Indigo.Design announced the public preview of Indigo.Design App Builder.

January 27, 2021

ARMO announced its launch out of stealth having secured $4.5 million in seed funding from Pitango First.

January 27, 2021

CloudSphere announced the appointment of Jane Gilson as the company’s CEO successor to Patrick McNally.

January 26, 2021

JFrog announced an agreement with Docker.

January 26, 2021

SUSE released Longhorn 1.1.

January 26, 2021

HAProxy released HAProxy Kubernetes Ingress Controller 1.5.

January 25, 2021

Progress announced the new release of Progress Kendo UI, a complete collection of JavaScript UI components.

January 25, 2021

CloudNatix announced the close of a $4.5M Seed round financing led by DNX Ventures, with the participation from a new investor Cota Capital and existing investors: Incubate Fund, Vela Partners and 468 Capital.

January 25, 2021

Quali announced $54 million in new funding, co-led by Greenfield Partners and JVP.

January 21, 2021

Platform9 released Platform9 Release 5.0, with a number of new features to provide operational efficiencies for its freedom, growth, and enterprise managed Kubernetes products.

January 21, 2021

Infragistics announced the release of Infragistics Ultimate 20.2, a complete UX and UI solution for  design and development teams  which is fully compatible with .NET 5, Microsoft’s latest  release of .NET development platform.

January 21, 2021

Couchbase Cloud is now available on Microsoft Azure.

January 20, 2021

Hitachi Vantara announced the availability of Hitachi Kubernetes Service, enabling customers to consistently and securely deploy, manage, monitor, and govern Kubernetes clusters across major cloud providers and on premises.

January 20, 2021

Internal announced the launch of an enterprise-ready app development platform for internal tools.

January 20, 2021

StackPulse announced a $20 million Series A led by GGV Capital.