Crash-Testing Software Like Cars: An Argument for Shifting DevOps Testing Right
October 15, 2020

James Isaacs
Cyara

The concept of pre-testing products prior to launch is an exercise in simple logic — make sure that the product you're offering performs as it should before you sell it. Automobile companies are a great example, as they test various car parts for defects before production and throughout the manufacturing process.

In a similar vein, the practice of "shift-left testing" has gained a foothold among DevOps teams that seek to leverage the benefits of early-stage software testing. However, automobile companies don't stop testing their products after the build phase is complete — ongoing safety tests that measure how vehicles perform and degrade over time and in different environments is a major facet of vehicular safety and innovation research.

So, why is it that the same concept of continuous testing post-production and post-deployment, a strategy of "shift right testing," hasn't received the same recognition from DevOps community?

Pre-Production Testing to Predict Crashes & Failures

Performing software tests regularly throughout the development cycle — a strategy known as "continuous testing" — is a widely practiced tactic employed by companies that seek to optimize their DevOps process. Continuous testing informs DevOps teams with a steady stream of data surrounding application errors that can be used to inform their software development and fix any potential risks or bugs that could negatively affect the software.

Many DevOps teams have already recognized the many advantages of shift-left testing, which is a key aspect of continuous testing. Shifting left by testing software earlier in the production cycle enables DevOps teams to identify and rectify any design issues as they present themselves and before they can cause any future errors in the software application. Software tests can be shifted as far left as the initial product design and conception, enabling DevOps teams to catch bugs as early as possible in the development cycle.

Further, shift-left testing prevents the need to perform a barrage of tests on a litany of software features post-development. Performing software tests later in the development cycle can also leave DevOps teams working against tight deadlines to fix unexpected issues, which can lead to haphazard testing that misses software bugs.

In addition, shift-left testing significantly lowers the cost of bug fixes compared to testing at the end of a development cycle.

Shifting left also enables developers to avoid "context switching," a back-and-forth development pattern of switching between features as they are tested — allowing developers to focus their attention entirely on the specific aspect of the software application to which they are currently assigned. Otherwise, context switching can inadvertently cause developers to try and cut corners to avoid shifting deadlines or milestones and maintain project trajectory — causing more harm than good in the long run if the end-user is subjected to sub-par experience, service, or capabilities.

The Next Logical Step — Shifting Right

In keeping with the comparison of DevOps teams to automobile manufacturers, the benefits of shift-left testing are fairly similar to pre-production testing of vehicle parts — identify any issues early on to avoid costly re-designs later on in the product's lifecycle. It's a fairly logical conclusion, even if you only sit and think about the concept for a few minutes.

In fact, shifting testing left was a crucial area of focus for the majority of companies included in the recent Forrester Wave for Continuous Functional Test Automation (CFTA). However, the strategy of shifting testing right should also be a crucial area of focus for software developers. But what are the benefits to software developers when it comes to testing post-deployment?

Shifting right is a strategy of testing software later in the development cycle — even once the product or software is launched and in production. Ensuring that the final software product fully functions in real-world environments is critical to the long-term success of any software or technology solution. Just because a software application or system fully functions in the development environment doesn't guarantee functionality in the real world, as some software issues will only manifest once the application interacts with end-users.

Moreover, oftentimes, companies' test environments and production environments might not be exactly the same. If applications are deployed in different cloud or on-premise environments than those in which they are tested, it's impossible to predict how those applications will perform.

Additionally, it's an inevitable fact of life that stuff breaks. Even the most tested, tried and true products experience errors and breakdowns throughout their lifetime. Unless organizations are continuously testing in production phases, they will not be aware of any application issues until their customers report them — which is too often too late to deliver an exceptional customer experience.

To truly implement a strategy of continuous testing, it's only logical to assume that tests should be conducted throughout the software lifecycle — from design all the way through production. This includes testing as early as possible, and testing as late as possible to achieve a complete stream of testing data that identifies defects and informs the entire software development cycle.

The Emergence of DevTestOps

As software developers progress towards a strategy of truly continuous testing and embrace shift-left and shift-right testing, it will be crucial for testing and DevOps teams to collaborate cohesively. This collaborative approach is birthing a new technology and software development discipline: DevTestOps.

DevTestOps is gaining traction as a discipline for three major reasons that have become apparent to DevOps teams as they embrace continuous testing, shift-left and shift-right. First, unless developers engage live software systems by testing in production, it's basically impossible to accurately gauge the end-user experience. For example, different organizations leverage different providers for their cloud-hosted services for their test environment and for their production environment. Can you be 100% positive that your application works the same in any cloud environment without testing it?

Second, things break. It's inevitable that product features eventually break, become corrupted, or are interrupted by new features. Even if an application passes each pre-deployment test with flying colors, it will eventually experience an error somewhere down the road. Organizations can remain quality-focused, and prove their vigilance to their customers, by enabling their testing and DevOps teams to continue working together to improve the product post-deployment.

Finally, achieving 100% reliability for DevOps practices is simply not possible. Through DevTestOps, DevOps teams can gain key insights into how they can improve future software products in the development phase.

Companies and DevOps teams that hope to perfect software development processes are embracing a continuous testing strategy, but they should be careful not to turn a blind eye to the right side of the software development lifecycle by ignoring the importance of shifting testing right. While shifting left and shifting right are two different testing strategies, they should be viewed as complimentary, not mutually exclusive. DevOps and testing teams must work together to develop the best products possible while simultaneously eliminating the need for back-and-forth development and testing patterns.

James Isaacs is President of Cyara
Share this

Industry News

May 25, 2022

JFrog introduced Project Pyrsia, an open-source software community initiative that utilizes blockchain technology to secure software packages (A.K.A Binaries) from vulnerabilities and malicious code.

May 25, 2022

Kasm Technologies, in partnership with Docker, has developed Kasm Workspaces as a Containerized Desktop Infrastructure platform for streaming remote workspaces directly to your web browser.

May 25, 2022

Cascadeo announced the integration of Amazon DevOps Guru with cascadeo.io, Cascadeo’s cloud monitoring and management platform that provides users with a single view of multi-cloud or hybrid infrastructure environments.

May 24, 2022

Oracle announced the availability of Java 18, the latest version of the programming language and development platform.

May 24, 2022

Docker announced the acquisition of Tilt, makers of a development environment as code for teams on Kubernetes.

May 24, 2022

F5 announced the release of F5 NGINX for Microsoft Azure, an Azure-native service offering developed in partnership with Microsoft, that helps customers deliver modern applications on Azure with just a few clicks.

May 24, 2022

Pegasystems announced a strategic partnership with Google Cloud that will help enable joint clients to accelerate their digital transformations with Pega’s low-code enterprise software on Google Cloud’s highly scalable cloud services.

May 23, 2022

Sauce Labs announced the release of contract testing with mocking on the Sauce Labs API Testing Platform.

May 23, 2022

Pure Storage announced a series of updates to its Portworx portfolio.

May 23, 2022

StackHawk has secured $20.7 million in capital.

May 19, 2022

Jellyfish announced the launch of Jellyfish Benchmarks, a way to add context around engineering metrics and performance by introducing a method for comparison.

May 19, 2022

Solo.io announced the addition and integration of Cilium networking into its Gloo Mesh platform, providing a complete application-networking solution for companies’ cloud-native digital transformation efforts.

May 19, 2022

Aqua Security announced multiple updates to Aqua Trivy, making it a unified scanner for cloud native security.

May 18, 2022

Red Hat unveiled updates across its portfolio of developer tools designed to help organizations build and deliver applications faster and more consistently across Kubernetes-based hybrid and multicloud environments.

May 18, 2022

Armory announced public early access to their new Continuous Deployment-as-a-Service product.