Crash-Testing Software Like Cars: An Argument for Shifting DevOps Testing Right
October 15, 2020

James Isaacs
Cyara

The concept of pre-testing products prior to launch is an exercise in simple logic — make sure that the product you're offering performs as it should before you sell it. Automobile companies are a great example, as they test various car parts for defects before production and throughout the manufacturing process.

In a similar vein, the practice of "shift-left testing" has gained a foothold among DevOps teams that seek to leverage the benefits of early-stage software testing. However, automobile companies don't stop testing their products after the build phase is complete — ongoing safety tests that measure how vehicles perform and degrade over time and in different environments is a major facet of vehicular safety and innovation research.

So, why is it that the same concept of continuous testing post-production and post-deployment, a strategy of "shift right testing," hasn't received the same recognition from DevOps community?

Pre-Production Testing to Predict Crashes & Failures

Performing software tests regularly throughout the development cycle — a strategy known as "continuous testing" — is a widely practiced tactic employed by companies that seek to optimize their DevOps process. Continuous testing informs DevOps teams with a steady stream of data surrounding application errors that can be used to inform their software development and fix any potential risks or bugs that could negatively affect the software.

Many DevOps teams have already recognized the many advantages of shift-left testing, which is a key aspect of continuous testing. Shifting left by testing software earlier in the production cycle enables DevOps teams to identify and rectify any design issues as they present themselves and before they can cause any future errors in the software application. Software tests can be shifted as far left as the initial product design and conception, enabling DevOps teams to catch bugs as early as possible in the development cycle.

Further, shift-left testing prevents the need to perform a barrage of tests on a litany of software features post-development. Performing software tests later in the development cycle can also leave DevOps teams working against tight deadlines to fix unexpected issues, which can lead to haphazard testing that misses software bugs.

In addition, shift-left testing significantly lowers the cost of bug fixes compared to testing at the end of a development cycle.

Shifting left also enables developers to avoid "context switching," a back-and-forth development pattern of switching between features as they are tested — allowing developers to focus their attention entirely on the specific aspect of the software application to which they are currently assigned. Otherwise, context switching can inadvertently cause developers to try and cut corners to avoid shifting deadlines or milestones and maintain project trajectory — causing more harm than good in the long run if the end-user is subjected to sub-par experience, service, or capabilities.

The Next Logical Step — Shifting Right

In keeping with the comparison of DevOps teams to automobile manufacturers, the benefits of shift-left testing are fairly similar to pre-production testing of vehicle parts — identify any issues early on to avoid costly re-designs later on in the product's lifecycle. It's a fairly logical conclusion, even if you only sit and think about the concept for a few minutes.

In fact, shifting testing left was a crucial area of focus for the majority of companies included in the recent Forrester Wave for Continuous Functional Test Automation (CFTA). However, the strategy of shifting testing right should also be a crucial area of focus for software developers. But what are the benefits to software developers when it comes to testing post-deployment?

Shifting right is a strategy of testing software later in the development cycle — even once the product or software is launched and in production. Ensuring that the final software product fully functions in real-world environments is critical to the long-term success of any software or technology solution. Just because a software application or system fully functions in the development environment doesn't guarantee functionality in the real world, as some software issues will only manifest once the application interacts with end-users.

Moreover, oftentimes, companies' test environments and production environments might not be exactly the same. If applications are deployed in different cloud or on-premise environments than those in which they are tested, it's impossible to predict how those applications will perform.

Additionally, it's an inevitable fact of life that stuff breaks. Even the most tested, tried and true products experience errors and breakdowns throughout their lifetime. Unless organizations are continuously testing in production phases, they will not be aware of any application issues until their customers report them — which is too often too late to deliver an exceptional customer experience.

To truly implement a strategy of continuous testing, it's only logical to assume that tests should be conducted throughout the software lifecycle — from design all the way through production. This includes testing as early as possible, and testing as late as possible to achieve a complete stream of testing data that identifies defects and informs the entire software development cycle.

The Emergence of DevTestOps

As software developers progress towards a strategy of truly continuous testing and embrace shift-left and shift-right testing, it will be crucial for testing and DevOps teams to collaborate cohesively. This collaborative approach is birthing a new technology and software development discipline: DevTestOps.

DevTestOps is gaining traction as a discipline for three major reasons that have become apparent to DevOps teams as they embrace continuous testing, shift-left and shift-right. First, unless developers engage live software systems by testing in production, it's basically impossible to accurately gauge the end-user experience. For example, different organizations leverage different providers for their cloud-hosted services for their test environment and for their production environment. Can you be 100% positive that your application works the same in any cloud environment without testing it?

Second, things break. It's inevitable that product features eventually break, become corrupted, or are interrupted by new features. Even if an application passes each pre-deployment test with flying colors, it will eventually experience an error somewhere down the road. Organizations can remain quality-focused, and prove their vigilance to their customers, by enabling their testing and DevOps teams to continue working together to improve the product post-deployment.

Finally, achieving 100% reliability for DevOps practices is simply not possible. Through DevTestOps, DevOps teams can gain key insights into how they can improve future software products in the development phase.

Companies and DevOps teams that hope to perfect software development processes are embracing a continuous testing strategy, but they should be careful not to turn a blind eye to the right side of the software development lifecycle by ignoring the importance of shifting testing right. While shifting left and shifting right are two different testing strategies, they should be viewed as complimentary, not mutually exclusive. DevOps and testing teams must work together to develop the best products possible while simultaneously eliminating the need for back-and-forth development and testing patterns.

James Isaacs is President of Cyara
Share this

Industry News

October 29, 2020

Cisco announced new software-delivered solutions designed to simplify IT operations across on-premise data centers and multicloud environments.

October 29, 2020

Bugsnag announced availability of user stability analytics, which will help developers gain a clearer understanding of how application errors are impacting the user experience and other key performance indicators (KPIs) for the business, as well as offer insights on whether to fix bugs or build new features.

October 29, 2020

HAProxy Technologies announced an open-source release of a VMware Open Virtual Appliance (OVA) virtual machine image of the HAProxy load balancer for vSphere, which HAProxy Technologies will maintain on GitHub.

October 28, 2020

Progress announced a number of new innovations designed to facilitate adoption and at-scale deployment of Chef offerings for both new and experienced users of the DevSecOps portfolio.

October 28, 2020

StackRox announced the release of KubeLinter, its new open source static analysis tool to identify misconfigurations in Kubernetes deployments.

October 28, 2020

Vercel announced Next.js 10 featuring a number of new capabilities that accelerate frontend developers’ ability to enrich end users’ web experiences globally.

October 27, 2020

ThinkTank has released a suite of applications designed to keep distributed agile teams aligned and engaged, regardless of physical location.

October 27, 2020

Cloudify, a Service Orchestration and Automation Platform, announced its latest 5.1 product release which aims to take one step further to permanently remove silos and roadblocks that are consistently associated with migration to the public cloud.

October 27, 2020

WhiteSource announced its new native integration for Microsoft Azure DevOps services.

October 26, 2020

NetApp unveiled a new serverless and storageless solution for containers from Spot by NetApp, a new autonomous hybrid cloud volume platform, and cloud-based virtual desktop solutions.

October 26, 2020

GeneXus released GeneXus 17, a new version of its platform that empowers enterprises to create and evolve new applications at unprecedented speed.

October 26, 2020

Alcide announced the company’s security solutions are now integrated with AWS Security Hub, sending real-time threat intelligence and compliance information to Amazon Web Services (AWS) for easy consumption by Security and DevSecOps teams.

October 22, 2020

Puppet announced Puppet Comply, a new product built to work with Puppet Enterprise aimed at assessing, remediating, and enforcing infrastructure configuration compliance policies at scale across traditional and cloud environments.

October 22, 2020

Harness announced two new modules: Continuous Integration Enterprise and Continuous Features.

October 22, 2020

Render announced automatic preview environments which are essential for rapid and collaborative development of modern applications.