How Testing Kills DevOps
January 25, 2018

Wayne Ariola
Tricentis

You've already recognized that business transformation requires digital transformation. Your organization is staffed with the best and brightest developers ready to implement the innovative, business-differentiating technologies you need to attract, engage, and retain customers. And you've invested in scaling Agile, driving DevOps adoption, automating the Continuous Delivery pipeline, and all the other components involved in moving from ideation to delivery as rapidly as possible. So what could possibly go wrong? Testing.

Perhaps you might think that software testing isn't as exciting as development, where abstract ideas are magically transformed into attractive interfaces you can showcase to customers and staff. However, it can have a tremendous impact on the success of your digital transformation strategy. In fact, testing is often the silent killer of these efforts. Why? Because software testing is still dominated by yesterday's tools and outdated processes — which don't meet the needs of today's accelerated development processes.

What does this disconnect mean for you?

Throttled Acceleration

Agile and DevOps initiatives aim to accelerate the process of delivering working software to the end user. However, even organizations who have adopted Agile and DevOps report dismal test automation rates: around 25-30%. This means that after the rest of your software delivery pipeline is automated and optimized, an outdated testing process eventually emerges as the bottleneck. Once it's clear that testing is clogging the delivery pipe, there are three options: accept the throttled speed, release without adequate testing, or transform the testing process.

Risk to Your Brand

Now that software is the primary interface to the business, a software failure is a business failure. Yet, with today's compressed and continuous delivery cycles, it's simply impossible to test everything before every release — even if testing is automated.

To protect your brand while accelerating software delivery, testing must be re-aligned to focus on your top business risks. Traditional testing takes a "bottom-up" approach to validate whether new functionality works as expected. Modern delivery processes require testing to automatically assess the overall impact to the core user experience and instantly determine if the release candidate has an acceptable level of business risk.

Poorly-Allocated Resources

There are hard costs associated with trying to retrofit outdated testing processes and tools into modern delivery processes. Many organizations try to bridge the testing gap by throwing an abundance of manual testers at the problem … typically through a global system integrator. By the most recent estimates, this approach consumes approximately 35% of an average IT application development budget — a total of $35 billion per year, globally. A way to help testers achieve high test automation rates would let you reallocate a large portion of that testing spend towards creative tasks which advance competitive differentiators.

The Path Forward: Continuous Testing

Software testing must change. Software testing in the new world of DevOps remains dominated by yesterday's application lifecycle management (ALM) tools and manual testing — and they simply don't meet the needs of today's accelerated development processes.

The fact of the matter is that previous efforts to automate software testing have not yielded the expected results due to:

High maintenance: Traditional script-based automated tests need frequent updating to keep pace with highly-dynamic, accelerated release processes. This results in an overwhelming amount of false positives that require burdensome maintenance and/or cause automation efforts to be abandoned.

Slow execution time: Traditional tests are time-consuming to execute, so it is not practical to run a meaningful regression test suite on each build. This means the team lacks instant feedback on whether their changes impact the existing user experience — undermining the goals of CI.

Frequent failure: With today's complex, interconnected applications, test environment inconsistencies commonly impede test automation efforts and result in false positives. Again, this requires burdensome follow-up and/or causes automation efforts to be abandoned.

Now, the pressure of digital transformation requires a level of test automation that far surpasses the capabilities of legacy testing platforms. Balancing the business's demand for speed with their tolerance for risk requires Continuous Testing, which provides real-time insight into the application's business risk.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.

Test automation is designed to produce a set of pass/fail data points correlated to user stories or application requirements. Continuous Testing, on the other hand, focuses on business risk and providing insight on whether the software can be released. To achieve this shift, we need to stop asking "are we done testing" and instead concentrate on "does the release candidate have an acceptable level of business risk?"

It's important to recognize that no tool or technology can instantly "give" you Continuous Testing. Like Agile and DevOps, Continuous Testing requires changes throughout people, processes, and technology. However, trying to initiate the associated change in people and processes when your technology is not up to the task will be an uphill battle from the start … and ultimately a losing one.

Wayne Ariola is CMO at Tricentis

The Latest

February 15, 2018

Organizations are focusing investments on security and analytics, while actively researching emerging technologies such as machine learning and artificial intelligence, according to the 2018 CIO Tech Poll: Tech Priorities survey ...

February 14, 2018

With so many other initiatives to manage, DevOps isn't a top priority for all companies now. Many organizations believe it's wiser to wait. But in fact, the business case to integrate now is very strong — in fact, it's "do or die" territory. This infographic outlines some key reasons why organizations should integrate their software delivery value stream immediately ...

February 12, 2018

Accelerating multi-cloud deployments are enabling organizations to select the cloud platform that best meets the requirements of a specific application, according to the 2018 State of Application Delivery report from F5 Networks. However, this also increases the challenges many companies face in managing operations and security across multiple clouds as they transform their application portfolio to compete in the digital economy ...

February 08, 2018

The slowness of enterprise IT departments to embrace automated, cloud-native solutions for the cloud infrastructure challenges they face has resulted in IT infrastructure that is often ungoverned and insecure. And this is despite the fact that the cloud can be more secure as traditional data centers ...

February 06, 2018

Bank IT teams must embrace that their DevOps capabilities will determine their agile capability. Agile breaks down the barrier between the business and IT, and operations must be treated as a critical element of an agile program. In modern software delivery, the business, development and operations must execute as a unified team. To achieve this, banks are increasingly turning to Continuous Integration (CI) practices as part of the solution ...

February 05, 2018

IT professionals show a heightened concern for cybersecurity risk related to API use, according to a new survey conducted by Imperva. Specifically, 63 percent of respondents are most worried about DDoS threats, bot attacks, and authentication enforcement for APIs ...

February 01, 2018

DevOps are pretty clear for application development, those same applications often have a database back-end. If DevOps is increasing the frequency and reliability of new features for applications, a slower pace of database development can slow down and hinder those same releases ...

January 30, 2018

Without a doubt, DevOps is becoming the go-to strategy for organizations of all industries and sizes looking to master digital transformation and provide the fastest value to customers through software delivery. It is becoming clear that organizations adopting DevOps need a true leader (or engineer) to keep the transformation on track. The following are some tips when hiring for DevOps ...

January 29, 2018

When was the last time your company experienced a significant database error? If it happened in the last 3 months, you’re in good company. In a recent study, 60% of respondents reported a crash or significant database error occurring in the last 6 months. Roughly one in ten respondents reported a serious database problem in the past week ...

January 25, 2018

You've already recognized that business transformation requires digital transformation. Your organization is staffed with the best and brightest developers ready to implement the innovative, business-differentiating technologies you need to attract, engage, and retain customers. And you've invested in scaling Agile, driving DevOps adoption, automating the Continuous Delivery pipeline, and all the other components involved in moving from ideation to delivery as rapidly as possible. So what could possibly go wrong? Testing ...

Share this