How Testing Kills DevOps
January 25, 2018

Wayne Ariola
Tricentis

You've already recognized that business transformation requires digital transformation. Your organization is staffed with the best and brightest developers ready to implement the innovative, business-differentiating technologies you need to attract, engage, and retain customers. And you've invested in scaling Agile, driving DevOps adoption, automating the Continuous Delivery pipeline, and all the other components involved in moving from ideation to delivery as rapidly as possible. So what could possibly go wrong? Testing.

Perhaps you might think that software testing isn't as exciting as development, where abstract ideas are magically transformed into attractive interfaces you can showcase to customers and staff. However, it can have a tremendous impact on the success of your digital transformation strategy. In fact, testing is often the silent killer of these efforts. Why? Because software testing is still dominated by yesterday's tools and outdated processes — which don't meet the needs of today's accelerated development processes.

What does this disconnect mean for you?

Throttled Acceleration

Agile and DevOps initiatives aim to accelerate the process of delivering working software to the end user. However, even organizations who have adopted Agile and DevOps report dismal test automation rates: around 25-30%. This means that after the rest of your software delivery pipeline is automated and optimized, an outdated testing process eventually emerges as the bottleneck. Once it's clear that testing is clogging the delivery pipe, there are three options: accept the throttled speed, release without adequate testing, or transform the testing process.

Risk to Your Brand

Now that software is the primary interface to the business, a software failure is a business failure. Yet, with today's compressed and continuous delivery cycles, it's simply impossible to test everything before every release — even if testing is automated.

To protect your brand while accelerating software delivery, testing must be re-aligned to focus on your top business risks. Traditional testing takes a "bottom-up" approach to validate whether new functionality works as expected. Modern delivery processes require testing to automatically assess the overall impact to the core user experience and instantly determine if the release candidate has an acceptable level of business risk.

Poorly-Allocated Resources

There are hard costs associated with trying to retrofit outdated testing processes and tools into modern delivery processes. Many organizations try to bridge the testing gap by throwing an abundance of manual testers at the problem … typically through a global system integrator. By the most recent estimates, this approach consumes approximately 35% of an average IT application development budget — a total of $35 billion per year, globally. A way to help testers achieve high test automation rates would let you reallocate a large portion of that testing spend towards creative tasks which advance competitive differentiators.

The Path Forward: Continuous Testing

Software testing must change. Software testing in the new world of DevOps remains dominated by yesterday's application lifecycle management (ALM) tools and manual testing — and they simply don't meet the needs of today's accelerated development processes.

The fact of the matter is that previous efforts to automate software testing have not yielded the expected results due to:

High maintenance: Traditional script-based automated tests need frequent updating to keep pace with highly-dynamic, accelerated release processes. This results in an overwhelming amount of false positives that require burdensome maintenance and/or cause automation efforts to be abandoned.

Slow execution time: Traditional tests are time-consuming to execute, so it is not practical to run a meaningful regression test suite on each build. This means the team lacks instant feedback on whether their changes impact the existing user experience — undermining the goals of CI.

Frequent failure: With today's complex, interconnected applications, test environment inconsistencies commonly impede test automation efforts and result in false positives. Again, this requires burdensome follow-up and/or causes automation efforts to be abandoned.

Now, the pressure of digital transformation requires a level of test automation that far surpasses the capabilities of legacy testing platforms. Balancing the business's demand for speed with their tolerance for risk requires Continuous Testing, which provides real-time insight into the application's business risk.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.

Test automation is designed to produce a set of pass/fail data points correlated to user stories or application requirements. Continuous Testing, on the other hand, focuses on business risk and providing insight on whether the software can be released. To achieve this shift, we need to stop asking "are we done testing" and instead concentrate on "does the release candidate have an acceptable level of business risk?"

It's important to recognize that no tool or technology can instantly "give" you Continuous Testing. Like Agile and DevOps, Continuous Testing requires changes throughout people, processes, and technology. However, trying to initiate the associated change in people and processes when your technology is not up to the task will be an uphill battle from the start … and ultimately a losing one.

Wayne Ariola is CMO at Tricentis

The Latest

December 11, 2018

Companies expect increased reliance on Cloud Native Applications (CNAs), however security concerns could prove to be a major obstacle, according to The State of Cloud Native Security ...

December 06, 2018

The general consensus tends to be that in the world of agile and DevOps, ITSM teams are increasingly being left behind. But the truth is, in more forward-thinking IT organizations, this isn’t the case. The fact is that ITSM is playing, or at least should play, a growing role in support of agile and DevOps initiatives. But this role still remains limited due to the fact that DevOps teams, and their management, are (more often than not) leaving them out as a tool of choice ...

December 05, 2018

The industry is revealing increasingly optimistic attitudes towards mainframes, with 93% of executives and 92% of all respondents viewing the mainframe as a strong long-term platform – the highest level in five years – according to the 2018 Mainframe Research Report from BMC ...

December 03, 2018

ActiveState surveyed developers and programmers in 92 countries to better understand their pain points and assess how businesses can better work with their organizations. The survey results establish a starting point for understanding the challenges that coders confront when working with open source runtimes ...

November 29, 2018

Organizations with established DevSecOps programs and practices greatly outperform their peers in how quickly they address flaws. The most active DevSecOps programs fix flaws more than 11.5 times faster than the typical organization, due to ongoing security checks during continuous delivery of software builds, largely the result of increased code scanning, according the latest State of Software Security (SOSS) report from CA Veracode ..

November 27, 2018

The push to make banking products digitally ready (and very quickly) has spurred the old “buy vs. build” debate in bank IT departments: Should we build our own software from scratch in-house? Or do we buy off-the-shelf solutions from third-parties? And while this dichotomy may have been a suitable mentality years ago at the start of the digital transformation revolution in banking, it simply no longer fits with the reality of today's more complex development landscape ...

November 26, 2018

With the rise of next-generation technologies, businesses have access to more data than ever, creating opportunities to develop new channels for revenue. Contributing to the increase in data is a growing reliance on the external supply chain. However, with the influx of data comes the necessity to understand the entire third-party ecosystem; its benefits and risks. Some of the most devastating breaches have been attributed to a third party ...

November 20, 2018

In today's digital economy, monitoring is a must. Your customers must be able to access your website and your apps, interact, purchase — and monitoring is one way to make sure this keeps happening. But the first question has to be: What should be monitored? With this in mind, APMdigest asked experts from across the IT industry for their opinions on what IT departments should be monitoring to ensure digital performance ...

November 19, 2018

Software developers and security teams have a well-known antagonistic relationship. Dev teams often feel plagued by the restrictive security standards placed on them by security teams that inhibit their ability to rapidly write applications, while security teams view developers as one of the biggest threats with which they have to grapple. There are three core challenges that must be addressed in order for security and DevOps to be in lockstep ...

November 15, 2018

Serverless infrastructure environments are set to become the dominant paradigm for enterprise technology deployments, according to a new report — Why the Fuss About Serverless? — released by Leading Edge Forum ...

Share this