Ending the Quality Assurance Winter: A Manifesto for Change
April 04, 2019

Paul Clauson
Functionize

Testing automation is in need of a revolution and so, we have published a manifesto for change. This manifesto makes some powerful statements about the direction we believe the test automation industry should be heading in. It's our case for change in the conservative world of testing. So, let's banish the QA winter and usher in the QA spring. A review of problems throughout a test lifecycle cements the need for autonomous testing.

The Test Lifecycle

Test creation: Scripts are still the main way of generating automated tests. Test recorders exist, but they just dumbly generate a new script from your interactions with the site under test. Usually, scripts have to be refined manually. In order to test cross-browser, you need to update your scripts. And testing responsive pages is even harder. As for nested DOMs, this is simply impossible for most systems.

Test execution: The majority of companies still maintain their own in-house test infrastructure, consisting of old, underpowered, end-of-life servers. Test teams use clunky systems, such as Selenium Grid, for managing orchestrations. So, tests can take hours to run, simply because of the inefficiency and age of the infrastructure. When (inevitably) things go wrong, the QA team finds itself relying on the DevOps team for help. Because infrastructure is local, backups of test results are probably not done effectively. And if anything goes wrong with the hardware you have no Plan B.

Test maintenance: In the depths of the QA winter, test maintenance is the bane of every test automation engineer's existence. Before Selenium, test maintenance just referred to updating test plans to keep up with changes to your site. Nowadays, every minor UI or styling change can break all your tests. This is a result of how brittle the selectors are in any script-based test system like Selenium. As a result, changing the border on a button can break tests; and, moving elements on the page is guaranteed to break things. We often hear of test teams spending 50 percent of their time just updating existing tests to reflect changes in the UI.

Testing Must be Strategic - Don't Leave it as an Afterthought

It's all too easy for management to overlook the importance of testing, until something breaks. At which point the test team becomes the target of everyone's ire. The problem with this view is it is rather short-sighted. If your test team failed to spot a bug it may not be a reflection on their performance. It is more likely to be a reflection on your lack of strategic focus on testing and quality management.

At the heart of this approach must come intelligent use of autonomous testing. We won't pretend autonomous testing can solve everything — sometimes you need manual and exploratory testing. Especially, if you are suffering from the curse of the unreproducible bug. But, used strategically, autonomous testing can save you from a lot of heartache and pain. It shouldn't be just an afterthought but should be one of the first things you think of when planning a new product.

Agile DevOps Requires Autonomous Testing

At its heart, DevOps is part of the Agile world. DevOps experts Len Bass, Ingo Weber and Liming Zhu have defined it as a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production while ensuring high quality.

In other words, DevOps teeters between developing and running. It's what you get when you apply the Agile approach to software development to deploying services in production.

A key part of this definition of DevOps is "ensuring high quality." As we all know, that is (or is meant to be) the definition of what a test engineer does. So, it's clear that there is a significant overlap between testing and DevOps. The second key part is the aim to "reduce the time" for a change to reach production. This is where autonomous testing becomes key.

In addition to the above considerations, our industry should apply these methodologies when it comes to the future of testing:

■ Plain English is the new test language

■ Tests should never break

■ Shifting testing left is not enough. Shift right

■ Software testing is about driving revenue, not just finding bugs

■ Quality is in the DNA of every great company

■ Autonomous testing is your brand's shield and protector

■ Intelligent automation just works

■ Testing must be cloud-first

The QA winter has endured for too long. It's time automated testing powered into the modern world. Here at Functionize, we are well on the way to creating a QA spring. A future where an intelligent test agent handles all your testing needs while freeing up resources, saving time, and driving revenues. A future where testing is a function everyone on your team can contribute to. Above all, a future where you can embed quality in everything you produce.

Paul Clauson is Chief Evangelist at Functionize
Share this

Industry News

July 09, 2020

ShiftLeft released a new version of NextGen Static Analysis (NG SAST), including new workflows, purpose-built for developers that significantly improve security, while enhancing productivity.

July 09, 2020

RunSafe Security announced a partnership with JFrog that will enable RunSafe to supercharge binary protections via a simple plugin that JFrog users can deploy within their Artifactory repositories and instantly protect binaries and containers.

July 09, 2020

LeanIX closed $80 million in Series D funding led by new investor Goldman Sachs Growth.

July 08, 2020

Afi.ai introduced Afi Data Platform, a cloud-based replication and resiliency service that helps to monitor, predict downtime and recover K8s applications.

July 08, 2020

D2iQ announced the release of Conductor, a new interactive learning platform that enables enterprises to access hands-on cloud native courses and training.

July 08, 2020

SUSE entered into a definitive agreement to acquire Rancher Labs.

July 07, 2020

Micro Focus announced AI-powered enhancements to the intelligent testing capabilities of the UFT Family, a unified set of solutions designed to reduce the overall complexity of automating the functional testing processes.

July 07, 2020

Push Technology announced the launch of a new Service API capability for Diffusion Cloud, Push’s Real-Time API Management Cloud Platform.

July 07, 2020

Lightrun exited stealth and announced $4M in seed funding for the first complete continuous debugging and observability platform for production applications.

July 01, 2020

JFrog announced the launch of ChartCenter, a free, security-focused central repository of Helm charts for the community.

July 01, 2020

Kong announced a significant upgrade to open source Kuma, Kuma 0.6, available today.

July 01, 2020

Compuware Corporation, a BMC company, announced new capabilities that further automate and integrate test data and test case execution, empowering IT teams to achieve high-performance application development quality, velocity and efficiency.

June 30, 2020

Couchbase announced the general availability of Couchbase Cloud, a fully-managed Database-as-a-Service (DBaaS).

June 30, 2020

Split Software announced new capabilities designed to accelerate the adoption of feature flags in large-scale organizations.

June 30, 2020

WhiteHat Security announced a discounted Web + Mobile Application Security bundle to help organizations secure the digital future.