Ending the Quality Assurance Winter: A Manifesto for Change
April 04, 2019

Paul Clauson
Functionize

Testing automation is in need of a revolution and so, we have published a manifesto for change. This manifesto makes some powerful statements about the direction we believe the test automation industry should be heading in. It's our case for change in the conservative world of testing. So, let's banish the QA winter and usher in the QA spring. A review of problems throughout a test lifecycle cements the need for autonomous testing.

The Test Lifecycle

Test creation: Scripts are still the main way of generating automated tests. Test recorders exist, but they just dumbly generate a new script from your interactions with the site under test. Usually, scripts have to be refined manually. In order to test cross-browser, you need to update your scripts. And testing responsive pages is even harder. As for nested DOMs, this is simply impossible for most systems.

Test execution: The majority of companies still maintain their own in-house test infrastructure, consisting of old, underpowered, end-of-life servers. Test teams use clunky systems, such as Selenium Grid, for managing orchestrations. So, tests can take hours to run, simply because of the inefficiency and age of the infrastructure. When (inevitably) things go wrong, the QA team finds itself relying on the DevOps team for help. Because infrastructure is local, backups of test results are probably not done effectively. And if anything goes wrong with the hardware you have no Plan B.

Test maintenance: In the depths of the QA winter, test maintenance is the bane of every test automation engineer's existence. Before Selenium, test maintenance just referred to updating test plans to keep up with changes to your site. Nowadays, every minor UI or styling change can break all your tests. This is a result of how brittle the selectors are in any script-based test system like Selenium. As a result, changing the border on a button can break tests; and, moving elements on the page is guaranteed to break things. We often hear of test teams spending 50 percent of their time just updating existing tests to reflect changes in the UI.

Testing Must be Strategic - Don't Leave it as an Afterthought

It's all too easy for management to overlook the importance of testing, until something breaks. At which point the test team becomes the target of everyone's ire. The problem with this view is it is rather short-sighted. If your test team failed to spot a bug it may not be a reflection on their performance. It is more likely to be a reflection on your lack of strategic focus on testing and quality management.

At the heart of this approach must come intelligent use of autonomous testing. We won't pretend autonomous testing can solve everything — sometimes you need manual and exploratory testing. Especially, if you are suffering from the curse of the unreproducible bug. But, used strategically, autonomous testing can save you from a lot of heartache and pain. It shouldn't be just an afterthought but should be one of the first things you think of when planning a new product.

Agile DevOps Requires Autonomous Testing

At its heart, DevOps is part of the Agile world. DevOps experts Len Bass, Ingo Weber and Liming Zhu have defined it as a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production while ensuring high quality.

In other words, DevOps teeters between developing and running. It's what you get when you apply the Agile approach to software development to deploying services in production.

A key part of this definition of DevOps is "ensuring high quality." As we all know, that is (or is meant to be) the definition of what a test engineer does. So, it's clear that there is a significant overlap between testing and DevOps. The second key part is the aim to "reduce the time" for a change to reach production. This is where autonomous testing becomes key.

In addition to the above considerations, our industry should apply these methodologies when it comes to the future of testing:

■ Plain English is the new test language

■ Tests should never break

■ Shifting testing left is not enough. Shift right

■ Software testing is about driving revenue, not just finding bugs

■ Quality is in the DNA of every great company

■ Autonomous testing is your brand's shield and protector

■ Intelligent automation just works

■ Testing must be cloud-first

The QA winter has endured for too long. It's time automated testing powered into the modern world. Here at Functionize, we are well on the way to creating a QA spring. A future where an intelligent test agent handles all your testing needs while freeing up resources, saving time, and driving revenues. A future where testing is a function everyone on your team can contribute to. Above all, a future where you can embed quality in everything you produce.

Paul Clauson is Chief Evangelist at Functionize
Share this

Industry News

November 24, 2020

Red Hat announced new capabilities and features for Red Hat OpenShift, the company's enterprise Kubernetes platform.

November 24, 2020

Sectigo released Chef, Jenkins, JetStack Cert-Manager, Puppet, and SaltStack integrations for its certificate management platform.

November 24, 2020

DataStax released K8ssandra, an open-source distribution of Apache Cassandra on Kubernetes.

November 23, 2020

Spectro Cloud has released a new, self-hosted version of its flagship product, Spectro Cloud.

November 23, 2020

GitLab completed integration of Peach Tech, a security software firm specializing in protocol fuzz testing and dynamic application security testing (DAST) API testing, and Fuzzit, a continuous fuzz testing solution providing coverage-guided testing.

November 23, 2020

Fugue announced the availability of its SaaS product in AWS Marketplace, further simplifying the process for Amazon Web Services customers to use Fugue to bring their environments into compliance quickly, demonstrate compliance at any time, and Shift Left on cloud security.

November 19, 2020

Rollbar announced AI-assisted workflows powered by its new automation-grade grouping engine.

November 19, 2020

Buildkite expanded its integration with GitHub and introduced a new onboarding experience.

November 19, 2020

Rancher Labs launched a new Partner Program for the OEM and embedded community.

November 18, 2020

Puppet announced its evolution to an integrated automation platform to enable key business initiatives such as scaling DevOps, risk reduction, policy as code, and evolving cloud strategies.

November 18, 2020

Adaptavist has joined the GitLab partner program as a Select partner.

November 18, 2020

Postman launched the beta version of public workspaces, a hub that makes it possible for both API producers and consumers to seamlessly communicate and collaborate in real time without team or organizational boundaries.

November 17, 2020

Red Hat introduced new capabilities for Red Hat Enterprise Linux and Red Hat OpenShift intended to help enterprises bring edge computing into hybrid cloud deployments.

November 17, 2020

Humio announced the availability of the Humio Operator.

November 17, 2020

Accurics announced that Terrascan, the open source static code analyzer that enables developers to build secure infrastructure as code (IaC), has been extended to support Helm and Kustomize.