Ending the Quality Assurance Winter: A Manifesto for Change
April 04, 2019

Paul Clauson
Functionize

Testing automation is in need of a revolution and so, we have published a manifesto for change. This manifesto makes some powerful statements about the direction we believe the test automation industry should be heading in. It's our case for change in the conservative world of testing. So, let's banish the QA winter and usher in the QA spring. A review of problems throughout a test lifecycle cements the need for autonomous testing.

The Test Lifecycle

Test creation: Scripts are still the main way of generating automated tests. Test recorders exist, but they just dumbly generate a new script from your interactions with the site under test. Usually, scripts have to be refined manually. In order to test cross-browser, you need to update your scripts. And testing responsive pages is even harder. As for nested DOMs, this is simply impossible for most systems.

Test execution: The majority of companies still maintain their own in-house test infrastructure, consisting of old, underpowered, end-of-life servers. Test teams use clunky systems, such as Selenium Grid, for managing orchestrations. So, tests can take hours to run, simply because of the inefficiency and age of the infrastructure. When (inevitably) things go wrong, the QA team finds itself relying on the DevOps team for help. Because infrastructure is local, backups of test results are probably not done effectively. And if anything goes wrong with the hardware you have no Plan B.

Test maintenance: In the depths of the QA winter, test maintenance is the bane of every test automation engineer's existence. Before Selenium, test maintenance just referred to updating test plans to keep up with changes to your site. Nowadays, every minor UI or styling change can break all your tests. This is a result of how brittle the selectors are in any script-based test system like Selenium. As a result, changing the border on a button can break tests; and, moving elements on the page is guaranteed to break things. We often hear of test teams spending 50 percent of their time just updating existing tests to reflect changes in the UI.

Testing Must be Strategic - Don't Leave it as an Afterthought

It's all too easy for management to overlook the importance of testing, until something breaks. At which point the test team becomes the target of everyone's ire. The problem with this view is it is rather short-sighted. If your test team failed to spot a bug it may not be a reflection on their performance. It is more likely to be a reflection on your lack of strategic focus on testing and quality management.

At the heart of this approach must come intelligent use of autonomous testing. We won't pretend autonomous testing can solve everything — sometimes you need manual and exploratory testing. Especially, if you are suffering from the curse of the unreproducible bug. But, used strategically, autonomous testing can save you from a lot of heartache and pain. It shouldn't be just an afterthought but should be one of the first things you think of when planning a new product.

Agile DevOps Requires Autonomous Testing

At its heart, DevOps is part of the Agile world. DevOps experts Len Bass, Ingo Weber and Liming Zhu have defined it as a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production while ensuring high quality.

In other words, DevOps teeters between developing and running. It's what you get when you apply the Agile approach to software development to deploying services in production.

A key part of this definition of DevOps is "ensuring high quality." As we all know, that is (or is meant to be) the definition of what a test engineer does. So, it's clear that there is a significant overlap between testing and DevOps. The second key part is the aim to "reduce the time" for a change to reach production. This is where autonomous testing becomes key.

In addition to the above considerations, our industry should apply these methodologies when it comes to the future of testing:

■ Plain English is the new test language

■ Tests should never break

■ Shifting testing left is not enough. Shift right

■ Software testing is about driving revenue, not just finding bugs

■ Quality is in the DNA of every great company

■ Autonomous testing is your brand's shield and protector

■ Intelligent automation just works

■ Testing must be cloud-first

The QA winter has endured for too long. It's time automated testing powered into the modern world. Here at Functionize, we are well on the way to creating a QA spring. A future where an intelligent test agent handles all your testing needs while freeing up resources, saving time, and driving revenues. A future where testing is a function everyone on your team can contribute to. Above all, a future where you can embed quality in everything you produce.

Paul Clauson is Chief Evangelist at Functionize
Share this

Industry News

March 03, 2021

Red Hat announced the latest release of Red Hat Process Automation, which delivers new developer tooling, extended support for eventing and streaming for event-driven architectures (EDA) through integration with Apache Kafka, and new monitoring capabilities through heatmap dashboards.

March 03, 2021

Leaders of the software development industry announced the formation of the Value Stream Management Consortium (VSMC).

March 03, 2021

Delphix and GenRocket announced a technology alliance designed to fulfill the needs of enterprise customers who desire a comprehensive test data solution that improves software quality.

March 02, 2021

JFrog announced that its DevOps Platform tools – JFrog Artifactory and JFrog Xray – are available with native deployment templates for customers using AWS GovCloud (US) and Azure Government clouds.

March 02, 2021

Spectro Cloud announced support for existing Kubernetes environments, including clusters on public cloud services such as Amazon EKS, Azure AKS and Google GKE, has been added to the Spectro Cloud Kubernetes management platform.

March 02, 2021

Idera announced the acquisition of PreEmptive Solutions, LLC, a provider of application protection and security.

March 01, 2021

CloudBolt Software announced the launch of OneFuse Community Edition, a free version of its codeless integration platform for automating, integrating, and extending private and hybrid cloud infrastructures.

March 01, 2021

DBmaestro launched support for Snowflake, the Data Cloud company.

March 01, 2021

Platform9 closed Series-D funding with an additional $12.5 million for a total of $37.5 million.

February 25, 2021

Red Hat announced Red Hat OpenShift 4.7, the latest version of the company’s enterprise Kubernetes platform.

February 25, 2021

Granulate announced the release of its open-source platform, the G-Profiler, a production profiling solution that measures the performance of code in production applications to facilitate compute optimization.

February 25, 2021

Checkmarx announced the launch of KICS (Keeping Infrastructure as Code Secure), an open source static analysis solution that enables developers to write more secure infrastructure as code (IaC).

February 24, 2021

Applause launched its Product Excellence Platform (PEP).

February 24, 2021

Mabl announced the beta release of their new native desktop application that empowers users to easily automate testing for browsers, mobile browsers, and APIs.

February 24, 2021

D2iQ announced the general availability of D2iQ Kaptain, the cloud native end-to-end platform for running ML workloads on Kubernetes.