Less Than 20% of QA Processes in "Good" Health
April 23, 2019

Derek Choy
Rainforest QA

In 2018, software defects in the US cost about $2.85 trillion. Needless to say, there's room for improvement. If you haven't taken a look at the QA health of your organization, you should consider it. How much money could be saved by better test coverage?

It might seem odd, but one of the most important metrics to an organization's QA health is customer satisfaction. CSAT reflects on product quality, and knowing where your coverage falls short can help shape a strategy that improves QA, product, quality, and CSAT.

Rainforest QA teamed up with TechWell to diagnose the state of QA health by surveying 250 QA testers, engineers, managers, and other QA professionals to understand how they approach QA in their organizations.


Weak Pulse

As it turns out, only 17% of respondents' QA teams are in "good" health or better, with more than half merely surviving, and 32% practically on life support. Why is good QA such a struggle for some? It could be that for many companies QA operates as a separate entity.

For an organization to get the most from its QA team, QA goals should be aligned with business goals. However, that's usually not the case. Only 48% of respondents report following that best-practice alignment. 20% of respondents report not having any QA goals at all. When it comes to test coverage, we see similar numbers.

Test coverage is an integral factor in product quality, so determining test coverage is an important step toward reaching that goal. Ideally, a QA team will work with development to align test coverage with code and measure how much code is exercised when tests are run. We find that 49% of respondents are following QA best practices for determining test coverage, but the other half of those surveyed currently do nothing to determine coverage.

Taking Control of QA

Here are some other eye-opening statistics we found:

■ 46% of respondents believe their QA process does not adequately align with development cycles
■ 10% of respondents are "extremely confident" that their current test coverage will ensure a high-quality product
■ The 48% of respondents who are following smart QA measurement practices keep track of metrics such as:
- number of tests executed
- number of bugs found in test vs. production
- percent of automated tests
- resolution

At this point, the main takeaway from the 2019 QA Health Survey won't be a surprise. Many QA organizations need to take better control of their QA fitness, and soon. Development and deployment processes are not slowing down. Users demand better features, more services, and higher quality.

Derek Choy is CIO at Rainforest QA
Share this

Industry News

September 24, 2020

NetApp announced the availability of Elastigroup for Microsoft Azure Spot Virtual Machines (VMs).

September 24, 2020

CloudBees announced a robust new set of DevSecOps capabilities for CloudBees CI and CloudBees CD. The new capabilities enable customers to perform early and frequent security checks and ensure that security is an integral part of the whole software delivery pipeline workflow, without sacrificing speed or increasing risk.

September 24, 2020

Pulumi announced the release of a Pulumi-native provider for Microsoft Azure that provides 100% coverage of Azure Resource Manager (ARM), the deployment and management service for Azure that enables users to create, update and delete resources in their Azure accounts.

September 23, 2020

Puppet announced new Windows services, integrations and enhancements aimed at making it easier to automate and manage infrastructure using tools Windows admins rely on. The latest updates include services around Group Policy Migration and Chocolatey, as well as enhancements to the Puppet VS Code Extension, and a new Puppet PowerShell DSC Builder module.

September 23, 2020

Red Hat announced the release of Red Hat OpenShift Container Storage 4.5, delivering Kubernetes-based data services for modern, cloud-native applications across the open hybrid cloud.

September 23, 2020

Copado, a native DevOps platform for Salesforce, has acquired ClickDeploy.

September 22, 2020

CloudBees announced general availability of the first two modules of its Software Delivery Management solution.

September 22, 2020

Applause announced the availability of its Bring Your Own Testers (BYOT) feature that enables clients to manage their internal teams – employees, friends, family members and existing customers – and invite them to test cycles in the Applause Platform alongside Applause’s vetted and expert community of testers.

September 22, 2020

Kasten announced the integration of the K10 data management platform with VMware vSphere and Tanzu Kubernetes Grid Service.

September 21, 2020

PagerDuty entered into a definitive agreement to acquire Rundeck, a provider of DevOps automation for enterprise.

September 21, 2020

Grafana Labs announced the release of Grafana Metrics Enterprise, a modern Prometheus-as-a-Service solution designed for the scale, architecture, and security needs of enterprises as they expand their observability initiatives.

September 21, 2020

Portshift's Cloud Workload Protection platform is now available through the Red Hat Marketplace.

September 17, 2020

env0, a developer of Infrastructure-as-Code (IaC) management software, announced the availability of its new open source solution for Terraform users, Terratag.

September 17, 2020

Push Technology announced a partnership with Innova Solutions, an ACS Solutions company, specializing in global information technology services.

September 17, 2020

Alcide achieved the AWS Outposts Ready designation, part of the Amazon Web Services (AWS) Service Ready Program.