State of Test Automation: Trends and Priorities for 2021
March 01, 2021

Eran Kinsbruner
Perfecto by Perforce

2020 was one of the most transformative years for software testing to date. Teams were forced to adapt to completely new work environments and learn to develop, test and innovate at warp speed. At Perfecto by Perforce, we were intrigued by the rollercoaster that was 2020 and wanted to glean more insights into the unique testing trends and challenges that surfaced as a result. As such, we surveyed more than 700 DevOps professionals for our 2021 State of Test Automation Report.

At a time when releases needed to happen faster, findings reported that teams actually slowed down — only 9% released daily, down 14% from the previous year. Digging into these numbers even more, we uncovered several key takeaways from the data, but the overarching theme was clear: teams need intelligent test automation and they need it now.


Teams Are Making Good Progress, but Struggling to Adopt Test Automation Across the Board

In an increasingly digital world, it's critical for teams to get on board with automation, as it allows them to do more testing in less time. Not only that, but it reduces the need for repetitive, manual tasks and can also deliver better results because of its ability to increase test coverage.

However, teams are struggling to fully get onboard. According to the report, 10% of teams are manually testing all tests. To date, 20% just started automating tests and nearly 50% are automating fewer than half of their them. Across the board, teams cited the lack of resources for creating test automation and proper coverage for test automation scenarios as being their top two biggest hurdles.

For those teams rarely automating tests, they were more likely to release once or twice per year and less likely to achieve continuous testing, integration and delivery. On the other hand, those with more mature automation practices were more likely to release daily or weekly, but also were challenged by false negatives, test flakiness and instability.

The "Shift Left" Mentality Is Taking Off, but Manual Testing Is Holding Teams Back

Development teams are really striving to "shift left" and begin testing as early in the lifecycle as possible so they can release more, faster. However, a majority find it difficult to get to that point. Our research found that 71% of organizations still struggle to fit test automation into the build process. Instead, they wait to test after the build and release less as a result.

Mature DevOps requires automated checkpoints throughout the build process. A major pain point for testing earlier can again be attributed to manual testing. Asked about the biggest obstacles to shifting left, 29% said manual testing, followed by 20% who said test cycle analysis and 18% who said test environment setup.

The smaller percentage of respondents who have been successful in shifting left faster are those who have democratized quality assurance across the entire organization, and implemented such practices as feature flags, A/B testing and cloud-based testing.

Top Priorities for the Coming Year Are Mobile Driven as a Result of the Pandemic

If 2020 showed us anything, it's how dependent we are on our mobile phones. According to an eMarketer report, US adults were projected to spend an additional 23 minutes per day on their phones amidst the pandemic. As a result, more than 60% of respondents to our survey said native mobile and mobile web apps are important or very important to their testing strategy in 2021. Within, a growing 13% are increasingly interested in foldable phones.

Other priority trends for 2021 include incorporating artificial intelligence (AI) and machine learning (ML) into codeless testing (61%), followed by 5G (29%) and progressive web apps (PWAS) (21%).

As a Result of Testing Challenges, Teams Are Putting Emphasis on Automation Investment

Digital transformation had been a long time coming even before the pandemic drastically expedited the transition. In less than a year, organizations had to go almost entirely digital as their sole way of engaging with customers, employees and partners alike. Accordingly, a greater emphasis has been put on web and mobile technologies, which require more rigorous testing, greater coverage and continuous monitoring.

The DevOps teams that succeed in this new digital age understand the value of test automation, and are actively addressing the automation skills gap by vetting more behavior-driven development (BDD) and scriptless solutions. Despite all the barriers to automation and shifting left, nearly half of all respondents said they plan to invest in test automation solutions within the year.

For those planning to invest, over 40% have an interest in using intelligent test automation solutions. The frameworks at top of mind include codeless (44%), Cypress (40%), Flutter (19%), Playwright (10%) and other (31%).

However, there is still a large population — 29% — that has no plans to invest at all.

Looking at Test Automation in 2021 and Beyond

Overall, the report revealed that the majority of organizations understand and support the implementation of automation and continuous testing, yet few are actually hitting it out of the park. In many instances, there are even misalignments within these organizations in which executives believe they are more automated than developers report.

Now that more testing is performed on digital platforms than ever before, enterprise teams must close the gap and embrace test automation tools if they want to keep up with their competition — and getting the C-suite on board is the first step.

Eran Kinsbruner is Chief Evangelist of Test Automation at Perfecto by Perforce
Share this

Industry News

September 27, 2022

DevOps Institute will host SKILup Festival in Singapore on November 15, 2022.

September 27, 2022

Delinea announced the latest release of DevOps Secrets Vault, its high-speed vault for DevOps and DevSecOps teams.

September 27, 2022

The Apptainer community announced version 1.1.0 of the popular container system for secure, high-performance computing (HPC). Improvements in the new version provide a smaller attack surface for production deployments while offering features that improve and simplify the user experience.

September 26, 2022

Secure Code Warrior unveiled Coding Labs, a new mechanism that allows developers to more easily move from learning to applying secure coding knowledge, leading to fewer vulnerabilities in code.

September 26, 2022

ActiveState announced the availability of the ActiveState Artifact Repository.

September 26, 2022

Split Software announced the availability of its Feature Data Platform in the Microsoft Azure Marketplace.

September 22, 2022

Katalon announced the launch of the Katalon Platform, a modern and comprehensive software quality management platform that enables teams of any size to easily and efficiently test, launch, and optimize apps, products, and software.

September 22, 2022

StackHawk announced its Deeper API Security Test Coverage release.

September 21, 2022

Platform9 announced the launch of its latest open source project, Arlon.

September 21, 2022

Redpanda Data announced Redpanda Console.

September 21, 2022

mabl announced its availability as a private listing on Google Cloud Marketplace.

September 21, 2022

Zesty announced a $75 million Series B funding round led by B Capital and Series A investor Sapphire Ventures.

September 20, 2022

Opsera, the Continuous Orchestration platform for DevOps, announced a free trial of its no-code Salesforce Release Management platform for fast and secure Salesforce releases.

September 20, 2022

Sysdig announced ToDo and Remediation Guru.

September 20, 2022

AutoRABIT announced CodeScan Shield.