State of Test Automation: Trends and Priorities for 2021
March 01, 2021

Eran Kinsbruner
Perfecto by Perforce

2020 was one of the most transformative years for software testing to date. Teams were forced to adapt to completely new work environments and learn to develop, test and innovate at warp speed. At Perfecto by Perforce, we were intrigued by the rollercoaster that was 2020 and wanted to glean more insights into the unique testing trends and challenges that surfaced as a result. As such, we surveyed more than 700 DevOps professionals for our 2021 State of Test Automation Report.

At a time when releases needed to happen faster, findings reported that teams actually slowed down — only 9% released daily, down 14% from the previous year. Digging into these numbers even more, we uncovered several key takeaways from the data, but the overarching theme was clear: teams need intelligent test automation and they need it now.


Teams Are Making Good Progress, but Struggling to Adopt Test Automation Across the Board

In an increasingly digital world, it's critical for teams to get on board with automation, as it allows them to do more testing in less time. Not only that, but it reduces the need for repetitive, manual tasks and can also deliver better results because of its ability to increase test coverage.

However, teams are struggling to fully get onboard. According to the report, 10% of teams are manually testing all tests. To date, 20% just started automating tests and nearly 50% are automating fewer than half of their them. Across the board, teams cited the lack of resources for creating test automation and proper coverage for test automation scenarios as being their top two biggest hurdles.

For those teams rarely automating tests, they were more likely to release once or twice per year and less likely to achieve continuous testing, integration and delivery. On the other hand, those with more mature automation practices were more likely to release daily or weekly, but also were challenged by false negatives, test flakiness and instability.

The "Shift Left" Mentality Is Taking Off, but Manual Testing Is Holding Teams Back

Development teams are really striving to "shift left" and begin testing as early in the lifecycle as possible so they can release more, faster. However, a majority find it difficult to get to that point. Our research found that 71% of organizations still struggle to fit test automation into the build process. Instead, they wait to test after the build and release less as a result.

Mature DevOps requires automated checkpoints throughout the build process. A major pain point for testing earlier can again be attributed to manual testing. Asked about the biggest obstacles to shifting left, 29% said manual testing, followed by 20% who said test cycle analysis and 18% who said test environment setup.

The smaller percentage of respondents who have been successful in shifting left faster are those who have democratized quality assurance across the entire organization, and implemented such practices as feature flags, A/B testing and cloud-based testing.

Top Priorities for the Coming Year Are Mobile Driven as a Result of the Pandemic

If 2020 showed us anything, it's how dependent we are on our mobile phones. According to an eMarketer report, US adults were projected to spend an additional 23 minutes per day on their phones amidst the pandemic. As a result, more than 60% of respondents to our survey said native mobile and mobile web apps are important or very important to their testing strategy in 2021. Within, a growing 13% are increasingly interested in foldable phones.

Other priority trends for 2021 include incorporating artificial intelligence (AI) and machine learning (ML) into codeless testing (61%), followed by 5G (29%) and progressive web apps (PWAS) (21%).

As a Result of Testing Challenges, Teams Are Putting Emphasis on Automation Investment

Digital transformation had been a long time coming even before the pandemic drastically expedited the transition. In less than a year, organizations had to go almost entirely digital as their sole way of engaging with customers, employees and partners alike. Accordingly, a greater emphasis has been put on web and mobile technologies, which require more rigorous testing, greater coverage and continuous monitoring.

The DevOps teams that succeed in this new digital age understand the value of test automation, and are actively addressing the automation skills gap by vetting more behavior-driven development (BDD) and scriptless solutions. Despite all the barriers to automation and shifting left, nearly half of all respondents said they plan to invest in test automation solutions within the year.

For those planning to invest, over 40% have an interest in using intelligent test automation solutions. The frameworks at top of mind include codeless (44%), Cypress (40%), Flutter (19%), Playwright (10%) and other (31%).

However, there is still a large population — 29% — that has no plans to invest at all.

Looking at Test Automation in 2021 and Beyond

Overall, the report revealed that the majority of organizations understand and support the implementation of automation and continuous testing, yet few are actually hitting it out of the park. In many instances, there are even misalignments within these organizations in which executives believe they are more automated than developers report.

Now that more testing is performed on digital platforms than ever before, enterprise teams must close the gap and embrace test automation tools if they want to keep up with their competition — and getting the C-suite on board is the first step.

Eran Kinsbruner is Chief Evangelist at Perfecto by Perforce
Share this

Industry News

September 23, 2021

Shortcut, the collaborative home for modern software teams, announced new Team-to-Workflow functionality.

September 23, 2021

Portainer announced the launch of its Portainer Business Charmed Operator, allowing for seamless integration with Canonical’s Charmed Kubernetes distribution.

September 22, 2021

Parasoft announced the launch and extension of its static application security testing (SAST) and API testing platform with penetration testing, shifting security testing into developer workflows.

September 22, 2021

Synopsys announced a new partnership with The Chertoff Group, a global advisory services firm that applies security expertise, technology insights, and policy intelligence to help clients build resilient organizations, gain competitive advantage, and accelerate growth.

September 21, 2021

ShiftLeft announced that its unified code security platform, ShiftLeft CORE, is now available on the AWS Marketplace.

September 21, 2021

JFrog announced its JFrog Artifactory and JFrog Xray solutions are now accredited in Iron Bank and available via Platform One.

September 21, 2021

GitKraken acquired BigBrassBand and its popular Git Integration for Jira.

September 20, 2021

D2iQ announced version 2.0 of the D2iQ Kubernetes Platform (DKP).

September 20, 2021

Platform9 joined Intel's Open Retail Initiative (ORI) and launched a new software-defined store solution.

September 20, 2021

FireMon has acquired DisruptOps.

September 16, 2021

SnapLogic released the latest version of its new SnapLogic Flows solution.

September 16, 2021

Postman launched a new version of its API Management platform.

September 16, 2021

Progress announced its R3 2021 release of Progress Telerik and Progress Kendo UI.

September 15, 2021

Progress announced the availability of Progress Sitefinity DX 14.0.

September 15, 2021

Launchable, the intelligence platform layer for all software testing, announced their latest addition to the platform, Flaky Tests Insights (beta).