State of Test Automation: Trends and Priorities for 2021
March 01, 2021

Eran Kinsbruner
Perfecto by Perforce

2020 was one of the most transformative years for software testing to date. Teams were forced to adapt to completely new work environments and learn to develop, test and innovate at warp speed. At Perfecto by Perforce, we were intrigued by the rollercoaster that was 2020 and wanted to glean more insights into the unique testing trends and challenges that surfaced as a result. As such, we surveyed more than 700 DevOps professionals for our 2021 State of Test Automation Report.

At a time when releases needed to happen faster, findings reported that teams actually slowed down — only 9% released daily, down 14% from the previous year. Digging into these numbers even more, we uncovered several key takeaways from the data, but the overarching theme was clear: teams need intelligent test automation and they need it now.


Teams Are Making Good Progress, but Struggling to Adopt Test Automation Across the Board

In an increasingly digital world, it's critical for teams to get on board with automation, as it allows them to do more testing in less time. Not only that, but it reduces the need for repetitive, manual tasks and can also deliver better results because of its ability to increase test coverage.

However, teams are struggling to fully get onboard. According to the report, 10% of teams are manually testing all tests. To date, 20% just started automating tests and nearly 50% are automating fewer than half of their them. Across the board, teams cited the lack of resources for creating test automation and proper coverage for test automation scenarios as being their top two biggest hurdles.

For those teams rarely automating tests, they were more likely to release once or twice per year and less likely to achieve continuous testing, integration and delivery. On the other hand, those with more mature automation practices were more likely to release daily or weekly, but also were challenged by false negatives, test flakiness and instability.

The "Shift Left" Mentality Is Taking Off, but Manual Testing Is Holding Teams Back

Development teams are really striving to "shift left" and begin testing as early in the lifecycle as possible so they can release more, faster. However, a majority find it difficult to get to that point. Our research found that 71% of organizations still struggle to fit test automation into the build process. Instead, they wait to test after the build and release less as a result.

Mature DevOps requires automated checkpoints throughout the build process. A major pain point for testing earlier can again be attributed to manual testing. Asked about the biggest obstacles to shifting left, 29% said manual testing, followed by 20% who said test cycle analysis and 18% who said test environment setup.

The smaller percentage of respondents who have been successful in shifting left faster are those who have democratized quality assurance across the entire organization, and implemented such practices as feature flags, A/B testing and cloud-based testing.

Top Priorities for the Coming Year Are Mobile Driven as a Result of the Pandemic

If 2020 showed us anything, it's how dependent we are on our mobile phones. According to an eMarketer report, US adults were projected to spend an additional 23 minutes per day on their phones amidst the pandemic. As a result, more than 60% of respondents to our survey said native mobile and mobile web apps are important or very important to their testing strategy in 2021. Within, a growing 13% are increasingly interested in foldable phones.

Other priority trends for 2021 include incorporating artificial intelligence (AI) and machine learning (ML) into codeless testing (61%), followed by 5G (29%) and progressive web apps (PWAS) (21%).

As a Result of Testing Challenges, Teams Are Putting Emphasis on Automation Investment

Digital transformation had been a long time coming even before the pandemic drastically expedited the transition. In less than a year, organizations had to go almost entirely digital as their sole way of engaging with customers, employees and partners alike. Accordingly, a greater emphasis has been put on web and mobile technologies, which require more rigorous testing, greater coverage and continuous monitoring.

The DevOps teams that succeed in this new digital age understand the value of test automation, and are actively addressing the automation skills gap by vetting more behavior-driven development (BDD) and scriptless solutions. Despite all the barriers to automation and shifting left, nearly half of all respondents said they plan to invest in test automation solutions within the year.

For those planning to invest, over 40% have an interest in using intelligent test automation solutions. The frameworks at top of mind include codeless (44%), Cypress (40%), Flutter (19%), Playwright (10%) and other (31%).

However, there is still a large population — 29% — that has no plans to invest at all.

Looking at Test Automation in 2021 and Beyond

Overall, the report revealed that the majority of organizations understand and support the implementation of automation and continuous testing, yet few are actually hitting it out of the park. In many instances, there are even misalignments within these organizations in which executives believe they are more automated than developers report.

Now that more testing is performed on digital platforms than ever before, enterprise teams must close the gap and embrace test automation tools if they want to keep up with their competition — and getting the C-suite on board is the first step.

Eran Kinsbruner is Senior Director of Product Marketing at Perfecto by Perforce
Share this

Industry News

April 15, 2021

Docker announced general availability of the Docker Desktop for Mac [Apple Silicon], enabling developers to leverage the advantages of the latest Macs powered by the M1 chip and extending the reach of their Docker collaborative application development platform to a new architecture.

April 15, 2021

Software AG announced new innovations of its webMethods platform for APIs, Integration and Microservices. With this release, companies can simplify and accelerate their digital transformation initiatives while also speeding their adoption of cloud.

April 15, 2021

Skuid, a toolkit for creating unique and adopted Salesforce experiences, introduced the Skuid Chicago release, a set of features and enhancements providing more declarative support to app designers and builders.

April 14, 2021

SmartBear has integrated TestComplete, its UI test automation tool, with BitBar, its native mobile device cloud.

April 14, 2021

Elastic announced an expanded strategic partnership with Confluent to deliver the best integrated product experience to the Apache Kafka and Elasticsearch community.

April 14, 2021

Threat Stack announced its ability to support AWS Graviton2-based instances through the Threat Stack Cloud Security Platform.

April 13, 2021

Broadcom and Google Cloud announced a strategic collaboration to accelerate innovation and strengthen cloud services integration within the core software franchises of Broadcom.

April 13, 2021

Nylas announced the launch of Components, JavaScript UI/UX solutions that allow developers to bring productivity features to market faster without needing to design front-end elements from scratch.

April 13, 2021

Perforce Software announces its new version control desktop client — Helix Sync — enabling non-coders such as artists and designers to version digital assets, with a simple drag-and-drop UI.

April 12, 2021

ShiftLeft introduced ShiftLeft CORE, a unified code security platform.

April 12, 2021

GrammaTech announced a new version of its CodeSonar SAST (static application security testing) product that helps developers build safer and more secure code without disrupting workflows.

April 12, 2021

Panaya announced a strategic partnership with Being Guided, a Salesforce Consulting Partner, specializing in the CRM and Salesforce ecosystem, to bring Panaya's ForeSight solution to a wider audience.

April 08, 2021

Palo Alto Networks announced the second generation of Checkov, the static analysis tool for infrastructure as code (IaC).

April 08, 2021

Postman now allows any team with up to three members to collaborate in Postman with unlimited shared workspaces and unlimited shared requests at no cost.

April 08, 2021

Taos, an IBM company, has announced 24x5 managed service availability.