State of Test Automation: Trends and Priorities for 2021
March 01, 2021

Eran Kinsbruner
Perfecto by Perforce

2020 was one of the most transformative years for software testing to date. Teams were forced to adapt to completely new work environments and learn to develop, test and innovate at warp speed. At Perfecto by Perforce, we were intrigued by the rollercoaster that was 2020 and wanted to glean more insights into the unique testing trends and challenges that surfaced as a result. As such, we surveyed more than 700 DevOps professionals for our 2021 State of Test Automation Report.

At a time when releases needed to happen faster, findings reported that teams actually slowed down — only 9% released daily, down 14% from the previous year. Digging into these numbers even more, we uncovered several key takeaways from the data, but the overarching theme was clear: teams need intelligent test automation and they need it now.


Teams Are Making Good Progress, but Struggling to Adopt Test Automation Across the Board

In an increasingly digital world, it's critical for teams to get on board with automation, as it allows them to do more testing in less time. Not only that, but it reduces the need for repetitive, manual tasks and can also deliver better results because of its ability to increase test coverage.

However, teams are struggling to fully get onboard. According to the report, 10% of teams are manually testing all tests. To date, 20% just started automating tests and nearly 50% are automating fewer than half of their them. Across the board, teams cited the lack of resources for creating test automation and proper coverage for test automation scenarios as being their top two biggest hurdles.

For those teams rarely automating tests, they were more likely to release once or twice per year and less likely to achieve continuous testing, integration and delivery. On the other hand, those with more mature automation practices were more likely to release daily or weekly, but also were challenged by false negatives, test flakiness and instability.

The "Shift Left" Mentality Is Taking Off, but Manual Testing Is Holding Teams Back

Development teams are really striving to "shift left" and begin testing as early in the lifecycle as possible so they can release more, faster. However, a majority find it difficult to get to that point. Our research found that 71% of organizations still struggle to fit test automation into the build process. Instead, they wait to test after the build and release less as a result.

Mature DevOps requires automated checkpoints throughout the build process. A major pain point for testing earlier can again be attributed to manual testing. Asked about the biggest obstacles to shifting left, 29% said manual testing, followed by 20% who said test cycle analysis and 18% who said test environment setup.

The smaller percentage of respondents who have been successful in shifting left faster are those who have democratized quality assurance across the entire organization, and implemented such practices as feature flags, A/B testing and cloud-based testing.

Top Priorities for the Coming Year Are Mobile Driven as a Result of the Pandemic

If 2020 showed us anything, it's how dependent we are on our mobile phones. According to an eMarketer report, US adults were projected to spend an additional 23 minutes per day on their phones amidst the pandemic. As a result, more than 60% of respondents to our survey said native mobile and mobile web apps are important or very important to their testing strategy in 2021. Within, a growing 13% are increasingly interested in foldable phones.

Other priority trends for 2021 include incorporating artificial intelligence (AI) and machine learning (ML) into codeless testing (61%), followed by 5G (29%) and progressive web apps (PWAS) (21%).

As a Result of Testing Challenges, Teams Are Putting Emphasis on Automation Investment

Digital transformation had been a long time coming even before the pandemic drastically expedited the transition. In less than a year, organizations had to go almost entirely digital as their sole way of engaging with customers, employees and partners alike. Accordingly, a greater emphasis has been put on web and mobile technologies, which require more rigorous testing, greater coverage and continuous monitoring.

The DevOps teams that succeed in this new digital age understand the value of test automation, and are actively addressing the automation skills gap by vetting more behavior-driven development (BDD) and scriptless solutions. Despite all the barriers to automation and shifting left, nearly half of all respondents said they plan to invest in test automation solutions within the year.

For those planning to invest, over 40% have an interest in using intelligent test automation solutions. The frameworks at top of mind include codeless (44%), Cypress (40%), Flutter (19%), Playwright (10%) and other (31%).

However, there is still a large population — 29% — that has no plans to invest at all.

Looking at Test Automation in 2021 and Beyond

Overall, the report revealed that the majority of organizations understand and support the implementation of automation and continuous testing, yet few are actually hitting it out of the park. In many instances, there are even misalignments within these organizations in which executives believe they are more automated than developers report.

Now that more testing is performed on digital platforms than ever before, enterprise teams must close the gap and embrace test automation tools if they want to keep up with their competition — and getting the C-suite on board is the first step.

Eran Kinsbruner is Chief Evangelist at Perfecto by Perforce
Share this

Industry News

May 19, 2022

Jellyfish announced the launch of Jellyfish Benchmarks, a way to add context around engineering metrics and performance by introducing a method for comparison.

May 19, 2022

Solo.io announced the addition and integration of Cilium networking into its Gloo Mesh platform, providing a complete application-networking solution for companies’ cloud-native digital transformation efforts.

May 19, 2022

Aqua Security announced multiple updates to Aqua Trivy, making it a unified scanner for cloud native security.

May 18, 2022

Red Hat unveiled updates across its portfolio of developer tools designed to help organizations build and deliver applications faster and more consistently across Kubernetes-based hybrid and multicloud environments.

May 18, 2022

Armory announced public early access to their new Continuous Deployment-as-a-Service product.

May 18, 2022

DataCore Software announced DataCore Bolt, enterprise-grade container-native storage software for DevOps.

May 17, 2022

DevOps Institute, a global professional association for advancing the human elements of DevOps, announced the release of the Upskilling IT 2022 report.

May 17, 2022

Replicated announced a host of new platform features and capabilities that enable their customers to accelerate enterprise adoption of their Kubernetes applications.

May 17, 2022

Codefresh announced that its flagship continuous delivery (CD) platform will be made accessible as a fully-hosted solution for DevOps teams seeking to quickly and easily achieve frictionless, GitOps-based continuous software delivery in the cloud.

May 16, 2022

Red Hat announced new capabilities and enhancements across its portfolio of open hybrid cloud solutions aimed at accelerating enterprise adoption of edge compute architectures through the Red Hat Edge initiative.

May 16, 2022

D2iQ announced a partnership with GitLab.

May 16, 2022

Kasten by Veeam announced the new Kasten by Veeam K10 V5.0 Kubernetes data management platform.

May 12, 2022

Red Hat introduced Red Hat Enterprise Linux 9, the Linux operating system designed to drive more consistent innovation across the open hybrid cloud, from bare metal servers to cloud providers and the farthest edge of enterprise networks.

May 12, 2022

Couchbase announced version 7.1 of Couchbase Server.

May 12, 2022

Copado added Copado Robotic Testing to Copado Essentials.