DEVOPSdigest

The software testing landscape is undergoing a transformative shift, driven by emerging technologies and evolving development practices. Rainforest QA's survey of over 625 software engineering leaders from technology companies across the United States, Canada, the United Kingdom, and Australia reveals new insights into how development teams are approaching end-to-end (E2E) test automation in 2024.

Test Automation Takes Root Early

One of the most striking findings challenges the conventional wisdom about when startups begin investing in automated testing. Contrary to an assumption that early-stage companies delay implementation of robust testing, our research shows an early, proactive approach to test automation.

A full 55% of teams with only 1-5 developers automate their E2E tests, and this number jumps to 69% for teams with 6-10 developers. This early adoption reflects a growing understanding that maintaining product quality is crucial from the earliest stages of software development.

Adoption rates of test automation tend to grow as development teams do. But it's not a universal practice. Even among larger teams, 10-15% continue to rely entirely on manual testing. This can be due to different factors, including the type of product being tested. For example, video games and VR software aren't supported (or well-supported) by test automation, so the only recourse is manual testing.

The AI and Testing Paradox

Generative AI has made significant inroads, with 81% of respondents reporting AI usage in their software testing workflows. The technology's adoption is accompanied by growing confidence — 56% of respondents report increased trust in AI's accuracy compared to a year ago, with an average trust score of 8.1 out of 10.

However, the impact of AI is more complex than its adoption rates suggest. Despite high enthusiasm, AI hasn't yet delivered clear productivity gains for teams using open-source testing frameworks. In fact, open-source teams using AI for test creation and maintenance actually spend slightly more time on these tasks compared to those not using AI. This doesn't mean AI is ineffective; rather, it suggests the technology and its implementations are still evolving.

There are glimmers of promise, particularly for smaller teams. Those using AI for test maintenance are more likely to keep their automated test suites up to date — a critical challenge for resource-constrained development teams.

No-Code: A Game-Changing Alternative

Perhaps the most interesting insight is the emergence of no-code testing tools as a transformative approach to test automation. Teams using no-code platforms spend notably less time creating and maintaining automated tests compared to those wrestling with open-source frameworks.

The advantages are especially pronounced for smaller teams. Specifically, 93% of small teams using no-code tools successfully keep their test suites up to date, compared to just 75% of those using open source. Plus, only 10% of small teams using no-code spent more than 20 hours per week on test maintenance, compared to 19% of teams using open-source frameworks.

This substantial difference highlights an opportunity for development teams who want to remove bottlenecks from their quality assurance processes and ship code faster.

Looking Ahead

The testing landscape is evolving at an unprecedented pace. While AI hasn't yet delivered on its full promise, its potential remains. Industry experts anticipate future tools will increasingly blend the intuitive interfaces of no-code platforms with AI's capabilities, creating more accessible and efficient testing solutions.

As software development continues to accelerate, how teams approach testing will be crucial in maintaining product quality, ensuring reliable releases, and maintaining competitiveness.