Exploratory Testing Cuts Risk of Software Failures
July 31, 2017

Wayne Ariola
Tricentis

Agile development compresses software testing cycles, jeopardizing risk coverage and opening the door for software failures. Here's what you can do:

The adoption of Agile and iterative development processes is forcing testing teams to conduct and complete testing more rapidly than in the past. Teams that previously had weeks or months to test must now accelerate testing to deliver even more comprehensive test results in a matter of hours or days. Today, testing must be performed under intense time pressure — often with reduced resources and budget. And that spells R-I-S-K. After all, how comprehensive can your testing be under such duress, and what is the risk of failure once the software goes into production?

A recent survey of over 2,400 respondents revealed that many enterprise testing teams are adopting exploratory testing in response to these challenges and risks.

Among these respondents, exploratory testing is increasingly being used to evaluate how an application performs from the perspective of the end user. Exploratory testing is often contrasted with formal testing, which focuses solely on verification (i.e., whether the acceptance criteria outlined in requirements specifications have been met). That's called validation. As such, formal testing monitors known risks, whereas exploratory testing focuses primarily on analyzing potential risks.

Verification and validation are independent procedures used together to check and confirm that a product meets the requirements and specifications and that it fulfills its intended purpose.

Because exploratory testing does not require laborious upfront planning, teams commonly apply it to start testing new software functionality as soon as it's completed. This promotes rapid defect detection within the compressed development cycles that are the norm today. And, because exploratory testing encourages branching and exploration of different testing ideas in a way that simulates the end user's perspective, it tends to uncover, and therefore snuff out, more critical defects than formal testing.

But why has exploratory testing become so widely adopted? And how are testing teams using it? Take a look at selected findings from the survey for more insight.

1. Agile Has Become the Primary Driver for Exploratory Testing Adoption

87 percent of respondents use exploratory testing to accelerate agile development cycles by providing feedback as quickly as possible to all parties concerned (e.g., development, business, and operations). Agile processes require teams to react quickly to changes and adapt accordingly. This is valuable since rapid feedback enables teams to “fail early,” when the failure can be remedied before a system goes into production.

2. Exploratory Testing Supplements Test Automation

91 percent of the respondents who are actively adopting or practicing DevOps consider exploratory testing a critical practice for risk reduction. More than 9 out of 10 respondents state that it is crucial to combine test automation and exploratory testing in a fast-paced development environment. To prevent process bottlenecks, teams use risk coverage criteria to select the most powerful set of automated tests to run at the various stages of the software delivery pipeline.

3. Exploratory Testing Accelerates Defect Detection

Respondents who practice exploratory testing estimate that, by exposing defects earlier (when defects are easier to eliminate), they accelerate delivery by approximately 20 percent. Exploratory testing exposes many defects that would otherwise be overlooked until real users encountered them in production.

4. Exploratory Testing Uncovers Types of Defects Overlooked by Formal Testing Techniques

Respondents practicing exploratory testing report that the top three issues exposed by exploratory testing are (95 percent) usability issues such as confusing interfaces or inconsistent usage patterns (95 percent); missing requirements (for example, functionality that is critical for the end user experience, but was not originally specified (87 percent); and problems with functionality that was implemented beyond the boundaries of specification, and thus not covered by specification-based tests (85 percent).

5. Exploratory Testing is Popular for Testing Usability

The most frequently software characteristic tested by exploratory testing is usability (93 percent), followed by performance (77 percent), security (62 percent), stability (54 percent), and safety (40 percent).

6. Exploratory testing is Geared for User Acceptance Testing, Regression Testing, and Smoke Testing

95 percent of respondents actively practicing exploratory testing state that exploratory testing is applied during user-acceptance testing, followed by 72 percent during regression testing, and 37 percent during smoke testing (testing that comprises a non-exhaustive set of tests to determine if a build is stable enough to proceed with further testing).

In any approach to software testing, the objective is to eliminate risk of software failure. With exploratory testing, software testers now have a potent addition to their testing regimens.

Wayne Ariola is CMO at Tricentis

The Latest

January 22, 2019

In 2019, competitive disruption will drive remaining laggards to a DevOps boiling point. As the industry moves to the plateau of productivity with DevOps automation and standard tooling, laggard executives will reach a management crisis point that will force actions ...

January 17, 2019

To better align business and IT objectives, enterprise organizations should focus on the core "problems" that individual business units face today in driving out real consumer value. Until the roadblocks and inhibitors — and, ultimately, the resultant technical debt — are removed from the equation, large enterprise organizations will continue struggling to succeed ...

January 16, 2019

Technical debt is what results when legacy platforms or highly integrated and dependent systems and processes inhibit large enterprise organizations from meeting the needs of internal business stakeholders. In many cases, the core objectives that drive real, monetizable business value are not aligned to the esoteric IT goals of "automation" and "Agile development." This creates a fundamental disconnect between business and IT ...

January 14, 2019

Budget season is an important time of the year for businesses because it gives senior IT and security leaders time to reflect on what went right this year and what initiatives need to be given priority in the new year. Recent research from Threat Stack shows security budgets are expected to increase by 19 percent over the next two years, but business leaders are still facing challenges determining where to allocate this budget in the face of rapidly evolving infrastructure ...

January 10, 2019

As organizations of all sizes are embracing hybrid and multi-cloud infrastructures, they are experiencing the many benefits of a more agile, distributed and high-speed environment where new applications and services can be built and delivered in days and weeks, rather than months and years. But as the adoption of these next generation architectures continues to grow, so do the complexities of securing the cloud workloads running on them ...

January 09, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 9, the final installment, covers microservices, containers and APIs ...

January 08, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 8, covers microservices and containers ...

January 07, 2019

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 7, covers the Cloud ...

December 20, 2018

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 6, covers DevOps Analytics, including AI and Machine Learning ...

December 19, 2018

DEVOPSdigest invited DevOps experts for their predictions on how DevOps and related technologies will evolve and impact business in 2019. Part 5 is all about testing ...

Share this