Exploratory Testing Cuts Risk of Software Failures
July 31, 2017

Wayne Ariola
Tricentis

Agile development compresses software testing cycles, jeopardizing risk coverage and opening the door for software failures. Here's what you can do:

The adoption of Agile and iterative development processes is forcing testing teams to conduct and complete testing more rapidly than in the past. Teams that previously had weeks or months to test must now accelerate testing to deliver even more comprehensive test results in a matter of hours or days. Today, testing must be performed under intense time pressure — often with reduced resources and budget. And that spells R-I-S-K. After all, how comprehensive can your testing be under such duress, and what is the risk of failure once the software goes into production?

A recent survey of over 2,400 respondents revealed that many enterprise testing teams are adopting exploratory testing in response to these challenges and risks.

Among these respondents, exploratory testing is increasingly being used to evaluate how an application performs from the perspective of the end user. Exploratory testing is often contrasted with formal testing, which focuses solely on verification (i.e., whether the acceptance criteria outlined in requirements specifications have been met). That's called validation. As such, formal testing monitors known risks, whereas exploratory testing focuses primarily on analyzing potential risks.

Verification and validation are independent procedures used together to check and confirm that a product meets the requirements and specifications and that it fulfills its intended purpose.

Because exploratory testing does not require laborious upfront planning, teams commonly apply it to start testing new software functionality as soon as it's completed. This promotes rapid defect detection within the compressed development cycles that are the norm today. And, because exploratory testing encourages branching and exploration of different testing ideas in a way that simulates the end user's perspective, it tends to uncover, and therefore snuff out, more critical defects than formal testing.

But why has exploratory testing become so widely adopted? And how are testing teams using it? Take a look at selected findings from the survey for more insight.

1. Agile Has Become the Primary Driver for Exploratory Testing Adoption

87 percent of respondents use exploratory testing to accelerate agile development cycles by providing feedback as quickly as possible to all parties concerned (e.g., development, business, and operations). Agile processes require teams to react quickly to changes and adapt accordingly. This is valuable since rapid feedback enables teams to “fail early,” when the failure can be remedied before a system goes into production.

2. Exploratory Testing Supplements Test Automation

91 percent of the respondents who are actively adopting or practicing DevOps consider exploratory testing a critical practice for risk reduction. More than 9 out of 10 respondents state that it is crucial to combine test automation and exploratory testing in a fast-paced development environment. To prevent process bottlenecks, teams use risk coverage criteria to select the most powerful set of automated tests to run at the various stages of the software delivery pipeline.

3. Exploratory Testing Accelerates Defect Detection

Respondents who practice exploratory testing estimate that, by exposing defects earlier (when defects are easier to eliminate), they accelerate delivery by approximately 20 percent. Exploratory testing exposes many defects that would otherwise be overlooked until real users encountered them in production.

4. Exploratory Testing Uncovers Types of Defects Overlooked by Formal Testing Techniques

Respondents practicing exploratory testing report that the top three issues exposed by exploratory testing are (95 percent) usability issues such as confusing interfaces or inconsistent usage patterns (95 percent); missing requirements (for example, functionality that is critical for the end user experience, but was not originally specified (87 percent); and problems with functionality that was implemented beyond the boundaries of specification, and thus not covered by specification-based tests (85 percent).

5. Exploratory Testing is Popular for Testing Usability

The most frequently software characteristic tested by exploratory testing is usability (93 percent), followed by performance (77 percent), security (62 percent), stability (54 percent), and safety (40 percent).

6. Exploratory testing is Geared for User Acceptance Testing, Regression Testing, and Smoke Testing

95 percent of respondents actively practicing exploratory testing state that exploratory testing is applied during user-acceptance testing, followed by 72 percent during regression testing, and 37 percent during smoke testing (testing that comprises a non-exhaustive set of tests to determine if a build is stable enough to proceed with further testing).

In any approach to software testing, the objective is to eliminate risk of software failure. With exploratory testing, software testers now have a potent addition to their testing regimens.

Wayne Ariola is CMO at Tricentis
Share this

Industry News

September 19, 2019

DevOps Institute has entered into a fundraising partnership with the global Teach For All network in conjunction with the 2020 Upskilling: Enterprise DevOps Skills Survey.

September 19, 2019

Idera announced an agreement to acquire the software assets and related entities of WhereScape Software LTD., a provider of data infrastructure automation software.

September 19, 2019

Automox announced the launch of Automox Alive, a community for IT and security practitioners that facilitates the sharing of extensible Automox Worklets, to improve cyber hygiene.

September 18, 2019

Parasoft SOAtest API and UI functional testing solution has won a 2019 API Award in the Best in Microservices Infrastructure category.

September 18, 2019

Micro Focus announced the general availability of Vertica in Eon Mode for Pure Storage.

September 18, 2019

Mindtree announced a new service, InnoApp for Kubernetes, enabling enterprises to deploy containerized cloud applications on Microsoft Azure, accelerating application setup and innovation.

September 17, 2019

Redgate launched SQL Change Automation 4.0, extending its database change management automation tool to the free Microsoft SQL Server Management Studio (SSMS) development environment for the first time.

September 17, 2019

Oracle and VMware announced an expanded partnership to help customers leverage the companies’ enterprise software and cloud solutions to make the move to the cloud.

September 17, 2019

FireMon announced the introduction of FireMon Automation, a comprehensive set of policy automation management solutions designed to chart the course to smart security process automation.

September 16, 2019

Oracle announced the general availability of Java SE 13 (JDK 13).

September 16, 2019

Data Intensity launched its Automation-as-a-Service offering.

September 16, 2019

Mobile Labs launched the final addition to its mobile device cloud suite: GigaFox Red and GigaFox Silver.

September 12, 2019

Rafay Systems announced the general availability of its turnkey, SaaS-based offering designed to confront a complex set of ongoing challenges enterprises and service providers face when modernizing their applications.

September 12, 2019

StackRox announced the availability of the StackRox App for the Sumo Logic Continuous Intelligence Platform.

September 12, 2019

Lacework is receiving $42 million from Sutter Hill Ventures and Liberty Global Ventures.