Exploratory Testing Cuts Risk of Software Failures
July 31, 2017

Wayne Ariola

Agile development compresses software testing cycles, jeopardizing risk coverage and opening the door for software failures. Here's what you can do:

The adoption of Agile and iterative development processes is forcing testing teams to conduct and complete testing more rapidly than in the past. Teams that previously had weeks or months to test must now accelerate testing to deliver even more comprehensive test results in a matter of hours or days. Today, testing must be performed under intense time pressure — often with reduced resources and budget. And that spells R-I-S-K. After all, how comprehensive can your testing be under such duress, and what is the risk of failure once the software goes into production?

A recent survey of over 2,400 respondents revealed that many enterprise testing teams are adopting exploratory testing in response to these challenges and risks.

Among these respondents, exploratory testing is increasingly being used to evaluate how an application performs from the perspective of the end user. Exploratory testing is often contrasted with formal testing, which focuses solely on verification (i.e., whether the acceptance criteria outlined in requirements specifications have been met). That's called validation. As such, formal testing monitors known risks, whereas exploratory testing focuses primarily on analyzing potential risks.

Verification and validation are independent procedures used together to check and confirm that a product meets the requirements and specifications and that it fulfills its intended purpose.

Because exploratory testing does not require laborious upfront planning, teams commonly apply it to start testing new software functionality as soon as it's completed. This promotes rapid defect detection within the compressed development cycles that are the norm today. And, because exploratory testing encourages branching and exploration of different testing ideas in a way that simulates the end user's perspective, it tends to uncover, and therefore snuff out, more critical defects than formal testing.

But why has exploratory testing become so widely adopted? And how are testing teams using it? Take a look at selected findings from the survey for more insight.

1. Agile Has Become the Primary Driver for Exploratory Testing Adoption

87 percent of respondents use exploratory testing to accelerate agile development cycles by providing feedback as quickly as possible to all parties concerned (e.g., development, business, and operations). Agile processes require teams to react quickly to changes and adapt accordingly. This is valuable since rapid feedback enables teams to “fail early,” when the failure can be remedied before a system goes into production.

2. Exploratory Testing Supplements Test Automation

91 percent of the respondents who are actively adopting or practicing DevOps consider exploratory testing a critical practice for risk reduction. More than 9 out of 10 respondents state that it is crucial to combine test automation and exploratory testing in a fast-paced development environment. To prevent process bottlenecks, teams use risk coverage criteria to select the most powerful set of automated tests to run at the various stages of the software delivery pipeline.

3. Exploratory Testing Accelerates Defect Detection

Respondents who practice exploratory testing estimate that, by exposing defects earlier (when defects are easier to eliminate), they accelerate delivery by approximately 20 percent. Exploratory testing exposes many defects that would otherwise be overlooked until real users encountered them in production.

4. Exploratory Testing Uncovers Types of Defects Overlooked by Formal Testing Techniques

Respondents practicing exploratory testing report that the top three issues exposed by exploratory testing are (95 percent) usability issues such as confusing interfaces or inconsistent usage patterns (95 percent); missing requirements (for example, functionality that is critical for the end user experience, but was not originally specified (87 percent); and problems with functionality that was implemented beyond the boundaries of specification, and thus not covered by specification-based tests (85 percent).

5. Exploratory Testing is Popular for Testing Usability

The most frequently software characteristic tested by exploratory testing is usability (93 percent), followed by performance (77 percent), security (62 percent), stability (54 percent), and safety (40 percent).

6. Exploratory testing is Geared for User Acceptance Testing, Regression Testing, and Smoke Testing

95 percent of respondents actively practicing exploratory testing state that exploratory testing is applied during user-acceptance testing, followed by 72 percent during regression testing, and 37 percent during smoke testing (testing that comprises a non-exhaustive set of tests to determine if a build is stable enough to proceed with further testing).

In any approach to software testing, the objective is to eliminate risk of software failure. With exploratory testing, software testers now have a potent addition to their testing regimens.

Wayne Ariola is CMO at Tricentis

The Latest

March 19, 2018

The global DevOps market size is expected to reach USD 12.85 billion by 2025, according to a new study by Grand View Research, registering an 18.60% CAGR during the forecast period ...

March 15, 2018

More than half of companies (52%) admit to cutting back on security measures to meet a business deadline or objective, according to a SecOps research report released by Threat Stack ...

March 13, 2018

While microservices can certainly be used for greenfield projects, the survey suggests that this is not the sole source of value. In fact, more than half of respondents indicate that they are also using microservices to re-architect existing projects. The reality we see is that microservices can offer value to users along their IT transformation journey — whether they are just looking to update their current application portfolio or are gearing up for new initiatives ...

March 12, 2018

As DevOps teams and developers are looking to make 2018 the year in which technical crises are avoided, continuous testing should be at the top of their resolutions list. Here are four steps developers and DevOps teams can take to ensure the benefits of continuous testing are effectively implemented throughout the development process ...

March 08, 2018

Digital leaders will outpace their rivals by adopting methodologies and mindsets that shorten software delivery cycles. They'll also get really, really good at rapid, iterative change following design thinking principles ...

March 06, 2018

There are six major pain points that companies experience when they try to deploy and run Kubernetes in their complex environments, and there are also some best practices companies can use to address those pain points ...

March 05, 2018

With more than 30 Kubernetes solutions in the marketplace, it's tempting to think Kubernetes and the vendor ecosystem has solved the problem of operationalizing containers at scale. Far from it. There are six major pain points that companies experience when they try to deploy and run Kubernetes in their complex environments, and there are also some best practices companies can use to address those pain points ...

March 01, 2018

With the growing adoption of tablets and smartphones, companies are constantly seeking new web technologies that support multiple device types in addition to traditional desktops. At the same time, they are continually adding capabilities to their web applications that help users visualize and analyze data regardless of the platform or device used. To keep up in this changing technology environment, organizations must deliver these complex applications quickly, with high quality, and yet find ways to maximize their investment in these apps over the long haul ...

February 27, 2018

While most organizations are committed to the full adoption of both agile and DevOps, many are struggling with key challenges and missing out on the extensive benefits these practices can have on their bottom line, according to a global study by CA Technologies ...

February 26, 2018

To help understand the current state of development trends, Dimensional Research and Micro Focus worked together to create the, Managing the migration to DevOps: A global survey of software developers report. The research shows that nearly all organizations are already adopting or are taking a strong interest in the processes necessary to implement DevOps. But, there are challenges to overcome as companies because they are often running both traditional waterfall and DevOps development and release processes in tandem — and plan to support both into the future ...

Share this