Exploratory Testing Cuts Risk of Software Failures
July 31, 2017

Wayne Ariola
Tricentis

Agile development compresses software testing cycles, jeopardizing risk coverage and opening the door for software failures. Here's what you can do:

The adoption of Agile and iterative development processes is forcing testing teams to conduct and complete testing more rapidly than in the past. Teams that previously had weeks or months to test must now accelerate testing to deliver even more comprehensive test results in a matter of hours or days. Today, testing must be performed under intense time pressure — often with reduced resources and budget. And that spells R-I-S-K. After all, how comprehensive can your testing be under such duress, and what is the risk of failure once the software goes into production?

A recent survey of over 2,400 respondents revealed that many enterprise testing teams are adopting exploratory testing in response to these challenges and risks.

Among these respondents, exploratory testing is increasingly being used to evaluate how an application performs from the perspective of the end user. Exploratory testing is often contrasted with formal testing, which focuses solely on verification (i.e., whether the acceptance criteria outlined in requirements specifications have been met). That's called validation. As such, formal testing monitors known risks, whereas exploratory testing focuses primarily on analyzing potential risks.

Verification and validation are independent procedures used together to check and confirm that a product meets the requirements and specifications and that it fulfills its intended purpose.

Because exploratory testing does not require laborious upfront planning, teams commonly apply it to start testing new software functionality as soon as it's completed. This promotes rapid defect detection within the compressed development cycles that are the norm today. And, because exploratory testing encourages branching and exploration of different testing ideas in a way that simulates the end user's perspective, it tends to uncover, and therefore snuff out, more critical defects than formal testing.

But why has exploratory testing become so widely adopted? And how are testing teams using it? Take a look at selected findings from the survey for more insight.

1. Agile Has Become the Primary Driver for Exploratory Testing Adoption

87 percent of respondents use exploratory testing to accelerate agile development cycles by providing feedback as quickly as possible to all parties concerned (e.g., development, business, and operations). Agile processes require teams to react quickly to changes and adapt accordingly. This is valuable since rapid feedback enables teams to “fail early,” when the failure can be remedied before a system goes into production.

2. Exploratory Testing Supplements Test Automation

91 percent of the respondents who are actively adopting or practicing DevOps consider exploratory testing a critical practice for risk reduction. More than 9 out of 10 respondents state that it is crucial to combine test automation and exploratory testing in a fast-paced development environment. To prevent process bottlenecks, teams use risk coverage criteria to select the most powerful set of automated tests to run at the various stages of the software delivery pipeline.

3. Exploratory Testing Accelerates Defect Detection

Respondents who practice exploratory testing estimate that, by exposing defects earlier (when defects are easier to eliminate), they accelerate delivery by approximately 20 percent. Exploratory testing exposes many defects that would otherwise be overlooked until real users encountered them in production.

4. Exploratory Testing Uncovers Types of Defects Overlooked by Formal Testing Techniques

Respondents practicing exploratory testing report that the top three issues exposed by exploratory testing are (95 percent) usability issues such as confusing interfaces or inconsistent usage patterns (95 percent); missing requirements (for example, functionality that is critical for the end user experience, but was not originally specified (87 percent); and problems with functionality that was implemented beyond the boundaries of specification, and thus not covered by specification-based tests (85 percent).

5. Exploratory Testing is Popular for Testing Usability

The most frequently software characteristic tested by exploratory testing is usability (93 percent), followed by performance (77 percent), security (62 percent), stability (54 percent), and safety (40 percent).

6. Exploratory testing is Geared for User Acceptance Testing, Regression Testing, and Smoke Testing

95 percent of respondents actively practicing exploratory testing state that exploratory testing is applied during user-acceptance testing, followed by 72 percent during regression testing, and 37 percent during smoke testing (testing that comprises a non-exhaustive set of tests to determine if a build is stable enough to proceed with further testing).

In any approach to software testing, the objective is to eliminate risk of software failure. With exploratory testing, software testers now have a potent addition to their testing regimens.

Wayne Ariola is CMO at Tricentis

The Latest

December 14, 2017

Around one in five business leaders indicating that their software budget had increased 50 percent or more over the past three years to support digital transformation projects. However, the increased software development investment has not translated to greater security budgets or awareness of the security risks insecure software introduces: only 50 percent of business leaders surveyed understand the risk that vulnerable software poses to their business, according to Securing the Digital Economy, a report from Veracode ...

December 13, 2017

Metrics-oriented thinking is key to continuous improvement – and a core tenant of any agile or DevOps philosophy. Metrics are factual and once agreed upon, these facts are used to drive discussions and methods. They also allow for a collaborative effort to execute decisions that contribute towards business outcomes ...

December 11, 2017

The benefits of DevOps are potentially enormous, but simply identifying the benefits is not enough. A faster time to market may be a good customer story, but with no directly measurable monetary return, the value of DevOps can still be questioned at board level. Businesses want more than promises if they are to sign off on financial decisions: they need to know the Return on Investment (ROI) as well, with facts and figures that demonstrate what they will gain ...

December 07, 2017

Modern businesses are migrating to a cloud-based model for hosting sensitive data to reap the benefits of agility and cost savings as well as to keep pace with customer demand. Cloud-Native methodologies such as DevSecOps, continuous delivery, containers and micro-services are essential building blocks in the digital business revolution. However, moving information and technologies from hardware to software poses a security concern – translating to a top challenge for both IT and the C-level, as applications built on top of micro-services and containers in a Cloud-Native environment utilize a wide variety of secrets for their proper functioning ...

December 06, 2017

There was a time in cybersecurity strategy when most IT leaders considered perimeter and endpoint guards like antivirus and authentication controls to be the sum of network protection. But as attacks continue to increase in frequency and sophistication, leaders and DevOps teams have been focusing on the role of backup and disaster recovery in mounting a strong defense ...

December 04, 2017

In this blog I will summarize and share with you some wisdom about the biggest problem – okay, problems – in the field of software testing right now. While this is not an exhaustive list, these four bad habits have emerged as the predominant themes ...

December 01, 2017

The majority of testers – 63 percent – are responsible for both API and UI testing, according to the State of Testing 2017 Survey conducted by SmartBear Software. With the growth of methodologies like Agile and DevOps, testing teams have been shrinking and the line between roles increasingly blending ...

November 29, 2017

Companies today face a digital dilemma. How can they understand and discern if their approach to transforming their company to meet today's digital consumer is the right one? ...

November 27, 2017

It has been argued that Dev and Ops teams should work more closely together for some time. For many, the benefits of a closer relationship are clear, and the debate has moved on from if to how, but for lots of companies there are several types of walls to tear down ...

Share this