DEVOPSdigest

Most businesses find it challenging to adapt their quality assurance and testing process to the agile way of working, according to The Continuous Testing Report, conducted by Capgemini and Sogeti, in collaboration with Broadcom Inc.'s Enterprise Software Division.

The crux of the challenge is that organizations find it difficult to frequently deploy a large number of releases faster into production, while also implementing an adequate, continuous, and fast validation process which prevents serious issues in production. In the absence of this balance, business performance and growth are at risk.

Without full adoption of Continuous Testing, businesses will reach a point where they will be unable to meet customer needs

The report indicates that without full adoption of Continuous Testing, businesses will reach a point where they will be unable to meet customer needs, making them vulnerable to more successful agile competitors.

While a majority (55%) of the enterprise surveyed have now adopted a Continuous Testing approach, its slow increase in maturity (compared to last year) demonstrates a critical challenge for organizations to overcome.

Up to 56% of the organizations admitted they have challenges with in-sprint testing. Respondents said their teams spend 44% of their time searching, managing and generating test data, while 36% stated that their teams spend more than half their time building and managing test environments. Most respondents (62%) said they are struggling to find skilled professionals to build their Continuous Testing strategy and a third said developing skills in testing AI systems was a priority.

These factors are compounded by the issue of larger teams being held back by legacy systems, applications and hierarchies which can make applying new ways of working more challenging. To overcome these challenges, companies must focus on embracing orchestration of quality engineering in Agile and DevOps.

"Continuous Testing is a critical element for gaining competitive advantage in an environment where companies must deliver products faster and faster to market in order to remain relevant. Organizations must accelerate their investment in quality engineering skills and continuous test solutions within their agile and DevOps teams to ensure that agile at scale does not fail," said Mark Buenen, Global Leader of Digital Assurance and Quality engineering, Capgemini Group. "To achieve this, they must empower feature teams with sufficient quality engineering expertise and enable the QA culture, QA automation and test environment provisioning with a flexible quality support team."

The report highlights a number of areas in which businesses can improve their approach to Continuous Testing:

Creating visibility over quality levels and meaningful KPIs

More than three-quarters (78%) of respondents said that "getting visibility throughout the development lifecycle" is a challenge when implementing Continuous Testing. The report suggests that the entire software development lifecycle needs to be brought together in a single source of truth, from release management through to deployment, with integrated tooling, quality checks, and metrics, to meet business needs.

Leveraging more intelligent solutions

According to the report, teams need to make more use of intelligent solutions to ensure they are selecting the right test cases and validating correctly. At present only 42% make use of artificial intelligence (AI) for predictive analytics, just 36% are deploying code coverage and 39% using analytics from operations.

Investment in Quality Assurance skills

To leverage those intelligent solutions, businesses need to invest in new skills, including knowledge of business processes, automation, data analysis and machine learning. Most respondents (62%) said they are struggling to find skilled professionals to build their Continuous Testing strategy. A third said developing skills in testing AI systems was a priority.

Test organization and environments

36% of respondents stated that they spend over half their time managing test environments – the same proportion as last year. Companies need to take a different approach, cites the report, building test environments that can be spun up, replicated, decommissioned, and managed at scale. This will involve practices included cloud provisioning (currently used by 53% of respondents), service virtualization (45%), and containerization (37%).

"Continuous quality is critical for Agile, DevOps, and Digital Transformation. Besides making test automation a priority, organizations also need to think of embedding quality into every phase of their software development lifecycle. This requires modern, developer-friendly, AI-powered tools that make continuous quality easy to adopt and practice for every stakeholder and every team— from business to technical users. Teams need to overcome traditional barriers to quality at scale with tools that enable shift left and shift right, and leverage AI to provide proactive, actionable insights to maximize quality," said Sushil Kumar, Head of DevOps and Continuous Testing Business, Enterprise Software Division, Broadcom.

Report methodology: The Continuous Testing Report 2020 brings together survey data and subject matter expert contributions to outline the challenges and potential approaches to transforming test practices in the age of Agile and DevOps. It is based on the considered opinions of several subject matter experts from Capgemini, Sogeti, and Broadcom, supported by the results of a global survey of 500 senior decision-makers in corporate IT functions, working for companies and public-sector organizations across eight different regions.