Mastering Continuous Testing
A Q&A with Perfecto Founder and CTO
February 11, 2019

Eran Kinsbruner
Perfecto

In today's world, our lives very much revolve around digital technologies. Consumers rely on their devices and apps for everything from transferring money to checking in with friends to recording updates about their health. With so many people relying so heavily on apps, it is critical for developers to assure continuous quality throughout the entire software development life cycle (SDLC). And it all starts with testing.

It's no secret that the majority of organizations struggle with implementing continuous testing (CT) strategies. In fact, one of the biggest stumbling blocks for implementing DevOps is the lack of CT. To help organizations harness the power of CT, I recently sat down with Perfecto's Founder and CTO, Yoram Mizrachi, to discuss the benefits of CT, the roadblocks teams need to overcome and what is ahead for the future of testing and software development.

Q: How do you define continuous testing?

Continuous testing (CT) is the ability to embed testing as part of the normal DevOps process and, at a minimum, it should happen daily. Non-CT means there are different rhythms for development and testing, which creates conflicts, sync errors and could result in wasted resources and time.

If you have a business case for DevOps in your organization, CT is a must-have and not a nice to have. CT is an enabler for DevOps practices and in today's testing landscape, you can't really run an Agile/DevOps practice if your testing is not continuous.

Q: What is driving the adoption of CT?

Like any other element in the DevOps cycle, everything should be automated and non-attended. DevOps is built on the practice that the machine and processes are keeping the rhythm. This means that, at any moment in time, teams have full visibility into where they are in the process. Day 1 of the sprint is no different than day n-1 of the sprint. The process of automated continuous integration/testing/delivery ensures this is the case.

Q: Who benefits the most from CT?

In DevOps practices, testing is fully owned by the feature team. This team is developing, building, testing and releasing continuously. There are teams that assign testing to developers while others are assigning testing to testers. Both are valid practices and result in the feature team leader being able to avoid surprises and make informed decisions based on facts.

Group leaders, dev leaders and business groups managers also benefit from CT and should actively promote the use of it since it aligns the overall goals for DevOps practices. CT gives these groups much better visibility into the development status during the cycle rather than at the end of the cycle.

Q: What roadblocks are associated with the adoption of DevOps and CT?

 

The biggest roadblock to CT is establishing trust. For decades, automated testing suffered from a lack of trust as part of the process. This stemmed from the reliability of test results, flakiness and ongoing maintenance. Getting to a point where red means red and green means green is essential for fully adopting – and embracing – CT.

In addition, teams will also need to overcome some of the initial challenges that come with automation. For example, selecting what to automate, which platforms to automate against and how to sustain automation as the market shifts and the product evolves.

Q: What advice would you give to organizations who think they can't engage in continuous testing?

DevOps isn't for every team – it's dependent on skillset, organizational structure and business demands. That being said, DevOps practices are becoming a prominent trend in development, and businesses that do not adopt and practice DevOps will likely fall behind competition in regards to people, skillset and business goals. Unless organizations have really specific or unique needs, there is no such thing as "can't engage." Today, when it comes to CT, it's a question of "how" and not "if."

Q: Where should organizations start when it comes to continuous testing?

Start small and focus on building a working practice that you can trust. Organizations who are starting out with CT will not see success if they create hundreds of scripts and dozens of platforms from the get-go. Instead, they should start with 20-25 scripts, about 4-5 platforms nightly and scale slowly from there.

Q: What is ahead for the future of testing and software development?

Cloud-based testing labs will become a standard practice, just like Amazon Web Services (AWS), Azure and GCP are standard for IT. The movement in code to higher level building blocks – from native to HTML to react – will also affect testing and we will see more packaged testing infrastructure that supports easier building of testing.

Because of the nature of applications, we will also see artificial intelligence (AI) being used more frequently in testing as teams work to overcome unknowns and unexpected elements during the testing period.

Finally, we will see more supporting analysis frameworks, likely based on AI/machine learning, which will guide developers and testers on what should be fixed first.

Eran Kinsbruner is Chief Evangelist at Perfecto
Share this

Industry News

August 06, 2020

Push Technology announced the launch of a new Kafka Adapter for their Diffusion Intelligent Data Mesh.

August 06, 2020

Appvia announced the launch of its Cost Prediction and Visibility tool, integrated within the latest version of its Kore platform.

August 06, 2020

LogiGear announced the newest addition to the TestArchitect™ family, TestArchitect Gondola.

August 05, 2020

Logz.io announced a partnership with HashiCorp, a provider in multi-cloud infrastructure automation software.

August 05, 2020

Digitate, a software venture of Tata Consultancy Services, announced the release of ignio™ AI.Assurance, an autonomous assurance product that enables enterprises to deliver better software faster, enhancing their business performance.

August 05, 2020

Harness acquired self-service Continuous Integration firm Drone.io, the creator of the open-source project Drone.

August 04, 2020

Aqua Security announced that its Cloud Native Security Platform is available through Red Hat® Marketplace, an open cloud marketplace that makes it easier to discover and access certified software for container-based environments across the hybrid cloud.

August 04, 2020

Threat Stack announced the availability of Threat Stack Container Security Monitoring for AWS Fargate.

August 04, 2020

OpenLogic by Perforce now provides an enterprise-class alternative to Oracle Java by offering OpenJDK distributions backed by OpenLogic support.

August 03, 2020

MuseDev launched on Github Marketplace the Early Access version of its code analysis platform, Muse, to help developers find and fix critical security, performance, and reliability bugs, efficiently, before they reach QA or production.

August 03, 2020

Styra announced Rego Policy Builder for the Styra Declarative Authorization Service (DAS).

August 03, 2020

Felicis Ventures has invested an additional $5M in Sourcegraph, bringing the total raised to over $46M, including a $23M Series B in March 2020 led by Craft Ventures.

July 30, 2020

New Relic delivered strategic updates to New Relic One.

July 30, 2020

IT Revolution announced the DevOps Enterprise Summit Las Vegas 2020 will be going virtual.

July 30, 2020

Adaptavist announced the acquisition of Go2Group, a US technology firm specializing in Agile and DevOps services and cloud solutions for the enterprise.