Test or Automated Check? How to Strategically Implement DevOps Automation
June 25, 2019

Malcolm Isaacs
Micro Focus

DevOps thrives on automation, and it's clear why: manual processes are slow, error-prone, and inconsistent. In practice, automation rates, especially for testing, are very low, so achieving effective test automation can be challenging, but there are steps that you can take to increase test automation in your organization.

Continuous Integration Needs Continuous Testing

Before diving into what you can do to increase testing automation, it's important to understand the intersection of continuous integration and continuous testing as both affect automation implementation.

Continuous Integration (CI) is a DevOps practice whereby code changes are pushed to a common code base many times every day, and are automatically integrated with the rest of the code base. This frequent merging activity carries the risk of destabilizing the code base and introducing defects into the software delivery pipeline.

To find out quickly if a change causes problems, teams use a Continuous Testing (CT) approach, whereby a code change automatically runs a set of tests, designed to provide speedy feedback to the developers.

Checking if a recent change breaks functionality that worked earlier is known as regression testing, and is an essential part of the CI pipeline, triggered whenever a change is committed, which can be many times each day.

In addition to regression testing, CT should check that new functionality works as expected, that it is easy to use, and is resilient to unexpected circumstances. It should also cover aspects such as application performance and security.

Continuous Testing Needs Test Automation

The team must get feedback on their changes as quickly as possible, but with so many tests to run, it is not practical to test manually. The solution is to automate the tests, so that they can run unattended, and complete tests quickly and reliably.

However, automating everything is not always possible. For example, evaluating the user experience of a new feature requires a subjective examination. We need to study and explore the new feature before we can determine if the user experience is satisfactory. Many teams turn to Exploratory Testing, which involves manually playing with the system, learning about it as you go, and applying the knowledge gained to conduct further testing.

Although the goal should be to automate whatever can be automated, there's no magic wand to "automate all the things."

Some of the challenges with test automation are:

■ Applications change frequently, requiring automation scripts to be constantly updated

■ Difficulty of provisioning stable test environments with appropriate test data

■ Lack of skilled automation engineers

Nevertheless, organizations are facing up to these challenges and looking to grow their automation. A word of caution though: Jez Humble and David Farley, in their book Continuous Delivery, recommend a gradual approach to automation, rather than attempting to automate everything at once.

Be Smart About Test Automation

Here are some tips to help you increase your test automation.

1. Start with tests that are small, focused, and run quickly

Tests should be small, independent, self-contained, check a very specific area of the code, and complete within a very short amount of time. These tests, known as unit tests, should be the first tests to run. You can remove dependencies on external or unavailable services and components by emulating them using a service virtualization tool. Once these tests have passed, more extensive testing can be performed.

2. Automate security testing and performance testing early on

Security testing and performance testing are often run late in the cycle, increasing the risk of serious issues being detected when they're more difficult to fix. To get quick feedback, design short load tests that run with each build, and include static application security tests that focus on the code that changed. You can run more comprehensive tests after the initial tests have run, or in parallel. The key is to provide valuable feedback as early as possible.

3). Put developers and testers on the same team, with the same tools

Automation rates increase when testers and developers work together. When they use the same tools and technologies for development and testing, communication becomes more efficient, automation becomes aligned with objectives, and issues can be identified and resolved more quickly. Consider practicing Test Driven Development (TDD), whereby tests are written first, and only then code is written to pass the test. This can reduce ambiguity, and ensure that developers and testers are on the same page.

4. Make your metrics visible to the team through a real-time dashboard

Establish a dashboard showing each build and pipeline, and what stage they are at. Include the status of tests that are running, which ones have passed and failed, and how long they are taking to run. Visibility into the pipelines through simple-to-understand, real-time graphs and charts on a central dashboard provides critical information that helps teams proactively identify and resolve issues.

What's Next for Test Automation?

The assertion that test automation isn't suited for in-depth explorations of a system is increasingly being challenged by artificial intelligence. AI and machine learning (ML) will help computers to "learn" how a webpage behaves, and develop, optimize, and maintain scripts that can test the page and how it interacts and behaves with the rest of the system — all without human intervention.

AI and ML are pushing the boundaries of automation. There will be significant advances over the next few years as the technology matures, blurring the lines between automated checking and intelligent testing.

Malcolm Isaacs is Senior Solutions Manager, Application Delivery Management, at Micro Focus
Share this

Industry News

June 01, 2020

IT Revolution announced a full conference agenda for DevOps Enterprise Summit London, June 23-25, 2020.

June 01, 2020

Caltech CTME announced that Simplilearn, a global provider of digital skills training, will collaborate with CTME (Caltech's Center for Technology and Management Education) to offer a specialized Post Graduate Program in DevOps software engineering.

June 01, 2020

DevOps Institute, a global member-based association for advancing the human elements of DevOps, announced the introduction of its SKILup Playbook Library, a dynamic collective body of knowledge (cBok) that aligns thought leadership from industry experts with a set of dynamic, orchestrated artifacts, research and assets.

May 28, 2020

Docker has extended its strategic collaboration with Microsoft to simplify code to cloud application development for developers and development teams by more closely integrating with Azure Container Instances (ACI).

May 28, 2020

Eggplant announced updates to its Digital Automation Intelligence (DAI) platform.

May 28, 2020

Aptum launched its Managed DevOps Service in partnership with CloudOps, a cloud consulting and professional services company specializing in DevOps.

May 27, 2020

Red Hat announced an expansion of its application services portfolio with the addition of Quarkus as a fully supported framework in Red Hat Runtimes.

May 27, 2020

Couchbase has completed a $105 million all-equity Series G round of fundraising.

May 27, 2020

Aqua Security closed a Series D round of $30M led by Greenspring Associates.

May 26, 2020

GitLab is releasing 13.0 of its DevSecOps platform to enable organizations to efficiently adapt and respond to new and dynamic business challenges.

May 26, 2020

Solo.io announced the availability of the Istio Developer Portal to streamline the developer onboarding process for improved developer experience and increased productivity with added security features.

May 26, 2020

WhiteHat Security will offer free application scanning services to any education institution to support secure online learning.

May 21, 2020

Exadel announced the Grand Prize winner of the “Appery.io COVID-19 Virtual Hackathon.”

May 21, 2020

CloudBees announced significant advances for its Software Delivery Management (SDM) platform – integrations with additional continuous integration and continuous delivery (CI/CD) engines, including Google Cloud Build and Tekton, and extension of the availability of CloudBees’ SDM Preview Program.

May 21, 2020

OutSystems is announcing over 70 development accelerators that ensure web and mobile applications created on the OutSystems low-code development platform can comply with the highest accessibility standards and regulations.