Test or Automated Check? How to Strategically Implement DevOps Automation
June 25, 2019

Malcolm Isaacs
Micro Focus

DevOps thrives on automation, and it's clear why: manual processes are slow, error-prone, and inconsistent. In practice, automation rates, especially for testing, are very low, so achieving effective test automation can be challenging, but there are steps that you can take to increase test automation in your organization.

Continuous Integration Needs Continuous Testing

Before diving into what you can do to increase testing automation, it's important to understand the intersection of continuous integration and continuous testing as both affect automation implementation.

Continuous Integration (CI) is a DevOps practice whereby code changes are pushed to a common code base many times every day, and are automatically integrated with the rest of the code base. This frequent merging activity carries the risk of destabilizing the code base and introducing defects into the software delivery pipeline.

To find out quickly if a change causes problems, teams use a Continuous Testing (CT) approach, whereby a code change automatically runs a set of tests, designed to provide speedy feedback to the developers.

Checking if a recent change breaks functionality that worked earlier is known as regression testing, and is an essential part of the CI pipeline, triggered whenever a change is committed, which can be many times each day.

In addition to regression testing, CT should check that new functionality works as expected, that it is easy to use, and is resilient to unexpected circumstances. It should also cover aspects such as application performance and security.

Continuous Testing Needs Test Automation

The team must get feedback on their changes as quickly as possible, but with so many tests to run, it is not practical to test manually. The solution is to automate the tests, so that they can run unattended, and complete tests quickly and reliably.

However, automating everything is not always possible. For example, evaluating the user experience of a new feature requires a subjective examination. We need to study and explore the new feature before we can determine if the user experience is satisfactory. Many teams turn to Exploratory Testing, which involves manually playing with the system, learning about it as you go, and applying the knowledge gained to conduct further testing.

Although the goal should be to automate whatever can be automated, there's no magic wand to "automate all the things."

Some of the challenges with test automation are:

■ Applications change frequently, requiring automation scripts to be constantly updated

■ Difficulty of provisioning stable test environments with appropriate test data

■ Lack of skilled automation engineers

Nevertheless, organizations are facing up to these challenges and looking to grow their automation. A word of caution though: Jez Humble and David Farley, in their book Continuous Delivery, recommend a gradual approach to automation, rather than attempting to automate everything at once.

Be Smart About Test Automation

Here are some tips to help you increase your test automation.

1. Start with tests that are small, focused, and run quickly

Tests should be small, independent, self-contained, check a very specific area of the code, and complete within a very short amount of time. These tests, known as unit tests, should be the first tests to run. You can remove dependencies on external or unavailable services and components by emulating them using a service virtualization tool. Once these tests have passed, more extensive testing can be performed.

2. Automate security testing and performance testing early on

Security testing and performance testing are often run late in the cycle, increasing the risk of serious issues being detected when they're more difficult to fix. To get quick feedback, design short load tests that run with each build, and include static application security tests that focus on the code that changed. You can run more comprehensive tests after the initial tests have run, or in parallel. The key is to provide valuable feedback as early as possible.

3). Put developers and testers on the same team, with the same tools

Automation rates increase when testers and developers work together. When they use the same tools and technologies for development and testing, communication becomes more efficient, automation becomes aligned with objectives, and issues can be identified and resolved more quickly. Consider practicing Test Driven Development (TDD), whereby tests are written first, and only then code is written to pass the test. This can reduce ambiguity, and ensure that developers and testers are on the same page.

4. Make your metrics visible to the team through a real-time dashboard

Establish a dashboard showing each build and pipeline, and what stage they are at. Include the status of tests that are running, which ones have passed and failed, and how long they are taking to run. Visibility into the pipelines through simple-to-understand, real-time graphs and charts on a central dashboard provides critical information that helps teams proactively identify and resolve issues.

What's Next for Test Automation?

The assertion that test automation isn't suited for in-depth explorations of a system is increasingly being challenged by artificial intelligence. AI and machine learning (ML) will help computers to "learn" how a webpage behaves, and develop, optimize, and maintain scripts that can test the page and how it interacts and behaves with the rest of the system — all without human intervention.

AI and ML are pushing the boundaries of automation. There will be significant advances over the next few years as the technology matures, blurring the lines between automated checking and intelligent testing.

Malcolm Isaacs is Senior Solutions Manager, Application Delivery Management, at Micro Focus
Share this

Industry News

July 30, 2020

New Relic delivered strategic updates to New Relic One.

July 30, 2020

IT Revolution announced the DevOps Enterprise Summit Las Vegas 2020 will be going virtual.

July 30, 2020

Adaptavist announced the acquisition of Go2Group, a US technology firm specializing in Agile and DevOps services and cloud solutions for the enterprise.

July 29, 2020

Panaya announced a new partnership with Worksoft providing SAP IT organizations with a best in class Change Intelligence solution that enables SAP ECC users to migrate or optimize their system risk-free.

July 29, 2020

Splice Machine launched the Splice Machine Kubernetes Ops Center, deployed with Helm Charts.

July 29, 2020

CirrusHQ, an Amazon Web Services (AWS) Advanced Consulting and Solution Provider partner, has achieved AWS DevOps Competency Status.

July 28, 2020

NetSPI launched Static Application Security Testing (SAST) and Secure Code Review (SCR) services.

July 28, 2020

Centrify debuted Delegated Machine Credentials (DMC) as part of the Centrify Privileged Access Service to reduce risk and empower automation in increasingly complex, infrastructure-as-code-based elastic environments.

July 28, 2020

Tricentis announced an expansion of its strategic partnership with SAP.

July 27, 2020

Puppet announced the release of an open source Scenario Planning Toolkit for enterprise technology leaders to make more informed, strategic decisions and meaningfully drive forward their businesses during uncertain times.

July 27, 2020

CloudPassage announced expanded container-related security capabilities for its Halo cloud security platform.

July 27, 2020

The latest release of Sysdig Secure DevOps Platform provides a 5-minute setup, a fast path to delivering container and Kubernetes security and visibility with a SaaS-first offering.

July 23, 2020

ACCELQ and its automation testing suite have been recognized by leading research firm, Forrester.

July 23, 2020

Puppet announced updates to Puppet Enterprise making it easier and faster to automate complex infrastructure operations for improved application deployments, fewer errors and disruptions, and greater operational repeatability and efficiency.

July 23, 2020

Tasktop announced that its latest release of the Tasktop Value Stream Management platform supports Jira Align.