Beginning Your Test Automation Journey
Part 1 of a three-part series on introducing and building test automation into your application development and deployment pipeline
April 01, 2019

Drew Horn
Applause

Many companies are embracing test automation in an effort to move quicker and more efficiently, while simultaneously saving on costs. Test automation continues to experience growth year over year. In fact, in 2017 KPMG found an 85 percent increase in test automation in a two-year period across all industry domains.

Automation is a tool that all companies should prioritize when operating in an Agile development environment. However, while one of the main goals of test automation is to move faster, maturing test automation within a deployment pipeline should be a progressive, iterative process. If you try to automate too much, or automate the wrong things, you can end up slowing down and even hurting development and testing efforts.

The success you have will hinge on how you set up your automation practice, so the more attention you pay to the early stages, the more benefits you will realize in the long run.

In your journey to mature test automation within your deployment pipeline, you will move through three distinct stages: beginner, intermediate, and expert. We will explore the beginner stage in part one of this series.

Taking the First Step

At its core, automation is all about doing things better, faster, and cheaper. It improves coverage by increasing the number of tests you can run and improves test quality through repeatability. However, you need to crawl before you can walk and eventually run, so, like most things, it is advisable to start small with your test automation efforts.

The beginner stage of test automation starts with assessing the maturity of your testing organization and defining a goal of where you want to go. You’ll also need to develop a (or integrate an existing) framework on which to run your unit, integration, and functional tests. You can assess your testing maturity by self-evaluating across five key criteria:

1. Team – What is the team makeup? Where are the expertise gaps? Are QA and Dev teams working in silos?

2. Technology – What does the deployment pipeline look like? How are automated tests triaged? How is automation integrated with a VSC/TCM/BTS?

3. Process – When are tests written? How are bugs triaged and tests updated? How fast are sprint cycles? How does feedback guide test strategy?

4. Reporting – What is the test coverage? What are the common devices used? How are test results viewed? How is a "go/no-go" decision made? What is the automation ROI?

5. Pain – What are the existing pain points? What bugs have been missed?

Once you’ve assessed your maturity and developed a framework, you will want to focus on one main thing: Creating quick wins for your team. This will help to build confidence in your automation practice and get the full weight of the team involved.

Your development team can begin by checking in unit tests to immediately receive pass/fail feedback. After seeing these unit tests pass on a consistent basis, the team should then begin to prioritize developing a core set of functional smoke tests.

How often a set of smoke tests should be run initially will depend on your team's experience and bandwidth. A good rule of thumb is to start with a nightly smoke test that you can review each morning. This allows you to focus on code quality, test reliability, and process as opposed to performance and optimization. Eventually, though, you will want these automated tests up and running as part of your overarching build efforts so they are automatically triggered upon a successful build.

Steady iteration, where success is built on success, will help you gain valuable experience with your automation. Once you see your early unit and smoke tests passing consistently, you can consider adding more smoke tests to your automation suite. In general, you will want to keep the runtime of your tests to 15 minutes or less. As you advance your automation practice, you will find additional ways to run more smoke tests in a 15-minute period, such as test parallelization. We'll cover these strategies in the next article.

Though it’s not the most exciting part of test automation, setting up a documentation process is an important step in building a successful automation practice. The entire team should agree on a definition of "done" for automation tasks that need to be written out during sprint planning. While developing and documenting your automation processes, you should also look to answer the following questions: When should tests be run initially? How is the team notified when tests fail? Who is responsible for fixing failed tests? What is the action for triaging failures and logging bugs? Document the answers to the questions so everyone understands the protocol, otherwise you can expect to run into problems in the future as you scale up the speed and breadth of your practice.

How long you spend in the beginner stage of your automation journey will depend on a number of factors. In general, the hardest part of this stage is just getting the ball rolling. Try to set a goal of about eight weeks to stand up your initial test automation practice. This is a good pace that will all you to keep momentum. When time constraints, lack of prior experience, and competing priorities are large factors, consider pulling in turn-key solutions to get up and running quickly. Remember that your strategy can always be adjusted later. It's much easier make tweaks with an existing practice in place that can provide value to the team almost immediately.

Read Part 2: Building Confidence in Automation.

Drew Horn is Senior Director of Automation at Applause
Share this

Industry News

April 11, 2024

Check Point® Software Technologies Ltd. announced new email security features that enhance its Check Point Harmony Email & Collaboration portfolio: Patented unified quarantine, DMARC monitoring, archiving, and Smart Banners.

April 11, 2024

Automation Anywhere announced an expanded partnership with Google Cloud to leverage the combined power of generative AI and its own specialized, generative AI automation models to give companies a powerful solution to optimize and transform their business.

April 11, 2024

Jetic announced the release of Jetlets, a low-code and no-code block template, that allows users to easily build any technically advanced integration use case, typically not covered by alternative integration platforms.

April 10, 2024

Progress announced new powerful capabilities and enhancements in the latest release of Progress® Sitefinity®.

April 10, 2024

Buildkite signed a multi-year strategic collaboration agreement (SCA) with Amazon Web Services (AWS), the world's most comprehensive and broadly adopted cloud, to accelerate delivery of cloud-native applications across multiple industries, including digital native, financial services, retail or any enterprise undergoing digital transformation.

April 10, 2024

AppViewX announced new functionality in the AppViewX CERT+ certificate lifecycle management automation product that helps organizations prepare for Google’s proposed 90-day TLS certificate validity policy.

April 09, 2024

Rocket Software is addressing the growing demand for integrated security, compliance, and automation in software development with its latest release of Rocket® DevOps, formerly known as Aldon®.

April 09, 2024

Wind River announced the latest release of Wind River Studio Developer, an edge-to-cloud DevSecOps platform that accelerates development, deployment, and operation of mission-critical systems.

April 09, 2024

appCD announced its generative infrastructure from code solution now supports Azure Kubernetes Service (AKS).

April 09, 2024

Synopsys announced the availability of Black Duck® Supply Chain Edition, a new software composition analysis (SCA) offering that enables organizations to mitigate upstream risk in their software supply chains.

April 09, 2024

DataStax announced innovative integrations with API extensions to Google Cloud’s Vertex AI Extension and Vertex AI Search, offering developers an easier time leveraging their own data.

April 08, 2024

Parasoft introduced C/C++test CT, a comprehensive solution tailored for large teams engaged in the development of safety- and security-critical C and C++ products.

April 08, 2024

Endor Labs announced a strategic partnership with GuidePoint Security.

April 08, 2024

Hasura announced the V3 of its platform, providing on-demand API composability with a new domain-centric supergraph modeling framework, a distributed supergraph execution engine and a rich and extensible ecosystem of open source connectors to address the challenges faced during integration of data and APIs.

April 04, 2024

DataStax has entered into a definitive agreement to acquire AI startup, Logspace, the creators of Langflow, an open source visual framework for building retrieval-augmented generation (RAG) applications.1