What Shifting Left and Automation Mean to Mainframe Testing
June 10, 2019

Chris O'Malley
Compuware

Toyota runs one of the most sophisticated and efficient production systems in the world. This is due in large part to a device called an "andon cord" on every production line that gives any team member the authority to halt production when a quality or process problem is found, until it is resolved.

One can hardly imagine Toyota denying those directly responsible for production a means of detecting and flagging defects. But this is exactly what happens in software delivery when mainframe developers don't have the tools to perform early and continuous testing.

Empowering each team member with the ability to identify and resolve defects at any point in the manufacturing process helps Toyota ensure high quality and reliability amidst the introduction of new technological innovations. Similarly, empowering all developers to perform unit testing — the type of testing performed at the earliest point in the software development cycle — can ensure high quality software delivery.

Why Mainframe Testing Must Shift Left and Be Automated

Most innovative web-based applications ultimately depend on finely tuned mainframe code to complete their basic functions. This tuning should begin with individually and independently scrutinizing the smallest part — the unit — of an application for proper operation. Just as Toyota empowers employees to maintain quality and reliability with andon cords, automating unit testing enables development teams to find and fix problems as early as possible by shifting testing left. This prevents defects from being baked into software and becoming more costly and time-consuming to undo later in the "production system."

After unit testing, developers and/or QA testers should be empowered with automated functional testing to test a slice of functionality of a larger system. These tests ensure that the application will do what users expect it to do and ultimately what the business and customers expect the software to accomplish.

Without automation, the overall software release pipeline must decelerate to accommodate the mainframe's manual testing processes. The other option is to sacrifice time and effort spent on testing, leading to the rollout of unstable, unusable products. Shifting testing left enables DevOps teams to move quickly without compromising quality.

Eventually, a lack of essential mainframe testing automation comes to resemble what happens in the famous "I Love Lucy" chocolate factory scene, where one vital step in a process — in Lucy's case, the wrapping of chocolates — cannot keep pace with the others.

Enabling Automated Testing on the Mainframe

Enabling automated testing of mainframe code requires a modern tool that should offer sophisticated features within a preferred developer experience, including the:

■ Automated creation and triggering of tests

■ Automated collection of virtualized test data

■ Identification of overall mainframe code quality trends

■ Sharing and reusing of test assets

■ Creation of repeatable tests

■ Enforcement of testing policies

Code coverage is another critical component of automated testing. Code coverage metrics provide insight into the degree to which source code is executed during a test — that is, which lines of code have or have not been executed, and what percentage of an application has or has not been tested. These measurements allow IT teams to understand the scope and effectiveness of their testing as the code is promoted towards production. 

Like automated testing, code coverage on the mainframe has traditionally been a challenge, since metrics would identify when mainframe code was tested, but not drill down to provide data on the actual percentage/portions of that mainframe code. Using an automated testing tool that integrates this more specific information allows developers to more quickly and accurately spot areas of uncovered code that need attention, just as they do in Java. 

Code quality management across platforms is extremely valuable to mainframe-backed organizations, since their ability to bring new digital deliverables to market is often contingent on simultaneously updating code across both back-end mainframe systems of record and front-end mobile/web systems of engagement. Incorporating mainframe code into cross-platform code quality initiatives brings a higher degree of confidence and rigor to end-to-end testing processes.

Treat the Mainframe Like Your Engine It Is

While new technologies have emerged to address previously unsolvable business problems, the mainframe has endured as the engine propelling large enterprise IT, because no other platform can match its security, availability and scalability in meeting the workload challenges brought on by the mobile explosion.

According to BMC's most recent mainframe user survey, 59 percent of respondents saw increasing mainframe transaction volumes in 2018, while 92 percent predict long-term viability for the platform — the third year in a row of increases and the highest level since 2013.

Recognizing the vital role that mainframe code continues to play, BMC's survey says almost half of mainframe users are now applying DevOps practices to their mainframe environments. But bringing the mainframe up to the speed and pace of DevOps requires mainframe code to be fully included, as early as possible, and across the entire DevOps toolchain. Automating testing is an essential and required step in enabling that shift.

Chris O'Malley is CEO of Compuware
Share this

Industry News

December 05, 2019

Parasoft announced the newest release of Parasoft C/C++test, the unified C and C++ development testing solution for enterprise and embedded applications.

December 05, 2019

Datadog announced Security Monitoring, a new product that enables real-time threat detection across the entire stack and deeper collaboration between security, developers, and operations teams.

December 05, 2019

Pulumi announced the availability of Pulumi Crosswalk for Kubernetes, an open source collection of frameworks, tools and user guides that help developers and operators work better together delivering production workloads using Kubernetes.

December 04, 2019

CloudBees announced a Preview Program for CloudBees CI/CD powered by Jenkins X, a Software as a Service (SaaS) continuous integration and continuous delivery solution running on Google Cloud Platform.

December 04, 2019

Rancher Labs announced the general availability of K3s, their lightweight, certified Kubernetes distribution purpose built for small footprint workloads, along with the beta release of Rio, their new application deployment engine for Kubernetes that delivers a fully integrated deployment experience from operations to pipeline.

December 04, 2019

WhiteSource announced a new integration with Codefresh, the Kubernetes-native CI/CD solution.

December 03, 2019

Styra is addressing one of the most significant enterprise blockers of Kubernetes: compliance. With Styra, enterprises can move Kubernetes clusters into production en masse while complying with traditional governance, audit, and compliance rules and regulations.

December 03, 2019

Nureva added 13 agile-themed templates to Span Workspace, Nureva’s expansive cloud-based digital canvas for visual planning and team collaboration.

December 03, 2019

Threat Stack announced support for AWS Fargate in the Threat Stack Cloud Security Platform.

December 02, 2019

Tricentis announced the publication of Enterprise Continuous Testing: Transforming Testing for Agile and DevOps, written by Tricentis Founder Wolfgang Platz and Cynthia Dunlop.

December 02, 2019

JFrog announced the availability of the popular JFrog Platform subscription package Cloud Pro X on AWS Marketplace.

December 02, 2019

MuleSoft will extend its Anypoint Runtime Fabric to run on Google Cloud.

November 26, 2019

NeuVector announced the immediate availability of its “Security Policy as Code” capability for Kubernetes services.

November 26, 2019

Agile Stacks announced the launch of KubeFlex, a new cloud-native software platform enabling zero-touch Kubernetes deployments in data centers and at the edge.

November 26, 2019

Bacula Systems announced significant enhancements to its backup module for Kubernetes clusters.