What Shifting Left and Automation Mean to Mainframe Testing
June 10, 2019

Chris O'Malley
Compuware

Toyota runs one of the most sophisticated and efficient production systems in the world. This is due in large part to a device called an "andon cord" on every production line that gives any team member the authority to halt production when a quality or process problem is found, until it is resolved.

One can hardly imagine Toyota denying those directly responsible for production a means of detecting and flagging defects. But this is exactly what happens in software delivery when mainframe developers don't have the tools to perform early and continuous testing.

Empowering each team member with the ability to identify and resolve defects at any point in the manufacturing process helps Toyota ensure high quality and reliability amidst the introduction of new technological innovations. Similarly, empowering all developers to perform unit testing — the type of testing performed at the earliest point in the software development cycle — can ensure high quality software delivery.

Why Mainframe Testing Must Shift Left and Be Automated

Most innovative web-based applications ultimately depend on finely tuned mainframe code to complete their basic functions. This tuning should begin with individually and independently scrutinizing the smallest part — the unit — of an application for proper operation. Just as Toyota empowers employees to maintain quality and reliability with andon cords, automating unit testing enables development teams to find and fix problems as early as possible by shifting testing left. This prevents defects from being baked into software and becoming more costly and time-consuming to undo later in the "production system."

After unit testing, developers and/or QA testers should be empowered with automated functional testing to test a slice of functionality of a larger system. These tests ensure that the application will do what users expect it to do and ultimately what the business and customers expect the software to accomplish.

Without automation, the overall software release pipeline must decelerate to accommodate the mainframe's manual testing processes. The other option is to sacrifice time and effort spent on testing, leading to the rollout of unstable, unusable products. Shifting testing left enables DevOps teams to move quickly without compromising quality.

Eventually, a lack of essential mainframe testing automation comes to resemble what happens in the famous "I Love Lucy" chocolate factory scene, where one vital step in a process — in Lucy's case, the wrapping of chocolates — cannot keep pace with the others.

Enabling Automated Testing on the Mainframe

Enabling automated testing of mainframe code requires a modern tool that should offer sophisticated features within a preferred developer experience, including the:

■ Automated creation and triggering of tests

■ Automated collection of virtualized test data

■ Identification of overall mainframe code quality trends

■ Sharing and reusing of test assets

■ Creation of repeatable tests

■ Enforcement of testing policies

Code coverage is another critical component of automated testing. Code coverage metrics provide insight into the degree to which source code is executed during a test — that is, which lines of code have or have not been executed, and what percentage of an application has or has not been tested. These measurements allow IT teams to understand the scope and effectiveness of their testing as the code is promoted towards production. 

Like automated testing, code coverage on the mainframe has traditionally been a challenge, since metrics would identify when mainframe code was tested, but not drill down to provide data on the actual percentage/portions of that mainframe code. Using an automated testing tool that integrates this more specific information allows developers to more quickly and accurately spot areas of uncovered code that need attention, just as they do in Java. 

Code quality management across platforms is extremely valuable to mainframe-backed organizations, since their ability to bring new digital deliverables to market is often contingent on simultaneously updating code across both back-end mainframe systems of record and front-end mobile/web systems of engagement. Incorporating mainframe code into cross-platform code quality initiatives brings a higher degree of confidence and rigor to end-to-end testing processes.

Treat the Mainframe Like Your Engine It Is

While new technologies have emerged to address previously unsolvable business problems, the mainframe has endured as the engine propelling large enterprise IT, because no other platform can match its security, availability and scalability in meeting the workload challenges brought on by the mobile explosion.

According to BMC's most recent mainframe user survey, 59 percent of respondents saw increasing mainframe transaction volumes in 2018, while 92 percent predict long-term viability for the platform — the third year in a row of increases and the highest level since 2013.

Recognizing the vital role that mainframe code continues to play, BMC's survey says almost half of mainframe users are now applying DevOps practices to their mainframe environments. But bringing the mainframe up to the speed and pace of DevOps requires mainframe code to be fully included, as early as possible, and across the entire DevOps toolchain. Automating testing is an essential and required step in enabling that shift.

Chris O'Malley is CEO of Compuware
Share this

Industry News

February 29, 2024

ManageEngine, the enterprise IT management division of Zoho Corporation, announced the integration between Endpoint Central, its flagship unified endpoint management solution, and Check Point's Harmony Mobile, a leading mobile threat defense solution, to help IT security teams automate the remediation of mobile threats.

February 29, 2024

Stack Overflow and Google Cloud announced a strategic partnership that will deliver new gen AI-powered capabilities to developers through the Stack Overflow platform, Google Cloud Console, and Gemini for Google Cloud.

February 29, 2024

The Cloud Native Computing Foundation® (CNCF®), which builds sustainable ecosystems for cloud native software, announced the graduation of Falco, a cloud native security tool designed for Linux systems and the de facto Kubernetes threat detection engine.

February 28, 2024

JFrog announced a new technology integration with Qwak, a fully managed ML Platform, that brings machine learning models alongside traditional software development processes to streamline, accelerate, and scale the secure delivery of ML applications.

February 28, 2024

ServiceNow, Hugging Face, and NVIDIA, announced the release of StarCoder2, a family of open‑access large language models (LLMs) for code generation that sets new standards for performance, transparency, and cost‑effectiveness.

February 28, 2024

GMO GlobalSign announced the availability of an Issuer for Kubernetes cert-manager.

February 27, 2024

MacStadium announced the launch of its online community to deepen the connections of application developers through knowledge sharing and collaboration.

February 27, 2024

Octopus Deploy announced the acquisition of Codefresh Inc.

February 26, 2024

Intel announced its new Edge Platform, a modular, open software platform enabling enterprises to develop, deploy, run, secure, and manage edge and AI applications at scale with cloud-like simplicity.

February 26, 2024

Tray.io announced AI-augmented API Management, a new Tray Universal Automation Cloud capability that turns any new or existing workflow into a reusable API, significantly decreasing the technical debt associated with the operational effort and costs of traditional API management (APIM).

February 26, 2024

Bitwarden Secrets Manager is now integrated with Ansible Playbook.

February 22, 2024

Check Point® Software Technologies Ltd. introduces Check Point Quantum Force series: an innovative lineup of ten high-performance firewalls designed to meet and exceed the stringent security demands of enterprise data centers, network perimeters, campuses, and businesses of all dimensions.

February 22, 2024

Tabnine announced that Tabnine Chat — the enterprise-grade, code-centric chat application that allows developers to interact with Tabnine AI models using natural language — is now available to all users.

February 22, 2024

Avaamo released Avaamo LLaMB™, a new low-code framework for building generative AI applications in the enterprise safely, securely, and fast.

February 21, 2024

CAST announced the winter release of CAST Imaging, an imaging system for software applications, with significant user experience (UX) enhancements and new features designed to simplify and accelerate processes for engineers who develop, maintain, modernize, complex software applications.