1700 DevOps Monitoring Experts Agree: Too Many Alerts from Too Many Tools Put Customers at Risk
May 18, 2016

Dan Turchin
BigPanda

We're all technology companies. Every second of downtime hurts. Monitoring at scale is hard. And that's just the beginning of what you shared in our recent survey.

We invited you to tell us about the state of monitoring. Tales of woe and glory from more than 1,700 ops experts provided the most articulate, profound, comprehensive summary of IT Ops life ever assembled.

We thought you'd all benefit from what you shared so we published the results. You represent five continents, large and small companies (modal reply: more than 10,000 employees), large and small teams (modal reply: less than 10 members), and both traditional IT and DevOps organizations.

Here's what fascinated me...

You rely on many tools to monitor your infrastructure.

■ Each team member is responsible for triaging between 10 and 50 alerts per day.

■ In an eight-hour shift, that means you're each working about 10 issues simultaneously assuming you don't inherit orphans from previous shifts (which you do!).

■ Translation: there are fire-swallowing, tightrope-walking, lion tamers working the e.coli route for Carnival Cruise Line with easier jobs than yours.

The more you've invested in agility and velocity, the more effective you are at reducing downtime.

■ Self-described "DevOps" organizations are more than twice as likely to deploy code and/or infrastructure changes at least a few times per day (31% for DevOps orgs vs. 15% overall).

■ They're also more than twice as likely to have cloud-based infrastructure (32% of DevOps orgs vs. 13% overall).

You're dissatisfied with the current reliability of your monitoring and incident management process.

■ Nearly 80% of you say the most challenging part of your job is suppressing alert noise.

■ The problem's not going away: more than 55% are dissatisfied with the current monitoring strategy. Your comments also indicate the problem won't improve in the next 12 months without a better way to manage the growing workload.

A bleak picture perhaps best summarized by Carlos from a midwestern credit union who says if he could change one thing about his organization's current monitoring strategy it would be "to focus on the only thing that matters: reducing noise." Carlos, you're right. Human beings alone can't fix a problem created by machines. We've been in this position before … before there was client-server, TCP, DNS, virtualization, cloud.

We've approached each challenge with the same tenacity, the same passion, the same commitment to solving problems with technology. We'll do it again. This time, with better automation and collaboration. Soon, machines and people will speak a common language. And when they do, we'll be the first to share how great technology plus your ingenuity makes life better for everyone.

Dan Turchin is VP Product at BigPanda.

The Latest

November 16, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 7, the final installment, covers IT Operations tools ...

November 15, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 6 covers DevOps and development tools ...

November 14, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 4 covers communication ...

November 13, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 4 covers more about combining Dev and Ops in teams ...

November 09, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 3 covers how to set up teams ...

November 08, 2017

Mainframes may be legacy, but they still run a lot of business. With so much in play, businesses run a great risk when practices for maintaining and developing on mainframe remain largely the same despite the rest of the organization undergoing significant change to keep pace with the latest DevOps trends ...

November 07, 2017

DEVOPSdigest asked experts from across the industry for their opinions on the best way to foster collaboration between Dev and Ops. Part 2 covers the personal interaction between Development and Operations ...

November 06, 2017

When you say "DevOps" one of the first words that comes to mind is "collaboration." But exactly how do we make this collaboration happen? This epic DEVOPSdigest list – to be posted in 7 parts over the next few weeks – intends to find the answer. DEVOPSdigest asked experts from across the industry – including consultants, analysts, organizations, users and the leading vendors – for their opinions on the best way to foster collaboration between Dev and Ops. Part 1 offers recommendations on how to get started with collaboration ...

November 03, 2017

It's 3:47am. You and the rest of the Ops team have been summoned from your peaceful slumber to mitigate an application delivery outage. Your mind races as you switch to problem solving mode. It's time to start thinking about how to make this mitigation FUN! ...

November 01, 2017

Throughout the software development and delivery process, each team plays a pivotal role in ensuring that the end result is exactly what is needed. However, all too often the software testing team, who come in towards the end of the cycle, is labeled as a bottleneck that slows down the process. But it is always important to see both sides of an argument and understand the pressures put on the testing team ...

Share this