Battle Plan 2018: Illuminate Blind Spots and Unknown Unknowns
December 27, 2017

Josh Gray
Cedexis

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.

Bonus points if you know who came up with that tongue twister. He was talking about terrorists, but we're here to discuss a different sort of war — the Battle for Bandwidth. These days, application and content delivery requires special tactics, an integrated strategy, and well-sourced intelligence. And the unknown unknowns are the true enemy because they inevitably lead to outages, slowdowns, and mutinous customers.

In early November, a major outage caused by a minor configuration error (a route leak, to be exact) at global backbone provider Level 3 created widespread connection issues on both U.S. coasts. Comcast, Verizon, Cox, and Vonage customers were particularly affected.

One small error can have mighty ripple effects, and the cause isn't always apparent to network admins and enterprise customers. The time it took to return the Down Detector maps from angry red to mellow yellow could have been shortened by looking at Real User Measurements (crowdsourced telemetry), realizing it wasn't a single site or ISP, and following a logic tree to find the culprit.

With Global Server Load Balancing, your delivery network is smart enough to see the barricade around the corner and switch routes on the fly — saving the day (and making the other guys look a bit dazed and confused).

Blind spots can be hiding more than outages. Your crack team of DevOps commandos can't run successful release missions if they can't check what's really going on in the field. You don't want them dashing around in the dark without a robust tactical plan based on all the parameters you can assess — when you turn unknown unknowns into known knowns from your various data streams, you can put them to work.

Continuous deployment isn't for the faint of heart — you better have your Kevlar and your night vision goggles. Companies like Salesforce are releasing updates dozens of times a day; but even a handful a week requires a careful strategy. You can use RUM to test an update by initially limiting roll-out to one data center. Check for 40x/50x errors. If you're seeing problems, you can check both user experience with your app (non-updated versions) in other places, and user experience at the same data center where you are testing the updated version, to deduce the source of trouble.

One of the biggest unknown unknowns in traffic management is what's going on in places you haven't served recently. If a story about Boise causes traffic to spike there, and that's not normally an audience hotspot for your service, chances are you won't have any measurements of your own to go on. Community intelligence turns these dark corners of your empire into known knowns through automated crowdsourcing of quality of experience metrics. When combined with real-time server health checks and third-party data streams, you have a powerful ability to make efficient, economical routing decisions, even for destinations you don't have any history with.

The more insight and intelligence can be used to accelerate the acquisition of known knowns, the better it is for your business and your bottom line. In the New Year, we should be less accepting of blind spots. They're expensive — they cost us time, money, and customers. Nobody has enough human problem solvers around to keep putting out fires and rigging up one-off workarounds. Our best talent should be working on the next release, the next big idea, or the next major dilemma (Net Neutrality game changers, anyone?) — not floundering around trying to guess what's holding up traffic. You can't control what you can't see, and on the hybrid IT battlefield, control keeps you on top of the hill. We're pretty sure Donald Rumsfeld would agree.

Josh Gray is Chief Architect at Cedexis

The Latest

December 11, 2018

Companies expect increased reliance on Cloud Native Applications (CNAs), however security concerns could prove to be a major obstacle, according to The State of Cloud Native Security ...

December 06, 2018

The general consensus tends to be that in the world of agile and DevOps, ITSM teams are increasingly being left behind. But the truth is, in more forward-thinking IT organizations, this isn’t the case. The fact is that ITSM is playing, or at least should play, a growing role in support of agile and DevOps initiatives. But this role still remains limited due to the fact that DevOps teams, and their management, are (more often than not) leaving them out as a tool of choice ...

December 05, 2018

The industry is revealing increasingly optimistic attitudes towards mainframes, with 93% of executives and 92% of all respondents viewing the mainframe as a strong long-term platform – the highest level in five years – according to the 2018 Mainframe Research Report from BMC ...

December 03, 2018

ActiveState surveyed developers and programmers in 92 countries to better understand their pain points and assess how businesses can better work with their organizations. The survey results establish a starting point for understanding the challenges that coders confront when working with open source runtimes ...

November 29, 2018

Organizations with established DevSecOps programs and practices greatly outperform their peers in how quickly they address flaws. The most active DevSecOps programs fix flaws more than 11.5 times faster than the typical organization, due to ongoing security checks during continuous delivery of software builds, largely the result of increased code scanning, according the latest State of Software Security (SOSS) report from CA Veracode ..

November 27, 2018

The push to make banking products digitally ready (and very quickly) has spurred the old “buy vs. build” debate in bank IT departments: Should we build our own software from scratch in-house? Or do we buy off-the-shelf solutions from third-parties? And while this dichotomy may have been a suitable mentality years ago at the start of the digital transformation revolution in banking, it simply no longer fits with the reality of today's more complex development landscape ...

November 26, 2018

With the rise of next-generation technologies, businesses have access to more data than ever, creating opportunities to develop new channels for revenue. Contributing to the increase in data is a growing reliance on the external supply chain. However, with the influx of data comes the necessity to understand the entire third-party ecosystem; its benefits and risks. Some of the most devastating breaches have been attributed to a third party ...

November 20, 2018

In today's digital economy, monitoring is a must. Your customers must be able to access your website and your apps, interact, purchase — and monitoring is one way to make sure this keeps happening. But the first question has to be: What should be monitored? With this in mind, APMdigest asked experts from across the IT industry for their opinions on what IT departments should be monitoring to ensure digital performance ...

November 19, 2018

Software developers and security teams have a well-known antagonistic relationship. Dev teams often feel plagued by the restrictive security standards placed on them by security teams that inhibit their ability to rapidly write applications, while security teams view developers as one of the biggest threats with which they have to grapple. There are three core challenges that must be addressed in order for security and DevOps to be in lockstep ...

November 15, 2018

Serverless infrastructure environments are set to become the dominant paradigm for enterprise technology deployments, according to a new report — Why the Fuss About Serverless? — released by Leading Edge Forum ...

Share this