Battle Plan 2018: Illuminate Blind Spots and Unknown Unknowns
December 27, 2017

Josh Gray
Cedexis

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.

Bonus points if you know who came up with that tongue twister. He was talking about terrorists, but we're here to discuss a different sort of war — the Battle for Bandwidth. These days, application and content delivery requires special tactics, an integrated strategy, and well-sourced intelligence. And the unknown unknowns are the true enemy because they inevitably lead to outages, slowdowns, and mutinous customers.

In early November, a major outage caused by a minor configuration error (a route leak, to be exact) at global backbone provider Level 3 created widespread connection issues on both U.S. coasts. Comcast, Verizon, Cox, and Vonage customers were particularly affected.

One small error can have mighty ripple effects, and the cause isn't always apparent to network admins and enterprise customers. The time it took to return the Down Detector maps from angry red to mellow yellow could have been shortened by looking at Real User Measurements (crowdsourced telemetry), realizing it wasn't a single site or ISP, and following a logic tree to find the culprit.

With Global Server Load Balancing, your delivery network is smart enough to see the barricade around the corner and switch routes on the fly — saving the day (and making the other guys look a bit dazed and confused).

Blind spots can be hiding more than outages. Your crack team of DevOps commandos can't run successful release missions if they can't check what's really going on in the field. You don't want them dashing around in the dark without a robust tactical plan based on all the parameters you can assess — when you turn unknown unknowns into known knowns from your various data streams, you can put them to work.

Continuous deployment isn't for the faint of heart — you better have your Kevlar and your night vision goggles. Companies like Salesforce are releasing updates dozens of times a day; but even a handful a week requires a careful strategy. You can use RUM to test an update by initially limiting roll-out to one data center. Check for 40x/50x errors. If you're seeing problems, you can check both user experience with your app (non-updated versions) in other places, and user experience at the same data center where you are testing the updated version, to deduce the source of trouble.

One of the biggest unknown unknowns in traffic management is what's going on in places you haven't served recently. If a story about Boise causes traffic to spike there, and that's not normally an audience hotspot for your service, chances are you won't have any measurements of your own to go on. Community intelligence turns these dark corners of your empire into known knowns through automated crowdsourcing of quality of experience metrics. When combined with real-time server health checks and third-party data streams, you have a powerful ability to make efficient, economical routing decisions, even for destinations you don't have any history with.

The more insight and intelligence can be used to accelerate the acquisition of known knowns, the better it is for your business and your bottom line. In the New Year, we should be less accepting of blind spots. They're expensive — they cost us time, money, and customers. Nobody has enough human problem solvers around to keep putting out fires and rigging up one-off workarounds. Our best talent should be working on the next release, the next big idea, or the next major dilemma (Net Neutrality game changers, anyone?) — not floundering around trying to guess what's holding up traffic. You can't control what you can't see, and on the hybrid IT battlefield, control keeps you on top of the hill. We're pretty sure Donald Rumsfeld would agree.

Josh Gray is Chief Architect at Cedexis
Share this

Industry News

July 18, 2024

Mission Cloud announced the launch of Mission Cloud Engagements - DevOps, a platform designed to transform how businesses manage and execute their AWS DevOps projects.

July 18, 2024

Accelario announces the release of its free TDM solution, including database virtualization and data anonymization.

July 18, 2024

RAVEL (formerly StratusCore) introduced RAVEL Orchestrate’s new Bare Metal Build Station functionality, which empowers IT and DevOps teams in SMBs or enterprises to intelligently prepare and deploy customized images to any physical machine connected to a network.

July 17, 2024

OpenText™ announced its solution to speed the triage and remediation of vulnerabilities throughout the stages of code development, OpenText Fortify Aviator, an AI-powered code security solution, saves developers significant time by enabling faster and easier auditing and remediation of static application security testing (SAST) vulnerabilities—all within a single solution​.

July 17, 2024

Tricentis announced the acquisition of SeaLights, a SaaS-based, software quality intelligence platform.

July 17, 2024

CAST is now available as software as a service (SaaS).

July 16, 2024

OpenText announced its latest product innovations with Cloud Editions (CE) 24.3.

July 16, 2024

Red Hat introduced new capabilities and enhancements for Red Hat OpenShift, the hybrid cloud application platform powered by Kubernetes, as well as the general availability of Red Hat Advanced Cluster Security Cloud Service.

July 16, 2024

DevEx Connect launched as a community-driven independent research, analyst and events organization focusing on everything under the DevEx umbrella, including DevOps, SRE and Platform Engineering.

July 15, 2024

Elastic announced support for Amazon Bedrock-hosted models in Elasticsearch Open Inference API and Playground.

July 11, 2024

Progress announced new and powerful enhancements in the latest release of Progress® LoadMaster® 360, its cloud-based unified application delivery platform. These enhancements help organizations protect their web applications against increasingly sophisticated cyberattacks and provide customers with an optimal application experience.

July 11, 2024

Virtusa announced a strategic partnership with Quality Clouds, a provider of SaaS governance solutions for Salesforce and ServiceNow platforms.

July 11, 2024

Zesty launched its newest offering, Commitment Manager for Amazon RDS (Relational Database Service).

July 10, 2024

MacStadium unveiled Orka Desktop, a free, local macOS virtualization tool.