What Are Engineers Saying About Cloud-Native's Impact on Observability?
January 06, 2021

Dotan Horovits

For the last few years, cloud-native technologies like microservices, serverless, and Kubernetes have been increasingly dominant in the technology stack of modern engineering teams. Why? First of all, they work seamlessly with modern cloud environments, they're scalable, resilient, and enable continuous delivery of software in an agile fashion.

But all these benefits don't come without drawbacks — and as over 1000 engineers report in Logz.io's DevOps Pulse 2020, many of these drawbacks relate to their complexity and their tendency to hinder observability into system performance.

While there's still some confusion in the industry about observability and it's distinction from more popular terms such as monitoring and APM, we're seeing increased adoption of many observability technologies, primarily tracing in 2020.

Could this be to combat the complexity of cloud-native and microservices architectures? Let's take a deeper dive into the data!

The Rise of Cloud-Native Tech

According to the DevOps Pulse, while the majority of workloads today run on containers and cloud computing infrastructure. However, many organizations are still in the process of transitioning to a cloud-native architecture.

56% of respondents store more than half of their infrastructure in the public cloud, while 42% store three quarters or more, an 8% increase from the survey's results in 2019. AWS maintains its dominance with over 66% of the respondents reported using it, but Google Cloud (11%) is catching up with Azure (12%) in the race to the second runner.

Not surprisingly, containers have also become a mature technology, with 50% of respondents having half of their applications or more containerized.

Although much younger, serverless adoption is growing, with only 25% currently not utilizing this technology. This is a 33% increase from 2019, in which 58% had not yet adopted serverless frameworks.

As we can see, cloud-native technology has become a cornerstone of modern architecture and is not going anywhere anytime soon. Open source plays an important role in this movement, with prominent open source projects such as Docker, Kubernetes and Prometheus becoming de-facto standards.

But as more engineers jump on the cloud-native bandwagon, the challenges associated with managing and monitoring these technologies are becoming increasingly apparent. Approximately 87% of respondents report their main observability challenges lie in Kubernetes, microservices, and serverless frameworks.

Kubernetes in particular still raises challenges in multiple areas. Among the top Kubernetes challenges are monitoring and troubleshooting (44%), security (35%), networking (34%), and cluster management (30%). This is due to the fact that adopting Kubernetes involves adapting new operational and security practices, as well as the fact that OSS community projects typically fall short on enterprise grade capabilities, documentation and support. This is primarily why managed Kubernetes services are so popular, as they reduce the complexity for the user while providing the flexibility of orchestrating containerized workloads with Kubernetes.

Is Distributed Tracing Our Next Silver Bullet?

While no technology is a perfect solution in and of itself, the rise of distributed tracing as the third telemetry data type for observability may help engineering teams tackle issues presented by microservice architecture..

Survey respondents indicated their top reasons for implementing distributed tracing are identifying faulty services during troubleshooting (34%), understanding request sequence and critical paths (24%), identifying request latency sources (22%), and understanding service dependencies (20%).

According to the DevOps Pulse, tracing solutions have increased in popularity this year. In 2019, only 19% of respondents had adopted distributed tracing solutions, but in this year's survey, the number rose to 26%. In addition, 68% of respondents say that distributed tracing plays an important role in their observability strategy. Among those who have not yet implemented distributed tracing, 64% plan to implement it in the next two years.

Jaeger is an open source project, and backed by the Cloud Native Computing Foundation (CNCF). Jaeger's growing adoption is a clear indication of the prominence of open source in observability. About 44% of the survey respondents indicated that over half of their observability tools are open source. Respondents indicated the clear benefits of open source in easing integrating with DevOps and cloud-native tech, the low cost of ownership and the virtue of being a part of, and benefiting from, the community.

Combined with logs and metrics for a complete observability solution, tracing can help crack performance issues in production, better understand our distributed system and accelerate troubleshooting and root-cause analysis.

There's still a ways to go until we get there as only 30% of respondents currently combine telemetry data from logs, metrics and traces to achieve full observability and improve production monitoring and analytics.

But we expect that in the next few years tracing adoption will continue to increase significantly until it, too, becomes a mature and widely used technology, and a unified approach to observability will be established.

Dotan Horovits is a Product Evangelist at Logz.io
Share this

Industry News

September 29, 2022

CloudBees announced the acquisition of ReleaseIQ to expand the company’s DevSecOps capabilities, empowering customers with a low-code, end-to-end release orchestration and visibility solution.

September 29, 2022

SmartBear continues expanding its commitment to the Atlassian Marketplace, adding Bugsnag for Jira and SwaggerHub Integration for Confluence.

Bugsnag developers monitoring application stability and documenting in Jira no longer need to interrupt their workflow to access the app. Developers working in SwaggerHub can use the macro to push API definitions and changes directly to other teams and business stakeholders that work within Confluence. By increasing the presence of SmartBear tools on the Atlassian Marketplace, the company continues meeting developers where they are.

September 29, 2022

Ox Security exited stealth today with $34M in funding led by Evolution Equity Partners, Team8, and M12, Microsoft's venture fund, with participation from Rain Capital.

September 29, 2022

cnvrg.io announced that the new Intel Developer Cloud is now available via the cnvrg.io Metacloud platform, providing a fully integrated software and hardware solution.

September 28, 2022

Kong introduced a number of new performance, security and extensibility features across its entire product portfolio, including major new releases of Kong Gateway, Kong Konnect, Kong Mesh, Kong Insomnia and Kong Ingress Controller, as well as new projects from the Kong Incubator.

September 28, 2022

BroadPeak Partners announced the availability of the new K3 API Connector.

September 28, 2022

Aqua Security announced a new end-to-end software supply chain security solution.

September 27, 2022

DevOps Institute will host SKILup Festival in Singapore on November 15, 2022.

September 27, 2022

Delinea announced the latest release of DevOps Secrets Vault, its high-speed vault for DevOps and DevSecOps teams.

September 27, 2022

The Apptainer community announced version 1.1.0 of the popular container system for secure, high-performance computing (HPC). Improvements in the new version provide a smaller attack surface for production deployments while offering features that improve and simplify the user experience.

September 26, 2022

Secure Code Warrior unveiled Coding Labs, a new mechanism that allows developers to more easily move from learning to applying secure coding knowledge, leading to fewer vulnerabilities in code.

September 26, 2022

ActiveState announced the availability of the ActiveState Artifact Repository.

September 26, 2022

Split Software announced the availability of its Feature Data Platform in the Microsoft Azure Marketplace.

September 22, 2022

Katalon announced the launch of the Katalon Platform, a modern and comprehensive software quality management platform that enables teams of any size to easily and efficiently test, launch, and optimize apps, products, and software.

September 22, 2022

StackHawk announced its Deeper API Security Test Coverage release.