What Comprises the Modern Data Cloud
January 11, 2021

Manav Mital
Cyral

In 1963, a team of talented engineers led by J.C.R. Licklider took on a challenge like no other before: to develop a technology that would allow a computer to be used by two or more persons simultaneously. With the challenge, they took a $2 million research grant, and thus commenced a journey, which paved the way to computer networks, virtualization technologies, and the Internet. But even four decades later, in the early 2000s, the world had still to discover one of the most impactful contributions of Licklider's team — Cloud computing.

At that point, the data still resided in on-premises databases, but the success of early Application Cloud giants like Salesforce, Workday, and Google signaled a shift in the established trend. Enterprise productivity, along with the data that fueled it, started moving to the shared services that lived outside the perimeter of the organization. It took companies like Amazon, Google and Microsoft another decade to reach the next critical milestone — the Infrastructure Cloud, which allowed organizations to not only run applications, but migrate their storage and compute resources to the cloud, recognizing massive gains in performance and efficiency. Despite these gains, organizations quickly discovered that having the means to store and process information wasn't enough to make it readily accessible. Data continued to suffer from the legacy of the on-premises days: endless siloes, inefficiencies, lack of interoperability and, as a result, poor security.

Fortunately, another turn of the decade has marked yet another milestone — a shift in data and cloud technology, which is often referred to as the Data Cloud. For the first time in history, in a matter of years, an entire ecosystem of cloud-based services appeared to store, process and analyze operational, business intelligence and other data. True to the diverse nature of data repositories, this ecosystem spans databases, data warehouses, and data pipelines. The best part? It works.

It may seem like the Data Cloud has appeared out of nowhere, but in reality, it's been a part of several large trends that the industry is going through:

Digital Transformation allows organizations to use digital technologies to create new customer experiences, business processes, or employee management to meet changing business, market and cultural requirements. The Data Cloud enables companies to efficiently capture all their diverse data, experiment with the various use cases, scale to accommodate their need to unlock the value from their data, and optimize the price for performance and value.

Data Democratization makes data available to a wide range of stakeholders in a business. It has been a key tenet of digital transformation, and helps unlock the value locked within proprietary data embedded in various pockets of the organization, which in turn helps improve both their top line and bottom line. Embracing the Data Cloud helps teams pick tools for managing, processing and analyzing data that make most sense to them.

Infrastructure as Code (IaC) allows developers to manage and configure infrastructure through text files in a human and machine-readable format. Easy to scale and prototype, IaC has seen a massive uptake as organizations embrace the cloud. Developers can now easily spin up Data Cloud services for their applications, and services to move data across from them to their data warehouse.

While still early in its growth, the Data Cloud has clearly proven its potential. From the beginning, the allure of the Data Cloud was in its ability to solve the siloing data problem. With its flexible architecture, it has quickly enabled teams to pick the technologies they most needed, providing easy access to the relevant data repositories, and eliminating the consistency and cost problems.

But quickly, other benefits of embracing the Data Cloud also became apparent:

■ The Data Cloud has brought the processing and business intelligence into the cloud, which has enabled virtually unlimited scale and performance, and unlocked many new use cases and opportunities.

■ The Data Cloud has enabled businesses to become data-driven, to make faster, better, decisions, and improve their customer reach, engagement and retention.

■ The Data Cloud has freed up a lot of internal resources normally required for infrastructure procurement, deployment and maintenance, given the as-a-service model of delivery. Those resources can now be used to continuously refine their products, unlock new opportunities and deliver deeper insights to all stakeholders.

Any type of cloud inherently introduces new challenges. Just like the application and infrastructure clouds, the Data Cloud brings with it security issues that have not been seen before. For example, one of the biggest promises of the Data Cloud is its flexibility and support for various applications, data sources, and third-party applications such as BI tools. With such flexibility, controlling access based on IPs becomes an insurmountable task, forcing organizations to find ways to integrate identity into the data source.

Further, governance and compliance becomes impossible to manage as services scale. Enforcing integrations with identity management for all data sources can easily turn into a compliance nightmare, and granting access to new users becomes such a burden that many people cut corners and resort to shared accounts and finding excuses to bypass the compliance requirements.

Finally, distributing data amongst numerous services inside a Data Cloud leads to a distributed governance model, which is a critical component in the cloud. Each tool requires different methods for managing identities and controlling permissions. They also each generate different logs and audit trails, complicating compliance and visibility.

Despite all the risks, it is clear that the promise of the data cloud far outweighs all the challenges. And as more organizations embrace the new ecosystem, more leaders start thinking about making their data not only more accessible, but also more secure. This thinking led to the emergence of a burgeoning cloud security industry in the past, and — if history is any indicator — we might be on the verge of seeing a brand new data cloud security category in the making.

Manav Mital is CEO of Cyral
Share this

Industry News

December 06, 2021

Ascend.io announced support for Amazon Redshift Serverless powered on Amazon Web Services, Inc. (AWS), a fully managed petabyte-scale cloud data warehouse.

December 06, 2021

Neosec formed a strategic partnership with Kong Inc. to integrate its API security platform with Kong Gateway to provide a complete enterprise-class solution for managing and securing APIs and microservices.

December 02, 2021

Mirantis announced DevOpsCare, powered by Lens, a vendor-agnostic, fully-managed CI/CD (continuous integration/continuous deployment) product for any Kubernetes environment, offering developers higher levels of productivity more quickly.

December 02, 2021

The D2iQ Kubernetes Platform (DKP) is now available in AWS Marketplace, a digital catalog with thousands of software listings from independent software vendors that make it easy to find, test, buy, and deploy software that runs on Amazon Web Services, Inc. (AWS).

December 01, 2021

Bugcrowd announced the availability of Bugcrowd's cybersecurity solutions on the AWS Marketplace, providing customers with easy access, simplified billing, quick deployment, and streamlined license management.

December 01, 2021

Kublr received Microsoft Azure Arc-enabled Kubernetes validation, including for Azure Arc-enabled Kubernetes for Data Services.

December 01, 2021

CloudSphere achieved Amazon Web Services (AWS) Migration and Modernization Competency for discovering, planning, and helping enterprise customers move business services to AWS to reduce cost, increase agility and improve security.

November 30, 2021

JFrog introduced a new container registry and package manager for running JFrog Artifactory with Kubernetes clusters on-premises, in the cloud, or both.

November 30, 2021

Docker announced the availability of Docker Official Images directly from Amazon Web Services (AWS).

November 30, 2021

Weaveworks announced the general availability of Weave GitOps Enterprise, a GitOps platform that automates continuous application delivery and Kubernetes operations at any scale.

November 30, 2021

Amazon Web Services announced AWS Mainframe Modernization, a new service that makes it faster and easier for customers to migrate mainframe and legacy workloads to the cloud, and enjoy the superior agility, elasticity, and cost savings of AWS.

November 29, 2021

Quali announced the newest release of Torque Enterprise, which includes enhanced integration with Terraform, new custom tagging capabilities, and improved cost visibility dashboards, unleashing an entirely new level of self-service access to application environments on demand.

November 29, 2021

Vertical Relevance (VR), a financial services-focused consulting firm, achieved Amazon Web Services (AWS) DevOps Competency status.

November 18, 2021

Loft Labs announced the launch of Loft version 2 with a focus on ease of use that overcomes the major complaint that Kubernetes is complex and hard to set up.

November 18, 2021

Perforce Software announced new functionality to speed remediation of discovered defects in automated scans.