What Comprises the Modern Data Cloud
January 11, 2021

Manav Mital
Cyral

In 1963, a team of talented engineers led by J.C.R. Licklider took on a challenge like no other before: to develop a technology that would allow a computer to be used by two or more persons simultaneously. With the challenge, they took a $2 million research grant, and thus commenced a journey, which paved the way to computer networks, virtualization technologies, and the Internet. But even four decades later, in the early 2000s, the world had still to discover one of the most impactful contributions of Licklider's team — Cloud computing.

At that point, the data still resided in on-premises databases, but the success of early Application Cloud giants like Salesforce, Workday, and Google signaled a shift in the established trend. Enterprise productivity, along with the data that fueled it, started moving to the shared services that lived outside the perimeter of the organization. It took companies like Amazon, Google and Microsoft another decade to reach the next critical milestone — the Infrastructure Cloud, which allowed organizations to not only run applications, but migrate their storage and compute resources to the cloud, recognizing massive gains in performance and efficiency. Despite these gains, organizations quickly discovered that having the means to store and process information wasn't enough to make it readily accessible. Data continued to suffer from the legacy of the on-premises days: endless siloes, inefficiencies, lack of interoperability and, as a result, poor security.

Fortunately, another turn of the decade has marked yet another milestone — a shift in data and cloud technology, which is often referred to as the Data Cloud. For the first time in history, in a matter of years, an entire ecosystem of cloud-based services appeared to store, process and analyze operational, business intelligence and other data. True to the diverse nature of data repositories, this ecosystem spans databases, data warehouses, and data pipelines. The best part? It works.

It may seem like the Data Cloud has appeared out of nowhere, but in reality, it's been a part of several large trends that the industry is going through:

Digital Transformation allows organizations to use digital technologies to create new customer experiences, business processes, or employee management to meet changing business, market and cultural requirements. The Data Cloud enables companies to efficiently capture all their diverse data, experiment with the various use cases, scale to accommodate their need to unlock the value from their data, and optimize the price for performance and value.

Data Democratization makes data available to a wide range of stakeholders in a business. It has been a key tenet of digital transformation, and helps unlock the value locked within proprietary data embedded in various pockets of the organization, which in turn helps improve both their top line and bottom line. Embracing the Data Cloud helps teams pick tools for managing, processing and analyzing data that make most sense to them.

Infrastructure as Code (IaC) allows developers to manage and configure infrastructure through text files in a human and machine-readable format. Easy to scale and prototype, IaC has seen a massive uptake as organizations embrace the cloud. Developers can now easily spin up Data Cloud services for their applications, and services to move data across from them to their data warehouse.

While still early in its growth, the Data Cloud has clearly proven its potential. From the beginning, the allure of the Data Cloud was in its ability to solve the siloing data problem. With its flexible architecture, it has quickly enabled teams to pick the technologies they most needed, providing easy access to the relevant data repositories, and eliminating the consistency and cost problems.

But quickly, other benefits of embracing the Data Cloud also became apparent:

■ The Data Cloud has brought the processing and business intelligence into the cloud, which has enabled virtually unlimited scale and performance, and unlocked many new use cases and opportunities.

■ The Data Cloud has enabled businesses to become data-driven, to make faster, better, decisions, and improve their customer reach, engagement and retention.

■ The Data Cloud has freed up a lot of internal resources normally required for infrastructure procurement, deployment and maintenance, given the as-a-service model of delivery. Those resources can now be used to continuously refine their products, unlock new opportunities and deliver deeper insights to all stakeholders.

Any type of cloud inherently introduces new challenges. Just like the application and infrastructure clouds, the Data Cloud brings with it security issues that have not been seen before. For example, one of the biggest promises of the Data Cloud is its flexibility and support for various applications, data sources, and third-party applications such as BI tools. With such flexibility, controlling access based on IPs becomes an insurmountable task, forcing organizations to find ways to integrate identity into the data source.

Further, governance and compliance becomes impossible to manage as services scale. Enforcing integrations with identity management for all data sources can easily turn into a compliance nightmare, and granting access to new users becomes such a burden that many people cut corners and resort to shared accounts and finding excuses to bypass the compliance requirements.

Finally, distributing data amongst numerous services inside a Data Cloud leads to a distributed governance model, which is a critical component in the cloud. Each tool requires different methods for managing identities and controlling permissions. They also each generate different logs and audit trails, complicating compliance and visibility.

Despite all the risks, it is clear that the promise of the data cloud far outweighs all the challenges. And as more organizations embrace the new ecosystem, more leaders start thinking about making their data not only more accessible, but also more secure. This thinking led to the emergence of a burgeoning cloud security industry in the past, and — if history is any indicator — we might be on the verge of seeing a brand new data cloud security category in the making.

Manav Mital is CEO of Cyral
Share this

Industry News

January 26, 2023

Ubuntu Pro, Canonical’s comprehensive subscription for secure open source and compliance, is now generally available.

January 26, 2023

Mirantis, freeing developers to create their most valuable code, today announced that it has acquired the Santa Clara, California-based Shipa to add automated application discovery, operations, security, and observability to the Lens Kubernetes Platform.

January 25, 2023

SmartBear has integrated the powerful contract testing capabilities of PactFlow with SwaggerHub.

January 25, 2023

Venafi introduced TLS Protect for Kubernetes.

January 25, 2023

Tricentis announced the general availability of Tricentis Test Automation, a cloud-based test automation solution that simplifies test creation, orchestration, and scalable test execution for easier collaboration among QA teams and their business stakeholders and faster, higher-quality, and more durable releases of web-based applications and business processes.

January 24, 2023

Harness announced the acquisition of Propelo.

January 23, 2023

Couchbase announced its Couchbase Capella Database-as-a-Service (DBaaS) offering on Azure.

January 23, 2023

Mendix and Software Improvement Group (SIG) have announced the release of Mendix Quality & Security Management (QSM), a new cybersecurity solution that provides continuous deep-dive insights into security and code quality to immediately address risks and vulnerabilities.

January 23, 2023

Trunk announces the public launch of CI Analytics.

January 23, 2023

Panaya announced a new Partnership Program in response to ongoing growth within its partner network over the past year.

January 23, 2023

Cloudian closed $60 million in new funding, bringing the company’s total funding to $233 million.

January 19, 2023

Progress announced the R1 2023 release of Progress Telerik and Progress Kendo UI.

January 19, 2023

Wallarm announced the early release of the Wallarm API Leak Management solution, an enhanced API security technology designed to help organizations identify and remediate attacks exploiting leaked API keys and secrets, while providing on-going protection against hacks in the event of a leak.

January 19, 2023

ThreatModeler launched Threat Model Marketplace, a cybersecurity asset marketplace offering pre-built, field-tested threat models to be downloaded — free for a limited time — and incorporated into new and ongoing threat modeling initiatives.

January 18, 2023

Software AG has launched new updates to its webMethods platform that will simplify the process by which developers can find, work on and deploy new APIs and integration tools or capabilities.