Rockset Unveils Real-Time SQL Analytics for Raw Events from Apache Kafka
September 30, 2019

Rockset announced the capability to analyze raw events from Apache Kafka in real time.

Kafka, backed by Confluent, is one of the most popular distributed streaming platforms and capable of handling trillions of events a day. Rockset takes an entirely new approach to ingesting, analyzing and serving data so that developers and business stakeholders can run powerful SQL analytics, including joins, on raw event data from Kafka. With this release, Rockset is also announcing a partnership with Confluent, with Rockset’s Kafka Connect Plugin listed as a Verified Gold Connector in Confluent Hub.

Increasingly, businesses are capturing real-time data to drive intelligent actions on the fly. However, traditional databases are not built to handle semi-structured data, making it difficult to operationalize event data like this in real time. In an effort to solve this issue and unlock analytics, considerable data engineering effort goes into building complex data pipelines that schematize and load NoSQL data from Kafka event streams into SQL-based systems. These pipelines are difficult to build, expensive to maintain and hours behind in terms of insights into events - making “real-time” operational analytics on event data next to impossible.

Rockset complements Kafka’s KSQL stream processing capabilities by serving as the “sink” that ingests the processed stream. With Rockset, new event data from Kafka is automatically represented as a dynamic SQL table and available for querying in seconds. Rockset uses Converged Indexing™ and a Distributed SQL Processing Engine under the hood to enable customers to filter, aggregate and join across different datasets from different sources in milliseconds, without upfront schema definitions.

“When you embrace modern real-time technologies like Kafka, you discover that NoSQL databases do not support the type of powerful analytics you need, and that’s when you turn to SQL databases. But it will take you hours to extract-transform-load these events into a traditional SQL database and that is just not fast enough for real-time use cases,” said Venkat Venkataramani, co-founder and CEO of Rockset. “Our goal is to give Kafka users the speed and simplicity they need for deriving maximum value from their event streams in seconds.”

With this release, Rockset supports the ability to:

- Visualize event data in leading real-time SQL dashboards with JDBC support, including Tableau, Apache Superset, Redash and Grafana.

- Create developer APIs for building microservices and applications for the Internet of Things (IoT), e-commerce, operational monitoring and more.

- Join Kafka event streams with business data in Amazon DynamoDB, Amazon Kinesis, Amazon S3, Google Cloud Storage and more.

Share this

Industry News

January 23, 2020

StackRox announced that the latest version of the StackRox Kubernetes Security Platform includes support for Google Anthos, the open application platform that enables users to modernize, build and run applications across on-premise and multiple public cloud environments.

January 23, 2020

CloudVector launched API Shark, the free API discovery and observability tool.

January 23, 2020

Thundra announced $4 million in Series A funding led by global investment firm Battery Ventures.

January 22, 2020

CollabNet VersionOne and XebiaLabs have merged.

January 22, 2020

Keyfactor announced DevOps integrations with automation and containerization providers Ansible, Docker, HashiCorp, Jenkins and Kubernetes to offer security-first services and solutions designed to seamlessly integrate with existing enterprise tools and applications.

January 22, 2020

Sysdig raised $70 million in Series E funding.

January 21, 2020

Red Hat announced the general availability of Red Hat OpenShift Container Storage 4 to deliver an integrated, multicloud experience to Red Hat OpenShift Container Platform users.

January 21, 2020

Snyk has secured a $150 million investment, led by Stripes, a leading New York-based growth equity firm.

January 21, 2020

vChainannounced the close of a $7M Series A investment round.

January 16, 2020

VAST Data announced the general availability of its new Container Storage Interface (CSI).

January 16, 2020

Fugue has open sourced Regula, a tool that evaluates Terraform infrastructure-as-code for security misconfigurations and compliance violations prior to deployment.

January 16, 2020

WhiteHat Security will offer free application scanning services to federal, state and municipal agencies in North America.

January 15, 2020

Micro Focus announced the release of Micro Focus AD Bridge 2.0, offering IT administrators the ability to extend Active Directory (AD) controls from on-premises resources, including Windows and Linux devices to the cloud - a solution not previously offered in the marketplace.

January 15, 2020

SaltStack announced the availability of three new open-source innovation modules: Heist, Umbra, and Idem.

January 15, 2020

ShiftLeft announced a partnership and deep integration with CircleCI that enables organizations to insert security directly into developer pull requests from code repositories.