DEVOPSdigest

The OpenSearch Software Foundation, the vendor-neutral home for the OpenSearch Project, announced the general availability of OpenSearch 3.0.

This major release delivers a 9.5x performance improvement over OpenSearch 1.3.

OpenSearch 3.0 enables users to increase efficiency, deliver superior performance, and accelerate AI application development via new data management, AI agent, and vector search capabilities. Simultaneously, enhancements such as GPU-supported capabilities can reduce costs by 3.75x.

"The enterprise search market is skyrocketing in tandem with the acceleration of AI, and it is projected to reach $8.9 billion by 2030," said Carl Meadows, Governing Board Chair at the OpenSearch Software Foundation and Director of Product Management at Amazon Web Services (AWS). "OpenSearch 3.0 is a powerful step forward in our mission to support the community with an open, scalable platform built for the future of search and analytics, and it reflects our commitment to open collaboration and innovation that drives real-world impact."

To support its large-scale search platform and manage a vast amount of vector data, OpenSearch introduced GPU-based acceleration, leveraging NVIDIA cuVS for indexing workflows. New vector engine features include:

- GPU Acceleration for OpenSearch Vector Engine: Delivers superior performance for large-scale vector workloads while significantly lowering operational spend by reducing index building time. By enabling GPU deployment, this experimental feature heightens performance for data-intensive workloads and accelerates index builds by up to 9.3x.

- Model Context Protocol (MCP) Support: Native MCP support allows AI agents to easily communicate with OpenSearch, enabling more comprehensive and customizable AI-powered solutions.

- Derived Source: Reduces storage consumption by one-third by removing redundant vector data sources and utilizing primary data to recreate source documents as needed for reindexing or source call back.

OpenSearch 3.0 provides major advancements in how the platform ingests, transports and manages data including:

- Support for gRPC: Enables faster and more efficient data transport and data processing for OpenSearch deployments. This experimental feature provides a new approach to data transport between clients, servers, and node-to-node communications in OpenSearch.

- Pull-based Ingestion: Enhances ingestion efficiency and gives OpenSearch more control over the flow of data and when it's retrieved by decoupling data sources and data consumers. This experimental feature also allows users to pull data from streaming systems like Apache Kafka and Amazon Kinesis.

- Reader and Writer Separation: Ensures consistent, high-quality performance for indexing and search workloads by configuring each in isolation, allowing both workloads to work at optimal speed and scale, rather than decreasing in efficiency when the other is taxed.

- Apache Calcite Integration: Enables intuitive, iterative query building and exploration by integrating the query builder into OpenSearch SQL and PPL. Simplifies use cases for security, observability and log analysis.

- Index Type Detection: Enhances productivity by automatically determining whether an OpenSearch index contains log-related data and speeding up log analysis feature selection.

Enhancements to the platform's search infrastructure – removing legacy code, adopting a modular architecture and aligning with the latest Java advancements – boosts maintainability, performance potential, and efficiency. Updates include:

- Lucene 10 Upgrade: Modernizes the platform's search infrastructure to ensure long-term innovation, improve indexing and search capabilities, and increase performance of parallel task execution.

- Java 21 Minimum Supported Runtime: Enables access to modern language features and performance improvements.

- Java Platform Module System Support: Improves organization, eliminates top level split packages and creates a foundation for refactoring the monolithic server module into separable libraries.

OpenSearch 3.0 is now available.