Kong Gateway 3.6 Released
February 15, 2024

Kong announced a suite of open-source AI plugins for Kong Gateway 3.6 that can turn any Kong Gateway deployment into an AI Gateway, offering unprecedented support for multi-Language Learning Models (LLMs) integration.

By upgrading to Kong Gateway 3.6, available today, users can access a suite of six new plugins that are entirely focused on AI and LLM usage. This will enable developers who want to integrate one or more LLMs into their products to be more productive and ship AI capabilities faster, while at the same time offering architects and platform teams a secure solution that ensures visibility, control and compliance on every AI request sent by the teams. Due to the tight integration with Kong Gateway, it will now be possible to easily orchestrate AI flows in the cloud or on self-hosted LLMs with industry leading performance and low latency, which are critical to the performance of AI-based applications.

“Today marks a significant milestone in our journey towards democratizing AI for developers and enterprises worldwide. By open-sourcing this suite of innovative AI capabilities, including no-code AI plugins, we’re removing the barriers to AI adoption and making it possible for developers to leverage multiple LLMs effortlessly and ship AI powered applications faster. At the same time, we’re providing governance and visibility to all the AI traffic that is being generated by an organization,“ said Marco Palladino, Chief Technology Officer and Co-Founder, Kong Inc.

By upgrading to Kong Gateway 3.6, AI builders can access this new suite of plugins entirely focused on AI and LLM usage. The suite of open source plugins delivers a range of new capabilities, including:

- Multi-LLM Integration: Kong Inc.'s "ai-proxy" plugin enables seamless integration of multiple Large Language Model (LLM) implementations, offering native support for industry leaders including OpenAI, Azure AI, Cohere, Anthropic, Mistral, and LLAMA. The standardized interface allows for simple switching between LLMs without modifying application code, facilitating the use of diverse models and rapid prototyping.

- Central AI Credential Management: The "ai-proxy" helps ensure secure and centralized storage of AI credentials within Kong Gateway. This design negates the need for credentials within applications, streamlining credential rotation and updates directly from the gateway

- Layer 7 AI Metrics Collection: Leveraging the "ai-proxy" plugin, users can now capture detailed Layer 7 AI analytics. This includes metrics such as request and response token counts, along with usage data for LLM providers and models. Integration with third-party platforms like Datadog, New Relic, and existing logging plugins in Kong Gateway, like TCP, Syslog, Prometheus, is supported, enriching observability and offering insights into developer preferences.

- No-Code AI Integrations: With the "ai-request-transformer" and "ai-response-transformer" plugins, AI capabilities are injected into API requests and responses without a single line of code. This allows for on-the-fly transformations like real-time API response translations for internationalization, enriching and converting API traffic effortlessly.

- AI Prompt Decoration: The "ai-prompt-decorator" plugin allows for the consistent configuration of AI prompt contexts, automating the inclusion of rules and instructions with each AI request to enforce organizational compliance and restrict discussions on sensitive topics. 

- AI Prompt Firewall: The "ai-prompt-guard" offers a governance layer, establishing rules to authorize or block free-form prompts created by applications. This helps ensure that prompts adhere to approved standards before being transmitted to LLM providers. 

- Comprehensive AI Egress with Extensive Features: The integration of these AI capabilities within Kong Gateway centralizes the management, security, and monitoring of AI traffic. It leverages over 1,000+ existing official and community plugins for robust access control, rate limiting, and the creation of advanced traffic control rules. The AI Gateway is equipped from day one with all Kong Gateway features, making it, we believe, the most capable in the AI ecosystem.

With this release, Kong's expertise in modern API infrastructure now extends to AI-driven use cases.

Share this

Industry News

April 23, 2024

mabl announced the addition of mobile application testing to its platform.

April 23, 2024

Spectro Cloud announced the achievement of a new Amazon Web Services (AWS) Competency designation.

April 22, 2024

GitLab announced the general availability of GitLab Duo Chat.

April 18, 2024

SmartBear announced a new version of its API design and documentation tool, SwaggerHub, integrating Stoplight’s API open source tools.

April 18, 2024

Red Hat announced updates to Red Hat Trusted Software Supply Chain.

April 18, 2024

Tricentis announced the latest update to the company’s AI offerings with the launch of Tricentis Copilot, a suite of solutions leveraging generative AI to enhance productivity throughout the entire testing lifecycle.

April 17, 2024

CIQ launched fully supported, upstream stable kernels for Rocky Linux via the CIQ Enterprise Linux Platform, providing enhanced performance, hardware compatibility and security.

April 17, 2024

Redgate launched an enterprise version of its database monitoring tool, providing a range of new features to address the challenges of scale and complexity faced by larger organizations.

April 17, 2024

Snyk announced the expansion of its current partnership with Google Cloud to advance secure code generated by Google Cloud’s generative-AI-powered collaborator service, Gemini Code Assist.

April 16, 2024

Kong announced the commercial availability of Kong Konnect Dedicated Cloud Gateways on Amazon Web Services (AWS).

April 16, 2024

Pegasystems announced the general availability of Pega Infinity ’24.1™.

April 16, 2024

Sylabs announces the launch of a new certification focusing on the Singularity container platform.

April 15, 2024

OpenText™ announced Cloud Editions (CE) 24.2, including OpenText DevOps Cloud and OpenText™ DevOps Aviator.

April 15, 2024

Postman announced its acquisition of Orbit, the community growth platform for developer companies.

April 11, 2024

Check Point® Software Technologies Ltd. announced new email security features that enhance its Check Point Harmony Email & Collaboration portfolio: Patented unified quarantine, DMARC monitoring, archiving, and Smart Banners.