OctoML Releases New Platform
June 22, 2022

OctoML released a major platform expansion to accelerate the development of AI-powered applications by eliminating bottlenecks in machine learning deployment.

This latest release enables app developers and IT operations teams to transform trained ML models into agile, portable, production-ready software functions that easily integrate with their existing application stacks and DevOps workflows.

"AI has the potential to change the world, but it first needs to become sustainable and accessible," said Luis Ceze, CEO, OctoML. "Today's manual, specialized ML deployment workflows are keeping application developers, DevOps engineers and IT operations teams on the sidelines. Our new solution is enabling them to work with models like the rest of their application stack, using their own DevOps workflows and tools. We aim to do that by giving customers the ability to transform models into performant, portable functions that can run on any hardware."

Models-as-functions can run at high performance anywhere from cloud to edge, remaining stable and consistent even as hardware infrastructure changes. This DevOps-inclusive approach eliminates redundancy by unifying two parallel deployment streams—one for AI and the other for traditional software. It also maximizes the success of the investments that have already been made in model creation and model operations.

The new OctoML platform release enables customers to work with existing tools and teams. Intelligent functions can be leveraged with each user's unique combination of model, development environment, developer tools, CI/CD framework, application stack and cloud—all while meeting cost and performance SLAs.

Key platform expansion features include:

- Machine Learning for Machine Learning capabilities—Automation detects and resolves dependencies, cleans and optimizes model code, accelerates and packages the model for any hardware target.

- OctoML CLI provides a local experience of OctoML's feature set and integrates with SaaS capabilities to create accelerated hardware-independent models-as-functions.

- Comprehensive fleet of 80+ deployment targets—in the cloud (AWS, Azure and GCP) and at the edge with accelerated computing, including GPUs, CPUs, NPUs from NVIDIA, Intel, AMD, ARM and AWS Graviton—used for automated compatibility testing, performance analysis and optimizations on actual hardware.

- Performance and compatibility insights backed by real-world scenarios (not simulated) to accurately inform deployment decisions and ensure SLAs around performance, cost and user experience are met.

- Expansive software catalog covering all major ML frameworks, acceleration engines such as Apache TVM, and software stacks from chip makers.

- NVIDIA Triton Inference Server is packaged as the integrated inference serving software with any model-as-a-function generated by the OctoML CLI or OctoML platform.

Combining NVIDIA Triton with OctoML enables users to more easily choose, integrate, and deploy Triton-powered inference from any framework on mainstream data center servers.

"NVIDIA Triton is the top choice for AI inference and model deployment for workloads of any size, across all major industries worldwide," said Shankar Chandrasekaran, Product Marketing Manager, NVIDIA. "Its portability, versatility and flexibility make it an ideal companion for the OctoML platform."

"NVIDIA Triton enables users to leverage all major deep learning frameworks and acceleration technologies across both GPUs and CPUs," said Jared Roesch, CTO, OctoML. "The OctoML workflow extends the user value of Triton-based deployments by seamlessly integrating OctoML acceleration technology, allowing you to get the most out of both the serving and model layers."

Share this

Industry News

January 26, 2023

Ubuntu Pro, Canonical’s comprehensive subscription for secure open source and compliance, is now generally available.

January 26, 2023

Mirantis, freeing developers to create their most valuable code, today announced that it has acquired the Santa Clara, California-based Shipa to add automated application discovery, operations, security, and observability to the Lens Kubernetes Platform.

January 25, 2023

SmartBear has integrated the powerful contract testing capabilities of PactFlow with SwaggerHub.

January 25, 2023

Venafi introduced TLS Protect for Kubernetes.

January 25, 2023

Tricentis announced the general availability of Tricentis Test Automation, a cloud-based test automation solution that simplifies test creation, orchestration, and scalable test execution for easier collaboration among QA teams and their business stakeholders and faster, higher-quality, and more durable releases of web-based applications and business processes.

January 24, 2023

Harness announced the acquisition of Propelo.

January 23, 2023

Couchbase announced its Couchbase Capella Database-as-a-Service (DBaaS) offering on Azure.

January 23, 2023

Mendix and Software Improvement Group (SIG) have announced the release of Mendix Quality & Security Management (QSM), a new cybersecurity solution that provides continuous deep-dive insights into security and code quality to immediately address risks and vulnerabilities.

January 23, 2023

Trunk announces the public launch of CI Analytics.

January 23, 2023

Panaya announced a new Partnership Program in response to ongoing growth within its partner network over the past year.

January 23, 2023

Cloudian closed $60 million in new funding, bringing the company’s total funding to $233 million.

January 19, 2023

Progress announced the R1 2023 release of Progress Telerik and Progress Kendo UI.

January 19, 2023

Wallarm announced the early release of the Wallarm API Leak Management solution, an enhanced API security technology designed to help organizations identify and remediate attacks exploiting leaked API keys and secrets, while providing on-going protection against hacks in the event of a leak.

January 19, 2023

ThreatModeler launched Threat Model Marketplace, a cybersecurity asset marketplace offering pre-built, field-tested threat models to be downloaded — free for a limited time — and incorporated into new and ongoing threat modeling initiatives.

January 18, 2023

Software AG has launched new updates to its webMethods platform that will simplify the process by which developers can find, work on and deploy new APIs and integration tools or capabilities.