DEVOPSdigest

Snowflake announced new innovations to enable developers, data engineers, and data scientists to build directly in the Data Cloud.

Snowflake’s latest advancements empower users to do more with their data, enhancing productivity and unlocking new ways to develop applications, pipelines, and machine learning models with Snowflake’s single data platform.

■ Snowflake Brings Python-Based App Development Directly to the Data Cloud: Streamlit, acquired by Snowflake in March 2022, enables tens of thousands of data scientists and other developers to easily build data applications with Python using its open source framework. Following the acquisition, Snowflake is now advancing its Streamlit integration (in development), so developers can bring their data and machine learning (ML) models to life as secure, interactive applications — all within Snowflake.

Snowflake’s Streamlit integration will bring together Streamlit’s ease of use and flexibility, with Snowflake’s scalability, governed data coverage, and security, so developers can build powerful applications without the traditional complexity involved with building and deploying web applications. The integration will allow developers to create applications with Python using their data in Snowflake, deploy and run these applications on Snowflake’s secure and governed platform, and share their applications with business teams to further unlock the value of data and ML models.

■ Snowflake Extends the Power of Python to All Users with Expansive Snowpark Ecosystem: With Python as the most1 popular language for data scientists and the third2 most popular language among all developers, Snowflake is now making Python and its rich ecosystem of open source libraries available to all users and teams with the general availability of Snowpark for Python. In the months since its public preview announcement and expanded Anaconda integration at Snowflake Summit 2022, Snowpark for Python has seen 6x growth in adoption, with hundreds of customers including Charter Communications, EDF, NerdWallet, Northern Trust, Sophos, and more building with their data using Snowpark.

With Snowpark as Snowflake’s developer framework, developers gain a streamlined architecture that natively supports users’ programming languages of choice including Java, Scala, SQL, and now Python. Snowpark for Python is part of the wider Snowpark ecosystem, bringing teams together so that they can collaborate and build on one unified platform with a highly secure Python sandbox, providing developers with the same scalability, elasticity, security, and compliance benefits they’ve come to expect when building with Snowflake. In addition, developers can eliminate data security and compliance roadblocks that have previously prevented projects from going into production. Snowflake is also releasing Snowpark-optimized warehouses (public preview in AWS), so Python developers can run large scale ML training and other memory-intensive operations directly in Snowflake, and Python Worksheets (private preview) to develop applications, data pipelines, and ML models inside Snowflake.

Partners like Anaconda, dbt Labs, and more have been instrumental in accelerating the adoption of Snowpark for Python and allowing developers to build with confidence. These advancements include Anaconda’s integration with Snowpark for Python, which make Anaconda’s open-source Python libraries seamlessly accessible to Snowflake users by eliminating the need for manual installs and package dependency management. In addition, dbt’s new Snowpark for Python support effortlessly combines the power of SQL and Python for modern analytics, enabling customers to further bridge the gap between analytics and data science teams.

■ Snowflake Simplifies Streaming Pipelines and Drives Increased Automation and Observability for Developers: Snowflake is also reimagining how users build data pipelines, making it easier to work with streaming data within a single platform, and further eliminating silos for customers. To do so, Snowflake is equipping them with the capabilities needed to eliminate complexity while leveraging core software development principles. Users can now improve productivity by onboarding data faster with Schema Inference (private preview), and execute pipelines effortlessly with Serverless Tasks (general availability) natively in Snowflake’s platform. Additionally, Snowflake is unveiling enhanced tools that further empower developers to build in the Data Cloud including:

- Dynamic Tables (private preview): Formerly introduced as Materialized Tables, Snowflake is removing the boundaries between streaming and batch pipelines by automating incremental processing through declarative data pipelines development for coding efficacy and ease. This also simplifies use cases including change data capture and snapshot isolation, and is native to Snowflake so it can be shared across all Snowflake accounts with full security and governance.

- Observability & Experiences: To further meet the needs of developers, Snowflake is investing in native observability and developer experience features so they can build, test, debug, deploy, and monitor data pipelines with increased productivity through alerting (private preview), logging (private preview), event tracing (private preview), task graphs and history (public preview), and more.

“As we continue to disrupt application development, we’re giving builders the data access and tools they need to accelerate their pace of innovation securely under Snowflake’s one unified platform,” said Torsten Grabs, Director of Product Management, Snowflake. “Snowflake’s advancements provide developers with the capabilities to build powerful applications, pipelines, and models with the utmost confidence, and eliminate complexity so they can drive value across their organizations with the Data Cloud.”