Monthly Snowflake Unofficial Release Notes #New features #Previews #Clients #Behavior Changes

Welcome to the fantastic Unofficial Release Notes for Snowflake! You’ll find all the latest features, drivers, and more in one convenient place. As an unofficial source, I am excited to share my insights and thoughts. Let‘’’s dive in! You can also find all of Snowflake‘’’s releases here.

This month, we provide coverage up to release 8.17.

I would appreciate your suggestions on continuing to combine these monthly release notes. Feel free to comment below or chat with me on LinkedIn.

What’s New in Snowflake

New Features

General Availability: Dynamic Tables, a new table type for continuous processing pipelines to be used. You‘’’re processing batch data daily or real-time data in minutes, dynamic tables allow you to create data pipelines that are easy to build, operate, and evolve, with GA delivering many new features sharing, Disaster recovery, better docs and observability, and Query evolution and SELECT * support
General Availability: Authentication enhancements: Authentication policies, control over how users authenticate by allowing you to specify which clients can authenticate and which authentication methods can be used with SAML2 and External OAuth security integrations
General Availability: Authentication enhancements: Identifier-first login flow, identify a user before presenting authentication options. In this flow, Snowflake prompts the user for their email address or username only, then displays authentication options based on the identity of the user
General Availability: Authentication enhancements: New properties for SAML2 security integrations, contains two new properties: ALLOWED_USER_DOMAINS and ALLOWED_EMAIL_PATTERNS and the user’s email address must match the values specified in these properties to authenticate with the identifier provider associated with the security integration. Identifier-first login needs to be enabled
General Availability: Authentication enhancements: Multiple identity providers support using multiple identity providers for federated authentication using SAML2 security integrations, which allows different users to authenticate with different identity providers. Identifier-first login needs to be enabled.
Preview: Snowflake Provider Studio allows you to work with all aspects of your Marketplace listings without needing to be an account administrator or a user with create listings privileges
Preview: Snowflake Copilot in AWS us-east-1 and us-west-2, an LLM-powered assistant that simplifies data analysis while maintaining robust data governance and seamlessly integrates into your existing Snowflake workflow. You can ask open-ended questions about your data structure, send follow-up inquiries, or even use it to refine and improve your own SQL queries
General Availability: New Region: AWS, EU (Zurich)
General Availability: Budgets, account-level monitoring and notification of Snowflake credit usage for a group of specific Snowflake objects
Preview: Access to Git repositories from Snowflake, fetch a full clone of your remote repository to a Snowflake repository stage
General Availability: Account Overview Page, improves visibility into incurred costs and provides information that can be a starting point for reporting and optimizing your spend
General Availability: Snowflake Connector for ServiceNow V2, provides instant access to up-to-date ServiceNow data without needing to integrate against API endpoints or manage third-party solutions manually. Built on the Snowflake Native App Framework, the connector ingests data from ServiceNow into Snowflake automatically leveraging built-in security and reliability capabilities
General Availability: New FAILOVER privilege for Client Redirect, enables promoting a secondary connection to serve as the primary connection. The privilege is granted to the ACCOUNTADMIN role by default, and this privilege can be granted to other roles to facilitate failover in a disaster recovery scenario. It’s still a Business Critical feature
Preview: Managing Listings using SQL, this does not apply to paid, personalized listings, or listings on private data exchanges
General Availability: Snowflake Connector for Google Analytics Raw Data and Snowflake Connector for Google Analytics Aggregate Data

Snowsight Updates

Preview: No limit on rows returned in worksheet results, this is a change limit to 10,000 rows of results or 16MB

Performance Updates

Faster query results in Snowsight, reduced the time it takes for query results to appear when run in Snowsight. Improvements are most noticeable for queries that return result sets larger than 10,000 rows

SQL Updates

General Availability: READ ONLY property available for tables, you can now make tables READ Only when you clone
General Availability: ST_INTERSECTION_AGG and ST_UNION_AGG functions, new Geospatial functions
General Availability: Fixed an issue with the PARSE_IP function
General Availability: Fixed an issue with the SPLIT_PART function
Preview: Several Geospatial SQL Functions, H3_COMPACT_CELLS, H3_COMPACT_CELLS_STRINGS, H3_IS_PENTAGON, H3_IS_VALID_CELL, H3_TRY_GRID_DISTANCE, H3_TRY_GRID_PATH, H3_UNCOMPACT_CELLS, and H3_UNCOMPACT_CELLS_STRINGS, if curious about each functionality go here
General Availability: SHOW ACCOUNTS, Lists all accounts in an organization, with the exclusion of managed accounts

Machine Learning Updates

Preview: Snowflake Cortex ML Function, Evaluation Metrics for Forecasting and Anomaly Detection, training-time evaluation metrics
Preview: Forecasting improvements in Snowflake Cortex ML Functions, improved the algorithm behind Time-Series Forecasting: Reduce the incidence of trend extrapolation errors (incorrect trends) and Improve accuracy on “noisy” time series, however, only in models trained after April 15th (8.14)
Preview: Snowflake Cortex LLM: The reka-flash model is now available for text completion tasks. reka-flash is a high-quality model optimized for fast processing. The model is now available in AWS US East (N. Virginia) and AWS US West (Oregon)
Preview: Snowflake Cortex LLM: The mixtral-8x7b model is now 56% more cost-efficient, with the credits per million tokens cut to 0.22 from 0.50
Preview: Snowflake Cortex LLM: The mixtral-8x7b model is now available for use in AWS Europe (Frankfurt) and Azure West Europe (Netherlands).

Data Clean Rooms Updates

Audience Overlap & Segmentation template, allows analysts to create segmentation groups for audience overlap results, replaces the old
Provider Activation with Enrichment, enrich the results with data from additional columns of their table or the provider’s table
Service Agreement: Providers and consumers who are Snowflake Service customers can now use Snowflake Data Clean Rooms under their existing Service Agreement with Snowflake.
Billing for clean room managed accounts: The consumer, not the provider, is billed for the consumer’s use of a clean room managed account. A managed account allows someone who is not a Snowflake Service customer to collaborate with a provider.
Additional regions: more AWS regions, including Canada, and Azure regions included as well, including Asia, Canada, Australia and US

Extensibility Updates

General Availability: Python UDTFs with vectorized process methods, provides a convenient way to operate over rows in batches when the method returns one output row for each input row

Snowpark Container Services Updates

Preview: Node Metrics, provides metrics about nodes in the compute pool on which Snowflake runs services, as well as metrics about the services themselves running on the compute pool
Preview: support for block storage volumes for your services

Data Pipelines/Data Loading/Unloading Updates

General Availability: Support for granting the READ and WRITE privileges on external stages, previously, you could grant only the USAGE privilege on an external stage
General Availability: New copy option: INCLUDE_METADATA, provides a user-defined mapping between target table columns to METADATA columns and can only be used with the MATCH_BY_COLUMN_NAME copy option

Open-Source Updates

Snowflake Artic, an LLM for Enterprises
I am still trying to figure out this part out, so I am going try to do better next monthly article

Client, Drivers, Libraries and Connectors Updates

New features:

Snowflake Connector for Google Analytics Raw Data 0.18.0 (procedure UPDATE_CONNECTION_CONFIGURATION to be used for re-authentication purposes)
Snowflake Connector for Google Analytics Aggregate Data 0.24.0 (The CONNECTOR_EXECUTION_LOG table is no longer visible to users)
Snowflake Connector for Google Analytics Aggregate Data 0.25.0 (connector tries to avoid 502 errors by reducing the date range for large reports)
Snowflake Connector for ServiceNow V2 5.0.0 (initial release,external function making API calls to ServiceNow are replaced with external access, Procedures return an object with response_code property, and Signatures and behavior of several views changed)
SQLAlchemy 1.5.2 (minimum SQLAlchemy version to 1.4.19, and sequence ordering in tests)
SQLAlchemy 1.5.3 (maximum SQLAlchemy dependency version to lower than 2.0.0)
Snowpark ML 1.4.0 (the apply method is no longer included as a target method by default, the registry now supports logging sentence transformer models, the version_name argument is no longer required when logging a model)
Snowpark Library for Scala and Java 1.11.0 (Java 17 stored procedure) — Compatible with Snowflake 8.12
Snowpark Library for Scala and Java 1.12.0 (Geometry data type, sum(String)function, and others)— Compatible Snowflake release: 8.14
Snowpark Library for Python 1.15.0 (truncate save mode in DataFrameWrite, functions to unload data from a DataFrame, distributed tracing using open telemetry APIs for action functions in DataFrame and DataFrameWriter, Added support for snow:// URLs to snowflake.snowpark.Session.file.get and snowflake.snowpark.Session.file.get_stream, register stored procedures and UDFs with a comment, UDAF client support is ready for public preview, dynamic pivot, Improved the generated query performance for both compilation and execution by converting duplicate subqueries to CTEs, lots of local testing updates)
Snowflake Connector for Python 3.8.1 (Improved externalbrowser authentication in containerized environments, support for parsing structured type information in schema queries, library update platformdirs, diagnostics to use system$allowlist instead of system$whitelist, updated the logging level from INFO to DEBUG when logging the executed query using SnowflakeCursor.execute)
Snowflake Connector for Python 3.9.0 (support for log settings in a logging configuration file, improved S3 acceleration logic when connecting to a China endpoint)
Snowflake Connector for Python 3.10.0 (support for structured types to fetch_pandas_all)
Snowflake CLI 2.2.0 (lots of deprecated features, support for fully qualified names (database.schema.name) in the Streamlit project definition name parameter, support for fully qualified image repository names in spcs image-repository commands, --if-not-exists option option to the snow spcs service create and snow spcs compute-pool create commands, --replace and --if-not-exists options for the snow spcs image-repository create command, Snowflake Connector for Python diagnostic reports, snow app deploy command that creates an application package and syncs the local changes to the stage without creating or updating the application, is_default column to the snow connection list output to highlight the default connection, Updated the snow snowpark package create command, Updated the snow snowpark build command, Added the --recursive option to the snow stage copy command to reproduce the directory structure locally when copying from a stage, Addedsnow git commands to support for Git repositories, Added the snow stage execute command to run the SQL EXECUTE IMMEDIATE command from a stage path, Added the --pattern option to the snow stage list-files command to support filtering results with regex, Added support for any source supported by pip in snow snowpark commands, Added the ability to fetch available packages list from Snowflake instead of directly from Anaconda with fallback to the old method)
ODBC Driver 3.3.0 (support for log settings in a logging configuration file, updated libraries curl, openssl, zlib)
JDBC Driver 3.15.1 (support for missing proxy and user password JVM parameters: http.proxyUser, http.proxyPassword, https.proxyUser, https.proxyPassword, bumped the nimbus-jose-jwt dependency to version 9.37.3)
JDBC Driver 3.16.0 (support for structured types, support for vector types, improved support for encrypted private keys)

Bug fixes:

Snowflake Connector for Google Analytics Raw Data 0.16.3 (temporary duplication of the dispatcher task), Snowflake Connector for Google Analytics Raw Data 0.16.3 (issue with disrupted scheduling of ingestions during execution of dispatcher task)
Snowflake Connector for Google Analytics Aggregate Data 0.23.0 (Date ranges for initial and incremental loads are split into 31-day intervals to reduce the number of 502 API errors)
Snowflake Connector for Google Analytics Aggregate Data 0.23.1 (Fixed start date calculation)
Snowpark ML 1.4.0 (multiple models are called in the same query, models after the first returned incorrect results, several modeling bug fixes)
Snowpark Library for Python 1.15.0 (where statement_params is not passed to query executions, fail causing snowflake.snowpark.Session.file.get_stream to fail for quoted stage locations, internal type hint in utils.py might raise AttributeError when the underlying module can not be found)
Snowflake Connector for Python 3.8.1 (bug that the truncated password in log is not masked)
Snowflake Connector for Python 3.9.1 (issue that caused a HTTP 400 error when connecting to a China endpoint)
Snowflake Connector for Python 3.10.0 (issue relating to incorrectly formed China S3 endpoints)
Snowflake CLI 2.2.0 (Added the --image-name option for the image name argument in the spcs image-repository list-tags command for consistency with other commands, fixed an issue where spcs image-registry login errors were not formatted correctly, project definitions no longer accept extra fields with any extra fields cause an error, fixed an issue with empty zip files for Snowpark build paths for builds that used the --project option, improved error messages for the snow snowpark build command, fixed version parsing for packages lookup on the Snowflake Anaconda channel, fixed an issue with handling database, schema, and role identifiers containing dashes, fixed a schema override bug in th snow connection test command, in Windows OSes, Snowflake CLI doesn’t show warnings when config file permissions are too wide for Windows systems, improved snow connection test error messages when a role, warehouse, or database does not exist)
PHP PDO Driver for Snowflake 2.0.3 (issue where timeout values in connection parameters weren’t honored)
ODBC Driver 3.3.0 (Fixed performance regression on Windows when using SQLGetData to retrieve the query results, fixed an issue where SQLProcedures() didn’t list stored procedures that returned a table without a column definition, upgraded the compiler on Windows and added more build flags to address security concerns, fixed an issue where memory allocation failures could cause applications to terminate with a potential resource leak, fixed an issue relating to out of bounds memory access errors, removed CRT functions banned by Microsoft due to security concerns, fixed an issue where using key pair authentication failed when a private key file path contained non-ASCII characters)
Node.js Driver 1.10.1 (fixed unhandled promise rejections on keypair authorization, fixed an issue with reading a timestamp type with high precision, fixed external browser authentication, fixed an issue with native Okta URL validation, fixed the data format in bulk upload .csv files, fix validation for short account names, axios to version 1.6.8)
JDBC Driver 3.15.1 (Moved the public suffix list to an internal package when shading, fixed an issue with ignoring default GCS credentials, fixed an issue with returning decimal or integer values in ARROW format, fixed an issue where the driver returned java.util.ConcurrentModificationException while calling SFAsyncResultSet.next, fixed an InvalidPathException issue on Windows due to nested file paths)
JDBC Driver 3.16.0 (fixed an issue with native OKTA retry logic, fixed an issue with unsupported reserved keywords, fixed an issue with retry attempts for GET query metadata requests)

Conclusion

To put things in perspective for April, it took 2,400 words to describe all releases I could find for Snowflake this month. I did not cover 100% of things. To say Snowflake is trying to generally available many features is wishing thinking, and they are furiously going at it. What are they doing during the summit? Will there be any new announcements this year, or will we focus on learning to use all these beautiful features?

Worthy mentioned among many that Dynamic Tables, authentication enhancements, clean rooms, and new regions made it to GA. While Git integration and copilot went to Public Preview. I am going to try these soon and write an article.

I hope you continue to enjoy these articles.

I am Augusto Rosa, VP of Engineering for Infostrux Solutions. Snowflake Data Super Hero and SME. Thanks for reading my blog post. You can follow me on LinkedIn.

Subscribe to Infostrux Medium Blogs https://medium.com/infostrux-solutions for the most interesting Data Engineering and Snowflake news.

The Unofficial Snowflake Monthly Release Notes: April 2024