Snowflake: parse the EXTERNAL TABLE statement family - #20
Merged
Conversation
Snowflake's `CREATE EXTERNAL TABLE` grammar did not parse: the fork's `parse_create_external_table` is the Hive-shaped one and the Snowflake dialect never intercepted `CREATE EXTERNAL TABLE`, so realistic DDL died in the parser. This teaches the Snowflake dialect the whole external-table statement family, while leaving the Hive path (which other dialects — and Snowflake itself, for the `STORED AS … LOCATION '<path>'` form — rely on) untouched via a tail-shape fallback. Added under the Snowflake dialect: * `CREATE [OR REPLACE] EXTERNAL TABLE [IF NOT EXISTS]` with virtual column definitions (`<col> <type> AS <expr>`, parenthesised or bare), `LOCATION`, `FILE_FORMAT` (named and inline), `PATTERN`, `REFRESH_ON_CREATE`, `AUTO_REFRESH`, `PARTITION BY`, `PARTITION_TYPE`, `TABLE_FORMAT`, `AWS_SNS_TOPIC`, `COPY GRANTS`, tags, row-access policy and `COMMENT`. * `DROP EXTERNAL TABLE` and `DESC[RIBE] EXTERNAL TABLE` (new `ObjectType::ExternalTable` / `DescribeObjectType::ExternalTable`). * `ALTER EXTERNAL TABLE … ADD FILES / REMOVE FILES / SET AUTO_REFRESH / ADD PARTITION / DROP PARTITION`, alongside the existing `REFRESH`. New `CreateTable` fields (`pattern`, `refresh_on_create`, `partition_type`, `table_format`, `aws_sns_topic`) and `AlterTableOperation` variants carry the Snowflake-only clauses; `Display` round-trips every member back to the same AST. The `ADD PARTITION` column/value pairs use a dedicated `ExternalTablePartitionColumn` struct so the `visitor` derive is satisfied. Deferred/rejected members (`TABLE_FORMAT = DELTA`, `PARTITION_TYPE = USER_SPECIFIED`, `ADD`/`DROP PARTITION`) parse so they can be rejected downstream rather than aborting a batch in the parser. Also brings the branch to a green CI baseline: fills in several stale full `CreateTable` struct literals in the duckdb/mssql/postgres tests, applies `cargo fmt`, clears `clippy -D warnings`, and fixes two rustdoc errors.
wojpadlo
force-pushed
the
snowflake/external-tables
branch
from
August 12, 2026 12:45
1df3bbd to
275d301
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Teaches the Snowflake dialect the full
EXTERNAL TABLEstatement family, which previously died in the parser (the fork'sparse_create_external_tableis Hive-shaped and the Snowflake dialect never interceptedCREATE EXTERNAL TABLE). Support for the LocalStack Snowflake emulator (snowflake-rs LAV-1574 / ADR 074 §6).The Hive
CREATE EXTERNAL TABLE … STORED AS … LOCATION '<path>'form — which other dialects, and Snowflake itself, rely on — is preserved: the Snowflake parser inspects the post-column tail and hands off to the Hive grammar when it is Hive-shaped.What parses now (Snowflake dialect)
CREATE [OR REPLACE] EXTERNAL TABLE [IF NOT EXISTS]with virtual columns (<col> <type> AS <expr>, parenthesised or bare),LOCATION=@stage/…,FILE_FORMAT=(…)(inline andFORMAT_NAME),PATTERN,REFRESH_ON_CREATE,AUTO_REFRESH,PARTITION BY,PARTITION_TYPE,TABLE_FORMAT,AWS_SNS_TOPIC,COPY GRANTS,WITH TAG,WITH ROW ACCESS POLICY,COMMENT.DROP EXTERNAL TABLEandDESC[RIBE] EXTERNAL TABLE— newObjectType::ExternalTable/DescribeObjectType::ExternalTable.ALTER EXTERNAL TABLE … ADD FILES / REMOVE FILES / SET AUTO_REFRESH / ADD PARTITION / DROP PARTITION, alongside the existingREFRESH ['subpath'].Deferred/rejected members (
TABLE_FORMAT = DELTA,PARTITION_TYPE = USER_SPECIFIED,ADD/DROP PARTITION) deliberately parse so the downstream consumer can reject them with a Snowflake-shaped error instead of aborting the rest of a batch in the parser.Implementation notes
CreateTablefields (pattern,refresh_on_create,partition_type,table_format,aws_sns_topic) andAlterTableOperationvariants (AddFiles,RemoveFiles,SetAutoRefresh,AddExternalPartition,DropExternalPartition) carry the Snowflake-only clauses;Displayround-trips every member back to the same AST.AWS_SNS_TOPIC,PARTITION_TYPE,REFRESH_ON_CREATE,USER_SPECIFIED.CreateTablestruct literals in the duckdb/mssql/postgres test suites socargo testcompiles on this base.Tests
parse_snowflake_external_table_family_roundtripsexercises every member of the family through aDisplay→ reparse round trip. Fullcargo testis green.🤖 Generated with Claude Code