Skip to content

feat: SQL Server 2025 support (plus as_columnstore docs and a store-failures regression test)#728

Open
joshmarkovic wants to merge 8 commits into
dbt-msft:masterfrom
joshmarkovic:feat/sql-server-2025
Open

feat: SQL Server 2025 support (plus as_columnstore docs and a store-failures regression test)#728
joshmarkovic wants to merge 8 commits into
dbt-msft:masterfrom
joshmarkovic:feat/sql-server-2025

Conversation

@joshmarkovic

Copy link
Copy Markdown
Contributor

Summary

This branch bundles three independent changes. Happy to split into separate PRs if that is easier to review.

1. SQL Server 2025 support (703862b)

  • Cover SQL Server 2025 in the integration-test matrix (pyodbc and mssql-python on Python 3.13 / ODBC Driver 18) and add it to the published server CI images.
  • Rename the server.Dockerfile build arg SQLServer_VERSION to MSSQL_VERSION so it matches the build-arg the workflow already passes. With the old name the matrix value was ignored and every server-* image was built from the 2022 default, so the 2017/2019 (and now 2025) images were silently all 2022.
  • Document 2025 as a supported version and refresh the stale dbt-core compatibility note (0.14 to 1.10).

2. Document the as_columnstore config (391080c, refs #440)

The as_columnstore config (default true, which builds a clustered columnstore index) was undocumented. This adds a README section, including the workaround for test failure storage: audit tables can contain (n)varchar(max) columns, which SQL Server does not allow in a columnstore index, so as_columnstore: false on those resources avoids the build error.

3. Regression test for store-failures on passing tests (7d8e90b, refs #601)

A passing test run with --store-failures should leave an empty audit table, not drop it (the documented dbt behavior, and what Postgres does). This adds a functional test asserting the audit relation persists as an empty user table across idempotent re-runs.

This closes a coverage gap: the existing test_store_test_failures.py suite only asserted the passing case for view-based store-failures, never the table-based case from #601.

Verified locally against SQL Server 2022 with a case-sensitive collation: the new test passes, and the existing 6-test store-failures suite still passes. The reported behavior no longer reproduces on current master, so the test serves as a regression guard.

joshmarkovic and others added 7 commits June 24, 2026 19:58
Cover SQL Server 2025 in the integration-test matrix (pyodbc and mssql-python on Python 3.13 / ODBC Driver 18) and add it to the published server CI images.

Rename the server Dockerfile build arg SQLServer_VERSION to MSSQL_VERSION to match the build-arg the workflow already passes; with the old name the matrix version was ignored and every image used the 2022 default.

Document 2025 as a supported version and refresh the stale dbt-core 0.14 compatibility note to 1.10.
Explain that table materializations build a clustered columnstore index by default, and that as_columnstore: false is required for tables with (n)varchar(max)/LOB columns such as dbt store_failures audit tables.
)

A passing test run with --store-failures must replace prior failures with an empty audit table rather than drop it. Adds a functional regression test for dbt-msft#601.
Co-authored-by: Axell Padilla <68310020+axellpadilla@users.noreply.github.com>
Addresses @axellpadilla's review on dbt-msft#721.

- integration tests: make 2025 the matrix baseline, replacing 2022 as the
  latest tier. Keep 2017, 2019 and 2022 as single legacy rows so all four
  versions stay covered by CI.
- README: remove the duplicate "Supported SQL Server versions" heading and
  the Azure SQL Database / Managed Instance rows that were marked tested.
  Azure is now described as not covered by CI but expected to be compatible.
The index feature emits OPTIMIZE_FOR_SEQUENTIAL_KEY and RESUMABLE (plus
RESUMABLE's MAX_DURATION) unconditionally. These are only recognized on
SQL Server 2019 (major 15)+, so on a genuine 2017 engine the CREATE INDEX
fails with "is not a recognized CREATE INDEX option" (msg 155).

This was masked until now: the server-2017/2019/2022 CI images were all
built FROM ...:2022-latest because publish-docker passed build-arg
MSSQL_VERSION while server.Dockerfile declared SQLServer_VERSION. This PR
fixes that arg name, so the 2017 leg finally runs real 2017 and exposed
the incompatibility.

We support 2017 (Microsoft still does), so detect the engine major
version and, when < 15, drop these options (with a warning) and build the
index without them instead of failing. ONLINE is kept (recognized on
2017, edition-gated not version-gated). Behavior on 2019+ is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joshmarkovic

joshmarkovic commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

The only failing job was the SQL Server 2017 leg, and it was exposed (not caused) by this PR.

Root cause: before this PR, devops/server.Dockerfile declared ARG SQLServer_VERSION while publish-docker.yml builds the images with build-args: MSSQL_VERSION=.... The names didn't match, so Docker silently ignored the build arg and every server-20xx image was built FROM mcr.microsoft.com/mssql/server:2022-latest. In other words, CI's "2017" leg has never actually run SQL Server 2017.

Proof that the old "2017" tests ran on a 2022 engine: master's most recent SQL2017 job reports Microsoft SQL Server 2022 (RTM-CU25) ... 16.0.4255.1, see run 28142611656, the SQL2017 job (search the log for Microsoft SQL Server). After this PR corrects the arg name and the images were rebuilt, the 2017 leg finally runs a genuine 2017 (14.0.3530.2), see the failing job on this PR.

On a real 2017 engine, two existing index tests failed because the create-index macro emitted options that only SQL Server 2019+ recognizes:

  • test_full_option_surface'optimize_for_sequential_key' is not a recognized CREATE INDEX option (155)
  • test_table_create_path_post_commit'resumable' is not a recognized CREATE INDEX option (155)

Fix (pushed): we support 2017 (Microsoft still does), so I gated those options on the detected engine major version. On versions below 2019 (major < 15), optimize_for_sequential_key, resumable and resumable's max_duration are dropped (with a warning) and the index is built without them instead of failing. ONLINE is kept (it's recognized on 2017; it's edition-gated, not version-gated). Behavior on 2019+ is unchanged, and both tests now pass on 2017 with no test changes.

@axellpadilla flagging for visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant