Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 15 additions & 10 deletions .github/workflows/integration-tests-sqlserver.yml
Comment thread
axellpadilla marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -43,31 +43,31 @@ jobs:
matrix:
python_version: ["3.10", "3.11", "3.12", "3.13"]
backend: [pyodbc, mssql-python]
# Baseline on 2022
sqlserver_version: ["2022"]
# Baseline on 2025 (newest)
sqlserver_version: ["2025"]
msodbc_version: ["18"]
collation: [SQL_Latin1_General_CP1_CI_AS]
exclude:
- backend: mssql-python
python_version: "3.10"
sqlserver_version: "2022"
sqlserver_version: "2025"
- backend: mssql-python
python_version: "3.11"
sqlserver_version: "2022"
sqlserver_version: "2025"
- backend: mssql-python
python_version: "3.12"
sqlserver_version: "2022"
sqlserver_version: "2025"
include:
# Keep pyodbc on every supported Python version, but retain
# SQL Server ODBC 17 coverage for the oldest and newest Python.
- backend: pyodbc
python_version: "3.10"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Older SQL Server versions stay on pyodbc only, with a single
Expand All @@ -82,17 +82,22 @@ jobs:
sqlserver_version: "2019"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Add the case-sensitive collation only on the latest SQL Server
# and latest Python/backend rows.
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2022"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CI_AS
# Add the case-sensitive collation on the SQL Server 2025 baseline
# and latest Python/backend rows.
- backend: pyodbc
python_version: "3.13"
sqlserver_version: "2025"
msodbc_version: "17"
collation: SQL_Latin1_General_CP1_CS_AS
# mssql-python stays on the latest Python only.
- backend: mssql-python
python_version: "3.13"
sqlserver_version: "2022"
sqlserver_version: "2025"
msodbc_version: "18"
collation: SQL_Latin1_General_CP1_CS_AS
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
publish-docker-server:
strategy:
matrix:
mssql_version: ["2017", "2019", "2022"]
mssql_version: ["2017", "2019", "2022", "2025"]
runs-on: ubuntu-latest
permissions:
contents: read
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- Add `drop_unmanaged_indexes` config (`false` (default) / `warn` / `true`) for indexes dbt didn't create.
- Validate cross-index config conflicts (multiple clustered indexes, clustered vs `as_columnstore`).
- Document the minimum supported SQL Server version (2017). Partitioning, `XML_COMPRESSION` and ordered columnstore are not yet expressible in the `indexes` config.
- Add SQL Server 2025 to the integration-test matrix (pyodbc and `mssql-python` backends, ODBC Driver 18) and document it as a supported version.
- Add `dbt_sqlserver_enable_safe_type_expansion` behaviour flag to allow safe column type widening during schema expansion: `varchar` → `nvarchar`, integer family promotions (`bit` → `tinyint` → `smallint` → `int` → `bigint`), and `numeric`/`decimal` precision/scale upgrades. Gated by the per-model `column_type_expansion_max_rows` config (default 1,000,000 rows). [#699](https://github.com/dbt-msft/dbt-sqlserver/issues/699).
- Add `prefer_single_alter_column` model config to use a single `ALTER COLUMN` statement instead of the add+update+drop+rename pattern when altering column types on tables.
- Add `string_type_instance()` to preserve the NVARCHAR/NCHAR type family during column expansion, fixing incorrect promotion of NVARCHAR/NCHAR to VARCHAR.
Expand Down
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,23 @@

[dbt](https://www.getdbt.com) adapter for Microsoft SQL Server and Azure SQL services.

The adapter supports dbt-core 0.14 or newer and follows the same versioning scheme.
E.g. version 1.1.x of the adapter will be compatible with dbt-core 1.1.x.
The adapter supports dbt-core 1.10 or newer and follows the same versioning scheme.
E.g. version 1.10.x of the adapter will be compatible with dbt-core 1.10.x.

The minimum supported SQL Server version is SQL Server 2017.
## Supported SQL Server versions

The adapter is tested against the following SQL Server versions:

| SQL Server version | Supported |
|---|---|
| SQL Server 2017 | ✅ (minimum supported version) |
| SQL Server 2019 | ✅ |
| SQL Server 2022 | ✅ |
| SQL Server 2025 | ✅ |

The minimum supported SQL Server version is SQL Server 2017; older versions are not supported.

SQL Server 2017, 2019, 2022, and 2025 are covered by the integration test suite. Azure SQL Database and Azure SQL Managed Instance are not covered by the integration test suite, but are expected to be compatible.

## Documentation

Expand Down Expand Up @@ -180,6 +193,29 @@ flags:

**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.

### `as_columnstore`

*(default: `true`)* When building a table, the adapter creates a [clustered columnstore index](https://learn.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview) (CCI) on it. Set `as_columnstore: false` to build a plain rowstore table instead.

This matters for any table containing a `(n)varchar(max)` or other LOB column, because SQL Server does not allow those data types to participate in a columnstore index. The table build fails with:

> Column '...' has a data type that cannot participate in a columnstore index.

A common case is dbt's [test failure storage](https://docs.getdbt.com/reference/resource-configs/store_failures): the audit tables can contain `VARCHAR(MAX)` columns (dbt's `STRING` type maps to `VARCHAR(MAX)`), so disable the CCI on those resources:

```yaml
# dbt_project.yml
data_tests:
+store_failures: true
+as_columnstore: false # avoids CCI on (n)varchar(max) audit columns
```

You can also set it per model:

```sql
{{ config(materialized="table", as_columnstore=false) }}
```

## Contributing

[![Unit tests](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml)
Expand Down
4 changes: 2 additions & 2 deletions devops/server.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ARG SQLServer_VERSION="2022"
FROM mcr.microsoft.com/mssql/server:${SQLServer_VERSION}-latest
ARG MSSQL_VERSION="2022"
FROM mcr.microsoft.com/mssql/server:${MSSQL_VERSION}-latest

ENV COLLATION="SQL_Latin1_General_CP1_CI_AS"

Expand Down
102 changes: 102 additions & 0 deletions tests/functional/adapter/mssql/test_store_failures_passing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# flake8: noqa: E501
"""Regression test for dbt-msft/dbt-sqlserver#601.

With ``--store-failures``, a test that *passes* must leave behind an empty
audit relation, not drop it. dbt's contract: "A test's results will always
replace previous failures for the same test, even if that test results in no
failures." The SQL Server adapter was reported to ``DROP`` the audit table on a
passing test instead of replacing it with an empty table (Postgres creates the
empty table).

This exercises the exact reported scenario: a passing test configured with
``store_failures`` materialized as a ``table``. It asserts the audit relation
exists, is a base table (not a view), is empty, and survives idempotent re-runs.
"""

import pytest

from dbt.tests.util import run_dbt

# the default audit schema (_dbt_test__audit) plus the test schema can exceed
# identifier limits; use a short suffix as the rest of the suite does.
TEST_AUDIT_SCHEMA_SUFFIX = "dbt_test__aud"

model__chipmunks = """
select 1 as id, 'alvin' as name
union all
select 2 as id, 'simon' as name
"""

# returns zero rows -> the test passes
test__passing_601 = """
{{ config(store_failures=true, store_failures_as='table') }}
select * from {{ ref('chipmunks') }}
where 1 = 2
"""


class TestStoreFailuresPassingKeepsEmptyTable:
@pytest.fixture(scope="class")
def models(self):
return {"chipmunks.sql": model__chipmunks}

@pytest.fixture(scope="class")
def tests(self):
return {"passing_601.sql": test__passing_601}

@pytest.fixture(scope="class")
def project_config_update(self):
return {
"vars": {"dbt_sqlserver_use_default_schema_concat": True},
"data_tests": {"+schema": TEST_AUDIT_SCHEMA_SUFFIX},
}

@pytest.fixture(scope="function", autouse=True)
def setup(self, project):
self.audit_schema = f"{project.test_schema}_{TEST_AUDIT_SCHEMA_SUFFIX}"
run_dbt(["run"])
yield
with project.adapter.connection_named("__test"):
relation = project.adapter.Relation.create(
database=project.database, schema=self.audit_schema
)
project.adapter.drop_schema(relation)

def _assert_empty_audit_table(self, project):
# type_desc proves the relation exists AND is a user table (not a view).
# On the #601 bug the relation is dropped, so this returns no rows.
# Queried via sys catalog (lowercase column names) so it is safe under a
# case-sensitive database collation.
rows = project.run_sql(
f"""
select o.type_desc
from sys.objects o
join sys.schemas s on o.schema_id = s.schema_id
where s.name = '{self.audit_schema}'
and o.name = 'passing_601'
""",
fetch="all",
)
assert len(rows) == 1 and rows[0][0] == "USER_TABLE", (
f"audit relation [{self.audit_schema}].[passing_601] should be a user "
f"table that persists after a passing store-failures run, got: "
f"{[tuple(r) for r in rows]}"
)
# and it must be empty (the failures were replaced with nothing)
count = project.run_sql(
f"select count(*) from [{self.audit_schema}].[passing_601]",
fetch="one",
)
assert count[0] == 0, f"audit table should be empty, has {count[0]} rows"

def test_passing_test_keeps_empty_audit_table(self, project):
results = run_dbt(["test", "--store-failures"], expect_pass=True)
assert len(results) == 1
assert results[0].status == "pass"
assert results[0].failures == 0
self._assert_empty_audit_table(project)

# idempotency: a second run must still leave the empty table in place
results = run_dbt(["test", "--store-failures"], expect_pass=True)
assert results[0].status == "pass"
self._assert_empty_audit_table(project)
Loading