Bug: Fix start of week CI#558
Merged
Merged
Conversation
test.sh read the positional dbt_version arg into a local variable used only to pick the tox env name, but never assigned it to the DBT_VERSION env var that profiles.yml interpolates into the schema/dataset name. Every matrix job within a warehouse therefore wrote to the same schema (dbt_artifacts_test_commit__<sha>), and parallel cloud-warehouse jobs collided — BigQuery surfaced 409 Already Exists / 429 rate limits / "table not found" on supported versions 1.9 and 1.10. Now exports DBT_VERSION from the arg; removed the misleading ensure_dbt_version() default from _lib.sh. Invisible locally because we only ever ran one job at a time.
They can't run on the runner's Python 3.12 (distutils, removed in 3.12) and predate the microbatch strategy the fixture uses. Matrix trimmed 42 -> 15 slots (1.9/1.10/1.11/latest). CI-coverage change only: require-dbt-version stays >=1.3.0 (still installs), tox envs for old versions kept for local debugging. Formal support-floor bump remains a separate 3.0.0 decision. README note added; reasoning recorded in specs/ci-rework/README.md §12.3.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix Monday regression: schema collision bug + drop EOL dbt versions from CI
The first weekly Tier 3 regression against
mainfailed broadly. Triage of all 43 matrix jobs found two independent root causes — one real bug, one environmental. This PR fixes both.Root cause 1 —
DBT_VERSIONschema collision (real bug)This is why supported versions (BigQuery 1.9, 1.10) failed.
scripts/ci/test.shread the positionaldbt_versionarg into a local variable used only to pick the tox env name — but never assigned it to theDBT_VERSIONenv var thatprofiles.ymlinterpolates into the schema/dataset name._lib.sh'sensure_dbt_version()then defaulted it to empty.Result: every matrix job within a warehouse wrote to the same schema (
dbt_artifacts_test_commit__<sha>— note the empty version segment). Undermax-parallel: 8, parallel jobs on the shared cloud warehouses collided:BigQuery failed hardest (strict table semantics + rate limits); Snowflake survived this run on luck but was equally exposed. Container-isolated warehouses (postgres/trino/sqlserver) were unaffected — they use fixed schemas. Invisible locally because we only ever ran one job at a time.
Fix:
test.shnow doesexport DBT_VERSION="${dbt_version}"; removed the misleadingensure_dbt_version()helper. Each pinned version gets a distinct schema, so parallel jobs no longer collide.Root cause 2 — EOL versions can't run on the runner
Every failure on 1.3–1.8 is environmental, not a package bug:
distutils(removed in Python 3.12) → die atdbt clean.microbatchincremental strategy (a dbt 1.9 feature) that the test fixture uses.Keeping them green would require a legacy Python 3.11 lane plus per-version gates — tech debt for versions dbt Labs has already EOL'd.
Decision: drop dbt 1.3–1.8 from the CI matrix. Matrix trimmed 42 → 15 slots (1.9 / 1.10 / 1.11 / latest).
⚠ This is a CI-coverage change, NOT a support drop
require-dbt-versionis unchanged (>=1.3.0) — older-dbt consumers can still install the package; it's simply no longer verified each release.tox.inienvs for 1.3–1.8 are kept — a maintainer can still runtox -e integration_snowflake_1_5_0on a local Python 3.11 to debug a consumer report.require-dbt-version→>=1.9, removing the tox envs, public announcement) remains a separate 3.0.0 decision. This is Phase 1 (CI descope) brought forward, not a pre-emption of that.README.md("Supported dbt versions").Files changed
scripts/ci/test.shDBT_VERSIONfrom the version arg (the fix)scripts/ci/_lib.shensure_dbt_version()helper.github/workflows/release.ymlREADME.mdspecs/ci-rework/README.md