acceptance: gate templates/default-python classic-compute tests on UC#5153
Draft
shreyas-goenka wants to merge 2 commits intomainfrom
Draft
acceptance: gate templates/default-python classic-compute tests on UC#5153shreyas-goenka wants to merge 2 commits intomainfrom
shreyas-goenka wants to merge 2 commits intomainfrom
Conversation
The default-python template's pipeline resource defaults to a hive_metastore
catalog when no UC catalog is detected on the workspace (see
libs/template/templates/default/template/{{.project_name}}/resources/{{.project_name}}_etl.pipeline.yml.tmpl).
Several of the test workspaces (gcp-prod-is, aws-prod-is) have HMS
disabled, so deploys intermittently fail with:
Error: cannot create resources.pipelines.X_etl: Hive Metastore is not
enabled for this workspace; please publish to Unity Catalog. (400
INVALID_PARAMETER_VALUE)
Set RequiresUnityCatalog = true on integration_classic and
combinations/classic so they only run on UC-enabled workspaces, where
default_catalog resolves to a real UC catalog and the pipeline gets a
proper "catalog:" field. The serverless variant of combinations is
unaffected because the template hardcodes "catalog: main" for serverless
pipelines without UC.
Co-authored-by: Isaac
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Co-authored-by: Isaac
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
default-pythontemplate's pipeline resource falls back to ahive_metastorecatalog when no UC catalog is detected (template source). SetRequiresUnityCatalog = trueonintegration_classicandcombinations/classicso they only run on UC-enabled workspaces and the pipeline gets a realcatalog:field.The
serverlessvariant ofcombinationsis unaffected because the template hardcodescatalog: mainfor serverless pipelines without UC.Test plan
This pull request was AI-assisted by Isaac.