[MINOR][CI][DOC] Document the Delta Spark UT pipeline and skip it for docs-only changes

[felipepessoto]

What changes are proposed in this pull request?

Two related follow-ups to #12388.

Docs. docs/developers/velox-backend-CI.md describes the Velox CI workflows but did not mention delta_spark_ut.yml. A contributor whose PR suddenly runs a multi-hour Delta suite had no entry point explaining what it is, or why it does not simply fail on a failing test. Adds a short section covering the baseline gate, when the job runs, and how to refresh the baseline.

CI. The Delta path filter matches util/delta-spark-ut/**, which includes the pipeline's own README, so fixing a typo there costs a full run -- 8 shards at roughly 75-100 minutes each, about 10 runner-hours, to test a change that cannot affect the suite. Markdown under that directory is now excluded. paths and paths-ignore cannot both be used, so this is a negated pattern, which must stay last because patterns are evaluated in order.

While here, both documents said the tests fail because Gluten "does not offload every Delta code path". That is not the reason: a plan Gluten cannot offload should fall back to vanilla Spark and the test should still pass. The failures are either real gaps -- fallback not happening where it should (#12538), metrics that differ from vanilla (#9003), native-side bugs (#12377, #12387) -- or expected rather than defects, where a test asserts on a query plan that legitimately differs once Gluten offloads. Both are described that way now.

How was this patch tested?

The described behaviour was checked against delta_spark_ut.yml on main: the schedule cron, the pull_request path filter, and the delta-spark-ut-known-failures artifact name. The new filter was simulated against real paths -- the README is skipped, while run-delta-tests.sh, known-failures.txt, the workflow itself and gluten-delta/** still trigger. This PR changes the workflow, so it exercises the suite once itself.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot CLI

[Copilot]

Pull request overview

Documents the existing Delta Spark UT GitHub Actions workflow in the Velox CI developer docs so contributors understand what the multi-hour Delta suite is, when it runs, and how its known-failures baseline gate works.

Changes:

• Add a “Delta Spark UT” section describing baseline-gated behavior (new failures vs now-passing baseline entries).
• Document workflow triggers (paths-filtered per-PR, nightly schedule, and manual workflow_dispatch).
• Add baseline refresh instructions and link to the pipeline README and follow-up tracker issue.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

docs/developers/velox-backend-CI.md:44

• The section says the workflow "fails only" on regressions or now‑passing baseline tests, but the gate also intentionally fails on data/infra errors (e.g., missing JUnit reports or corrupt/truncated TEST-*.xml) to avoid passing on partial/empty results. This “only” wording is too absolute and could mislead contributors debugging a red run.

`delta_spark_ut.yml` runs delta-io/delta's own `spark` test suite against a Gluten Velox bundle, so Gluten is validated against a real Delta release. A number of those tests fail today -- not because Gluten declines to offload a plan, which should fall back to vanilla Spark transparently, but because of real gaps: fallback not happening where it should, metrics that differ from vanilla, and native-side bugs. So the job does not gate on "any failure": it compares each run against a committed baseline of known failures in `.github/workflows/util/delta-spark-ut/known-failures.txt` and fails only on a **new** failure, or on a baseline test that starts **passing** (which means the baseline needs updating).

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/delta_spark_ut.yml:64

• The negated paths pattern looks like it won’t actually match Markdown files under subdirectories (or README.md) because ** only acts as a “globstar” when it’s a full path segment. With **.md, a README edit may still trigger the workflow, defeating the intent to skip docs-only changes.

      - '!.github/workflows/util/delta-spark-ut/**.md'

[felipepessoto]

Addressed the two suppressed review comments:

"fails only ... is too absolute" — correct, fixed in 015c5b5. The gate also exits non-zero when a run produced no usable results (no JUnit reports, a truncated TEST-*.xml, or fewer shards than expected), so someone debugging a red run shouldn't be hunting for a test failure that isn't there. The docs now say so.

"!...**.md won't match, because ** is only a globstar as a full path segment" — I don't think this one holds, so I've left the pattern as is. Per the filter pattern cheat sheet, ** "matches any character including slash (/)", and '**.js' is documented as matching index.js, js/index.js and src/js/app.js — so it isn't segment-restricted, and **.md matches both README.md in that directory and any nested Markdown.

Worth noting for reviewers what this does and doesn't buy: for pull_request, paths: is evaluated against the whole PR diff rather than the latest push, so the exclusion makes a docs-only PR free — it doesn't stop this PR from running the suite, since this PR also changes delta_spark_ut.yml. That run is expected and is worth having once, given the workflow itself changed.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/util/delta-spark-ut/README.md:129

• The README explains earlier that an unoffloadable plan should fall back to vanilla Spark and the test should still pass, so using “known fallback” here as an example reason for adding an expected failure is internally inconsistent/misleading. Consider rephrasing this example to focus on plan-assertion tests (legit plan differences once Gluten offloads) or intentionally-accepted tracked gaps/bugs.

- **You intentionally added a new expected failure** (e.g. a test that hits a
  known fallback or native-side gap). Add the exact `Suite#test` line(s) the gate prints under
  *Regressions* to `known-failures.txt`, ideally with a comment explaining why.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/delta_spark_ut.yml:64

• The negated path filter !.github/workflows/util/delta-spark-ut/**.md does not exclude Markdown files at arbitrary depth under that directory (it won’t match e.g. .github/workflows/util/delta-spark-ut/subdir/notes.md). If the intent is to skip all Markdown under the pipeline directory, use **/*.md so nested docs don’t trigger the expensive workflow.

      # Documentation can't change what the suite does, and this job costs ~10
      # runner-hours, so don't spend that on a README edit. Negated patterns are
      # evaluated in order and override the positives above, so this must stay
      # last. (`paths` and `paths-ignore` cannot both be used, hence the `!`.)
      - '!.github/workflows/util/delta-spark-ut/**.md'

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[felipepessoto]

Changed the pattern to **/*.md in 3777a64, though not quite for the stated reason.

Per the filter pattern cheat sheet, ** "matches any character including slash (/)", and '**.js' is documented as matching src/js/app.js — so **.md should already have matched at arbitrary depth.

That said, docs/**/*.md is the spelling the same table documents for "a file with a .md suffix anywhere in the docs directory", and its examples cover the top-level file (docs/README.md) as well as nested ones (docs/a/markdown/file.md). Since this pattern has now drawn two review comments, and a silent non-match would mean every README edit costs a full ~10 runner-hour run, the unambiguous spelling is worth more than the terse one. No behaviour change intended: Markdown under the directory is excluded, and run-delta-tests.sh, known-failures.txt and the workflow itself still trigger the suite.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[philo-he]

@felipepessoto, thanks for your continued efforts. Some minor comments. Please check if they make sense.

[philo-he]

Nit:

Can we remove "Documentation can't chang ...and any nested notes." for brevity?

[philo-he]

Nit:

All of this text sits on line 44, which is hard to work with in an editor. It also means a future one-word edit will show the entire line as changed in the diff. Generally it's better to wrap at 100-150 characters, or put each sentence on its own line. Same applies to the other long lines in this document. Thanks!