build2.proj All the Things

.github/copilot-instructions.md

@@ -11,17 +11,18 @@
 
 ## 📚 Project Overview
 This project is a .NET data provider for SQL Server, enabling .NET applications to interact with SQL Server databases. It supports various features like connection pooling, transaction management, and asynchronous operations.
-The project builds from a **single unified project** at `src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj` that multi-targets `net462`, `net8.0`, and `net9.0`. The legacy `netfx/` and `netcore/` directories are being phased out — only their `ref/` folders (which define the public API surface) remain active.
+The project builds from a **single unified project** at `src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj`. It targets `net8.0` and `net9.0` on all supported hosts, and adds `net462` only when building on Windows. The legacy `netfx/` and `netcore/` directories are being phased out — only their `ref/` folders (which define the public API surface) remain active.
 The project includes:
 - **Public APIs**: Defined in `netcore/ref/` and `netfx/ref/` directories.
 - **Implementations**: All source code in `src/Microsoft.Data.SqlClient/src/`.
 - **Tests**: Located in the `tests/` directory, covering unit and integration tests.
-  - **Unit Tests**: Located in `tests/UnitTests/` directory, which includes tests for individual components and methods.
-  - **Functional Tests**: Located in `tests/FunctionalTests/` directory, which includes tests for various features and functionalities that can be run without a SQL Server instance.
-  - **Manual Tests**: Located in `tests/ManualTests/` directory, which includes tests that require a SQL Server instance to run.
+  - **Unit Tests**: Located in `src/Microsoft.Data.SqlClient/tests/UnitTests/`.
+  - **Functional Tests**: Located in `src/Microsoft.Data.SqlClient/tests/FunctionalTests/`.
+  - **Manual Tests**: Located in `src/Microsoft.Data.SqlClient/tests/ManualTests/`.
+  - **Performance/Stress Tests**: Located in `src/Microsoft.Data.SqlClient/tests/PerformanceTests/` and `src/Microsoft.Data.SqlClient/tests/StressTests/`.
 - **Documentation**: Found in the `doc/` directory, including API documentation, usage examples.
 - **Policies**: Contribution guidelines, coding standards, and review policies in the `policy/` directory.
-- **Building**: The project uses MSBuild for building and testing, with configurations and targets defined in the `build.proj` file, whereas instructions are provided in the `BUILDGUIDE.md` file.
+- **Building**: The repo uses `build.proj` for orchestrated build/test/pack workflows, `src/Microsoft.Data.SqlClient.slnx` for solution-centric development tooling, and Azure DevOps YAML under `eng/pipelines/` plus `eng/pipelines/onebranch/` for CI and official release flows. See `BUILDGUIDE.md` for local build details.
 - **CI/CD**: ADO Pipelines for CI/CD and Pull request validation are defined in the `eng/` directory, ensuring code quality and automated testing.
 
 ## 📦 Products

.github/instructions/ado-pipelines.instructions.md

@@ -22,14 +22,14 @@ Top-level CI/PR pipeline files:
 - `dotnet-sqlclient-ci-project-reference-pipeline.yml` — CI with Project references (Release)
 - `sqlclient-pr-package-ref-pipeline.yml` — PR validation with Package references
 - `sqlclient-pr-project-ref-pipeline.yml` — PR validation with Project references
-- `stress-tests-pipeline.yml` — Stress tests triggered after successful CI-Package runs
+- `stress/stress-tests-pipeline.yml` — Stress test pipeline and templates
 
 Reusable templates are organized under:
 - `common/templates/jobs/` — Job templates (`ci-build-nugets-job`, `ci-code-coverage-job`, `ci-run-tests-job`)
 - `common/templates/stages/` — Stage templates (`ci-run-tests-stage`)
 - `common/templates/steps/` — Step templates (build, test, config, publish)
 - `jobs/` — Package-specific CI jobs (pack/test Abstractions, Azure, Logging, stress)
-- `stages/` — Package-specific CI stages (build Logging → Abstractions → SqlClient → Azure → verify → stress)
+- `stages/` — Package-specific CI stages (generate secrets, build SqlServer/Logging/Abstractions/SqlClient/Azure, verify packages)
 - `libraries/` — Shared variables (`ci-build-variables.yml`)
 - `steps/` — SDK install steps
 
@@ -43,25 +43,28 @@ Key parameters:
 - `testJobTimeout` (required) — test job timeout in minutes
 - `targetFrameworks` — Windows test TFMs; default `[net462, net8.0, net9.0, net10.0]`
 - `targetFrameworksUnix` — Unix test TFMs; default `[net8.0, net9.0, net10.0]`
+- `netcoreVersionTestUtils` — default runtime for shared test utilities; default `net10.0`
 - `testSets` — test partitions; default `[1, 2, 3]`
 - `useManagedSNI` — SNI variants to test; default `[false, true]`
 - `runAlwaysEncryptedTests` — include AE test set; default `true`
-- `enableStressTests` — enable stress test stage; default `false`
+- `runLegacySqlTests` — include SQL Server 2016/2017 manual-test legs; default `true`
 - `debug` — enable debug output; default `false`
 - `dotnetVerbosity` — MSBuild verbosity; default `normal`
 
 ## Build Stage Order
 
 Stages execute in dependency order (Package reference mode requires artifacts from prior stages):
 1. `generate_secrets` — Generate test secrets
-2. `build_logging_package_stage` — Build Logging package
-3. `build_abstractions_package_stage` — Build Abstractions (depends on Logging)
-4. `build_sqlclient_package_stage` — Build SqlClient + AKV Provider (depends on Abstractions + Logging)
-5. `build_azure_package_stage` — Build Azure extensions (depends on Abstractions + Logging + SqlClient)
-6. `verify_nuget_packages_stage` — Verify NuGet package metadata
-7. `stress_tests_stage` — Optional stress tests
+2. `build_sqlserver_package_stage` — Build `Microsoft.SqlServer.Server`
+3. `build_logging_package_stage` — Build Logging package
+4. `build_abstractions_package_stage` — Build Abstractions (package mode depends on Logging)
+5. `build_sqlclient_package_stage` — Build SqlClient and produce the AKV provider package
+6. `build_azure_package_stage` — Build Azure extensions
+7. `verify_nuget_packages_stage` — Verify NuGet package metadata
 8. `ci_run_tests_stage` — Run MDS and AKV test suites
 
+Stress testing is no longer a stage threaded through `dotnet-sqlclient-ci-core.yml`; it lives under `eng/pipelines/stress/` as a separate pipeline flow.
+
 When adding a new build stage, respect the dependency graph and pass artifact names/versions to downstream stages.
 
 ## PR vs CI Pipeline Differences
@@ -70,12 +73,12 @@ PR pipelines:
 - Trigger on PRs to `dev/*`, `feat/*`, `main`; exclude `eng/pipelines/onebranch/*` paths
 - Use reduced TFM matrix: `[net462, net8.0, net9.0]` (excludes net10.0)
 - Timeout: 90 minutes
-- Package-ref PR disables Always Encrypted tests in Debug config
+- Package-ref PR disables Always Encrypted tests in Debug config and also disables legacy SQL Server test legs to keep validation fast
 
 CI pipelines:
 - Trigger on push to `main` (GitHub) and `internal/main` (ADO) with `batch: true`
 - Scheduled weekday builds (see individual pipeline files for cron times)
-- Full TFM matrix including net10.0
+- Full TFM matrix including net10.0 test legs and legacy SQL Server manual-test coverage
 
 ## Test Configuration
 
@@ -97,7 +100,7 @@ Flaky test quarantine:
 
 SNI testing — `useManagedSNI` controls testing with native SNI (`false`) or managed SNI (`true`)
 
-Test timeout — `--blame-hang-timeout 10m` (configured in `build.proj`); tests exceeding 10 minutes are killed
+Test timeout — `--blame-hang-timeout 10m` (configured in `build.proj` and threaded through CI test steps); tests exceeding 10 minutes are killed
 
 ## Variables
 

.github/instructions/onebranch-pipeline-design.instructions.md

@@ -34,7 +34,7 @@ Defined in `stages/build-stages.yml`. Four build stages plus validation, ordered
 - **`build_abstractions`** (Stage 2) — Abstractions; `dependsOn: build_independent`; downloads Logging artifact
 - **`build_dependent`** (Stage 3) — SqlClient and Extensions.Azure in parallel; `dependsOn: build_abstractions`; downloads Abstractions + Logging artifacts
 - **`build_addons`** (Stage 4) — AKV Provider; `dependsOn: build_dependent`; downloads SqlClient + Abstractions + Logging artifacts
-- **`mds_package_validation`** — Validates signed SqlClient package; `dependsOn: build_dependent`; runs in parallel with Stage 4
+- **`sqlclient_package_validation`** — Validates signed SqlClient package; `dependsOn: build_dependent`; runs in parallel with Stage 4
 
 Each build job copies PDB files into `$(JOB_OUTPUT)/symbols/` so they are included in the auto-published pipeline artifact alongside the NuGet packages in `$(JOB_OUTPUT)/packages/`.
 
@@ -47,17 +47,16 @@ Stage conditional rules:
 
 ## Job Templates
 
-- **`build-signed-csproj-package-job.yml`** — Generic job for csproj-based packages (Logging, SqlServer.Server, Abstractions, Azure, AKV Provider). Flow: Build DLLs → ESRP DLL signing → NuGet pack (`NoBuild=true`) → ESRP NuGet signing → Copy PDBs to artifact
-- **`build-signed-sqlclient-package-job.yml`** — SqlClient-specific job (nuspec-based). Flow: Build all configurations → ESRP DLL signing (main + resource DLLs) → NuGet pack via nuspec → ESRP NuGet signing → Copy PDBs to artifact
+- **`build-buildproj-job.yml`** — Shared build.proj-driven package job used for all shipped packages. Flow: build via `build.proj` → optional ESRP DLL signing → pack via `build.proj` → optional ESRP NuGet signing → copy outputs for APIScan/artifacts
 - **`validate-signed-package-job.yml`** — Validates signed MDS package (signature, strong names, folder structure, target frameworks)
 - **`publish-nuget-package-job.yml`** — Reusable release job using OneBranch `templateContext.type: releaseJob` with `inputs` for artifact download; pushes via `NuGetCommand@2`
 - **`publish-symbols-job.yml`** — Reusable symbols job: downloads a build artifact, locates PDBs under `symbols/`, and invokes `publish-symbols-step.yml`
 
-When adding a new csproj-based package:
-- Use `build-signed-csproj-package-job.yml` with appropriate `packageName`, `packageFullName`, `versionProperties`, and `downloadArtifacts`
-- Add build and pack targets to `build.proj`
+When adding a new package to the OneBranch flow:
+- Extend `build-buildproj-job.yml` inputs with the new package metadata and dependency artifacts
+- Add or update the corresponding build/pack targets in `build.proj`
 - Add version variables to `variables/common-variables.yml`
-- Add artifact name variable to `variables/onebranch-variables.yml`
+- Add artifact name variables to `variables/onebranch-variables.yml`
 
 ## Symbols Publishing Stage
 
@@ -124,8 +123,7 @@ Variable groups:
 
 - Uses ESRP v6 tasks (`EsrpMalwareScanning@6`, `EsrpCodeSigning@6`) with MSI/federated identity authentication
 - Signing only runs when `isOfficial: true` — non-official pipelines skip ESRP steps
-- csproj-based packages: sign DLLs first → pack with `NoBuild=true` → sign NuGet package (ensures NuGet contains signed DLLs)
-- SqlClient: sign DLLs (including resource DLLs) → nuspec pack → sign NuGet package
+- The shared OneBranch job signs DLLs before packing and signs the resulting NuGet package afterward so the published package contains signed binaries
 - DLL signing uses keyCode `CP-230012` (Authenticode); NuGet signing uses keyCode `CP-401405`
 - All ESRP credentials come from variable groups — never hardcode secrets in YAML
 

.github/prompts/ado-work-item-agent.prompt.md

@@ -23,7 +23,7 @@ Perform the following steps to address the work item. Think step-by-step.
 - Locate the relevant code in `src/` or `tests/`.
 
 ### 2. Planning and Branching
-- Propose a descriptive branch name following the pattern `dev/username/branch-name` (e.g., `dev/jdoe/fix-connection-pool`).
+- Propose a descriptive branch name following the repository rule `dev/automation/<branch-name>` (e.g., `dev/automation/fix-connection-pool`).
 - Identify any dependencies or potential breaking changes.
 
 ### 3. Implementation
@@ -33,13 +33,13 @@ Perform the following steps to address the work item. Think step-by-step.
 
 ### 4. Testing and Verification
 - **Mandatory**: All changes must be tested.
-- Create new unit tests in `tests/UnitTests` or functional tests in `tests/FunctionalTests` as appropriate.
+- Create new unit tests in `src/Microsoft.Data.SqlClient/tests/UnitTests` or functional tests in `src/Microsoft.Data.SqlClient/tests/FunctionalTests` as appropriate.
 - Verify that the tests pass.
 
 ### 5. Documentation and Finalization
 - If public APIs are modified, update the documentation in `doc/`.
 - Provide a clear summary of changes for the Pull Request.
-- Suggest an entry for [CHANGELOG.md](CHANGELOG.md) if the change is significant.
+- Suggest a release-note entry under `release-notes/` or in the PR description if the change is significant; do not edit `CHANGELOG.md` directly.
 
 ## Input
 **Work Item ID**: ${input:workItemId}

.github/prompts/fix-bug.prompt.md

@@ -25,9 +25,9 @@ Follow this workflow step-by-step:
 ## 3. Write a Failing Test
 - Create a test that reproduces the bug BEFORE implementing the fix.
 - Choose the correct test project:
-  - `tests/UnitTests/` — for isolated logic tests (no SQL Server needed)
-  - `tests/FunctionalTests/` — for API behavior tests (no SQL Server needed)
-  - `tests/ManualTests/` — for integration tests (requires SQL Server)
+  - `src/Microsoft.Data.SqlClient/tests/UnitTests/` — for isolated logic tests (no SQL Server needed)
+  - `src/Microsoft.Data.SqlClient/tests/FunctionalTests/` — for API behavior tests (no SQL Server needed)
+  - `src/Microsoft.Data.SqlClient/tests/ManualTests/` — for integration tests (requires SQL Server)
 - Follow existing naming conventions: `{ClassName}Tests.cs` with methods named `{MethodName}_{Scenario}_{ExpectedResult}`.
 - If the bug is platform-specific, add appropriate `[ConditionalFact]` or `[ConditionalTheory]` attributes with `[PlatformSpecific]`.
 - **Cover both sync and async code paths** if the affected API has both variants (e.g., `Open`/`OpenAsync`, `ExecuteReader`/`ExecuteReaderAsync`). Sync and async paths often have different internal implementations and a bug may manifest in only one.

.github/prompts/implement-feature.prompt.md

@@ -62,9 +62,9 @@ Before writing code, produce a brief implementation plan covering:
 4. Test against multiple SQL Server versions.
 
 ## 5. Write Tests
-- **Unit tests** in `tests/UnitTests/` for isolated logic.
-- **Functional tests** in `tests/FunctionalTests/` for API behavior without SQL Server.
-- **Manual tests** in `tests/ManualTests/` for full integration with SQL Server.
+- **Unit tests** in `src/Microsoft.Data.SqlClient/tests/UnitTests/` for isolated logic.
+- **Functional tests** in `src/Microsoft.Data.SqlClient/tests/FunctionalTests/` for API behavior without SQL Server.
+- **Manual tests** in `src/Microsoft.Data.SqlClient/tests/ManualTests/` for full integration with SQL Server.
 - Cover:
   - Happy path and edge cases
   - **Both sync and async code paths** where the feature exposes both variants

.github/prompts/release-notes.prompt.md

@@ -28,7 +28,7 @@ Each package has its own versioning and dependency sources. Use these to determi
 | Package | Version Source | Dependency Source |
 |---------|---------------|-------------------|
 | `Microsoft.Data.SqlClient` | [tools/props/Versions.props](tools/props/Versions.props) (`MdsVersionDefault`) | [Directory.Packages.props](Directory.Packages.props) and the [project file](src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj) |
-| `AzureKeyVaultProvider` | [tools/props/Versions.props](tools/props/Versions.props) (`AkvVersionDefault`) | [AKV project file](src/Microsoft.Data.SqlClient/add-ons/AzureKeyVaultProvider/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.csproj) and [Directory.Packages.props](Directory.Packages.props) |
+| `AzureKeyVaultProvider` | [tools/props/Versions.props](tools/props/Versions.props) (`AkvVersionDefault`) | [AKV project file](src/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider/src/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.csproj) and [Directory.Packages.props](Directory.Packages.props) |
 | `Microsoft.SqlServer.Server` | [tools/props/Versions.props](tools/props/Versions.props) (`SqlServerPackageVersion`) | [SqlServer project file](src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj) |
 | `Extensions.Abstractions` | [AbstractionsVersions.props](src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/AbstractionsVersions.props) | [Abstractions.csproj](src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj) |
 | `Extensions.Azure` | [AzureVersions.props](src/Microsoft.Data.SqlClient.Extensions/Azure/src/AzureVersions.props) | [Azure.csproj](src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj) |

.github/prompts/update-build-pipelines.prompt.md

@@ -15,35 +15,40 @@ Follow this workflow step-by-step:
 ## 1. Understand the Pipeline Architecture
 - Read the relevant pipeline file(s) in `eng/pipelines/`.
 - Key pipelines:
-  - `dotnet-sqlclient-ci-core.yml` — Core CI pipeline (reusable by reference pipelines)
+  - `dotnet-sqlclient-ci-core.yml` — Core CI pipeline template used by CI and PR definitions
   - `dotnet-sqlclient-ci-project-reference-pipeline.yml` — CI with project references
   - `dotnet-sqlclient-ci-package-reference-pipeline.yml` — CI with package references
   - `sqlclient-pr-project-ref-pipeline.yml` — PR validation (project references)
   - `sqlclient-pr-package-ref-pipeline.yml` — PR validation (package references)
-  - `dotnet-sqlclient-signing-pipeline.yml` — Package signing
-  - `akv-official-pipeline.yml` — AKV provider official build (1ES/OneBranch)
-  - `stress-tests-pipeline.yml` — Stress tests
+  - `onebranch/sqlclient-official.yml` — official OneBranch build/release pipeline
+  - `onebranch/sqlclient-non-official.yml` — non-official OneBranch build/release pipeline
+  - `stress/stress-tests-pipeline.yml` — stress test pipeline
 - Shared templates live in `eng/pipelines/common/templates/` (jobs/, stages/, steps/).
-- Variables are defined in `eng/pipelines/variables/` and `eng/pipelines/libraries/`.
+- CI variables are defined in `eng/pipelines/libraries/`; OneBranch variables are defined in `eng/pipelines/onebranch/variables/`.
 
 ## 2. Identify What Needs to Change
 - Determine which pipeline files are affected.
 - Check if the change impacts shared templates that are reused across multiple pipelines.
 - Identify if new parameters, variables, or stages need to be added.
 - Review existing parameters to understand the current configuration surface:
-  - `targetFrameworks` / `targetFrameworksUnix` — test target frameworks
+  - `targetFrameworks` / `targetFrameworksUnix` — Windows and Unix test TFMs
+  - `netcoreVersionTestUtils` — runtime used by shared test utilities
   - `referenceType` — Project or Package reference
   - `buildConfiguration` — Debug/Release
   - `useManagedSNI` — Managed vs Native SNI testing
+  - `runLegacySqlTests` — whether to include SQL Server 2016/2017 legs
 
 ## 3. Implement the Change
 - Modify YAML files following the existing patterns and indentation style.
 - When adding new stages, follow the existing stage ordering:
-  1. `build_abstractions_package_stage`
-  2. `build_sqlclient_package_stage`
-  3. `build_azure_package_stage`
-  4. `stress_tests_stage` (optional)
-  5. `run_tests_stage`
+  1. `generate_secrets`
+  2. `build_sqlserver_package_stage`
+  3. `build_logging_package_stage`
+  4. `build_abstractions_package_stage`
+  5. `build_sqlclient_package_stage`
+  6. `build_azure_package_stage`
+  7. `verify_nuget_packages_stage`
+  8. `ci_run_tests_stage`
 - When adding new test parameters, ensure they are wired through to test execution steps.
 - When modifying shared templates, verify all consuming pipelines still work.
 
@@ -52,6 +57,7 @@ Follow this workflow step-by-step:
 - Test filters by platform: `nonnetfxtests`, `nonnetcoreapptests`, `nonwindowstests`, `nonlinuxtests`.
 - SNI testing matrix: both Native (`useManagedSNI=false`) and Managed (`useManagedSNI=true`).
 - Always Encrypted tests controlled by `runAlwaysEncryptedTests` parameter.
+- Stress coverage is maintained under `eng/pipelines/stress/`, not as a stage inside `dotnet-sqlclient-ci-core.yml`.
 
 ## 5. Validate
 - Verify YAML syntax is valid.