feat(kernel): route azure-oauth (Azure AD U2M) to the kernel OAuth U2M flow

On use_kernel=True, auth_type='azure-oauth' was rejected with a pointer to
databricks-oauth. Instead route it to the kernel's oauth-u2m, identically to
databricks-oauth: the kernel runs the in-house workspace-federated browser
flow, which Azure Databricks workspaces support (the workspace federates the
login to Microsoft Entra).

It forwards the connector's databricks-sql-python app bundle, NOT the Thrift
Azure app (96eecda7 / redirect port 8030): that app is registered for Thrift's
direct-Entra flow (login.microsoftonline.com config + {tenant}/user_impersonation
scopes), which the kernel does not perform. So on the kernel path azure-oauth
and databricks-oauth are the same flow.

The U2M-vs-M2M ambiguity guard now also covers azure-oauth, so
azure-oauth + oauth_client_secret is rejected as ambiguous rather than
silently falling through to the M2M branch.

Connector-only; the kernel already exposes oauth-u2m. (PECOBLR-4120.)

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>

Author: eric-wang-1990

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 # Unreleased
 - Kernel backend (`use_kernel=True`): OAuth U2M with `auth_type="databricks-oauth"` now forwards the connector's `databricks-sql-python` OAuth-app bundle (`client_id` + `sql offline_access` scopes + redirect port) into the kernel, so a bare U2M connection authenticates as `databricks-sql-python` — parity with the Thrift path — instead of inheriting the kernel's own `databricks-sql-connector` default. A caller-supplied `oauth_client_id` (with its coupled `oauth_redirect_port`) is honored, as is a caller-supplied `oauth_scopes`; absent one, the connector default (`sql offline_access`) is forwarded. Note: the kernel binds a single U2M redirect port, so unlike the Thrift path (which tries the full `8020..8024` range) the kernel path uses only one port and does not fall back to the next port if it is already bound — pass `oauth_redirect_port` (with `oauth_client_id`) to pick a free one on a port collision (PECOBLR-4040)
-- Kernel backend (`use_kernel=True`): **Azure Entra (Azure AD) service-principal M2M is now supported.** `auth_type="azure-sp-m2m"` forwards `azure_client_id` / `azure_client_secret`; the kernel is the Azure-aware auth core — it builds the Entra v2.0 token endpoint and the `{app_id}/.default` scope, and **auto-discovers the tenant** from the workspace's `/aad/auth` redirect when `azure_tenant_id` is omitted (matching Thrift). The `Authorization` bearer is the Databricks-audience data token, which alone authenticates a workspace-member SP. Set `azure_workspace_resource_id` and the kernel also sends the Azure SP management token (`X-Databricks-Azure-SP-Management-Token`) + `X-Databricks-Azure-Workspace-Resource-Id` header (matching the JDBC driver), so a service principal with an Azure RBAC role but no workspace membership can authenticate; omit it and no ARM management-scope token is fetched. Azure AD **U2M** (`auth_type="azure-oauth"`) is **not** supported on the kernel path — use `auth_type="databricks-oauth"`, whose in-house OAuth browser flow works against Azure workspaces (the workspace federates login to Entra); `azure-oauth` on `use_kernel=True` is rejected with that pointer (PECOBLR-4141; PECOBLR-4120)
+- Kernel backend (`use_kernel=True`): **Azure Entra (Azure AD) service-principal M2M is now supported.** `auth_type="azure-sp-m2m"` forwards `azure_client_id` / `azure_client_secret`; the kernel is the Azure-aware auth core — it builds the Entra v2.0 token endpoint and the `{app_id}/.default` scope, and **auto-discovers the tenant** from the workspace's `/aad/auth` redirect when `azure_tenant_id` is omitted (matching Thrift). The `Authorization` bearer is the Databricks-audience data token, which alone authenticates a workspace-member SP. Set `azure_workspace_resource_id` and the kernel also sends the Azure SP management token (`X-Databricks-Azure-SP-Management-Token`) + `X-Databricks-Azure-Workspace-Resource-Id` header (matching the JDBC driver), so a service principal with an Azure RBAC role but no workspace membership can authenticate; omit it and no ARM management-scope token is fetched. Azure AD **U2M** (`auth_type="azure-oauth"`) now routes to the kernel's OAuth U2M flow, identically to `auth_type="databricks-oauth"`: the kernel runs the in-house workspace-federated browser flow, which Azure workspaces support (the workspace federates login to Entra). It forwards the connector's `databricks-sql-python` OAuth app, not the Thrift Azure app (`96eecda7` / port 8030), which is registered for Thrift's direct-Entra flow the kernel does not perform (PECOBLR-4141; PECOBLR-4120)
 
 # 4.4.0 (2026-07-22)
 - Raised the minimum supported Python version to 3.10, dropping the end-of-life 3.8/3.9, to update the lockfile and clear CVE-flagged dependencies in the repo (databricks/databricks-sql-python#798)

CONNECTION_PARAMETERS.md

@@ -69,7 +69,7 @@ to change without notice.
 | Option                                              | Type                 | Thrift | Kernel | Default Value             | Note                                                                                                                                                            |
 | --------------------------------------------------- | -------------------- | :----: | :----: | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `access_token` (PAT)                                | `str`                |   ✅   |   ✅   | `None`                    | Personal Access Token / bearer token. The default auth mode when set; otherwise auth falls back to OAuth.                                                       |
-| `auth_type`                                         | `str`                |   ✅   |   ✅   | `None` ⇒ Databricks OAuth | `databricks-oauth` (U2M), `azure-oauth` (Azure AD U2M), or `azure-sp-m2m` (Azure service-principal M2M). On the kernel path: `databricks-oauth` and `azure-sp-m2m` are supported (#919); `azure-oauth` is **not** — use `databricks-oauth` instead (its in-house browser flow works against Azure workspaces), and `azure-oauth` on `use_kernel=True` is rejected with that pointer. All three work on the Thrift path. |
+| `auth_type`                                         | `str`                |   ✅   |   ✅   | `None` ⇒ Databricks OAuth | `databricks-oauth` (U2M), `azure-oauth` (Azure AD U2M), or `azure-sp-m2m` (Azure service-principal M2M). All three are supported on the kernel path (#919): `azure-oauth` routes to the kernel's OAuth U2M flow identically to `databricks-oauth` (the in-house workspace-federated browser flow, which Azure workspaces support), so it uses the `databricks-sql-python` app rather than Thrift's direct-Entra Azure app. All three work on the Thrift path. |
 | `oauth_client_id` (U2M)                             | `str`                |   ✅   |   ✅   | built-in client id        | Custom U2M client id. Forwarded on both; when absent, each path applies its own built-in default.                                                              |
 | `oauth_redirect_port` (U2M)                         | `int`                |   ✅   |   ✅   | `None`                    | Localhost redirect port for the browser flow. On **both** backends it is only honored when a custom `oauth_client_id` is also supplied — then that single port becomes the redirect URI. With the built-in client id (or when omitted) the connector uses the full registered range 8020–8024 and binds the first free port, so a bare `oauth_redirect_port` has no effect. (Thrift: `auth.py` `oauth_redirect_port_range`; Kernel: same logic, forwarded as `redirect_ports`.) |
 | `oauth_client_secret` (OAuth M2M)                   | `str`                |   ❌   |   ✅   | `None`                    | **Kernel-only in practice.** The Thrift auth path never reads `oauth_client_secret`; use `credentials_provider` or an Azure service principal for M2M on Thrift. |

src/databricks/sql/backend/kernel/auth_bridge.py

@@ -15,8 +15,8 @@
   connector's own OAuth provider because the kernel re-mints tokens
   itself and the client secret is not recoverable from a built
   provider.
-- **OAuth U2M** — for ``auth_type`` ``databricks-oauth`` (the browser
-  authorization-code flow), the connector's ``databricks-sql-python``
+- **OAuth U2M** — for ``auth_type`` ``databricks-oauth`` *or* ``azure-oauth``
+  (the browser authorization-code flow), the connector's ``databricks-sql-python``
   app bundle (``client_id`` + ``redirect_ports`` list, with the optional
   ``oauth_client_id`` / ``oauth_redirect_port`` overriding it) is
   forwarded to the kernel's ``auth_type='oauth-u2m'`` and the kernel
@@ -25,10 +25,16 @@
   owns the endpoints, scopes, app ids, and tenant discovery); the binding
   forwards the selector + credentials and does not construct endpoints:
 
-  - ``azure-oauth`` (U2M) → **not supported on the kernel path**; rejected with
-    a pointer to ``databricks-oauth``, whose in-house U2M browser flow works
-    against Azure workspaces (the workspace federates login to Entra). A
-    dedicated Azure U2M flow may return later (PECOBLR-4120).
+  - ``azure-oauth`` (U2M) → routed to the kernel's ``oauth-u2m`` via the shared
+    U2M path above, exactly like ``databricks-oauth``. The kernel runs the
+    in-house workspace-federated browser flow (the workspace federates login to
+    Entra), which Azure Databricks workspaces support; it forwards the
+    ``databricks-sql-python`` bundle, **not** the Thrift Azure app
+    (``96eecda7`` / redirect port ``8030``) — that app is registered for
+    Thrift's *direct-Entra* flow (``login.microsoftonline.com`` config +
+    ``{tenant}/user_impersonation`` scopes), which the kernel does not perform.
+    So on the kernel path ``azure-oauth`` and ``databricks-oauth`` are the same
+    flow (PECOBLR-4120).
   - ``azure-sp-m2m`` (M2M) → forward ``auth_type='azure-sp-m2m'`` with the
     Azure service-principal ``azure_client_id`` / ``azure_client_secret`` (plus
     optional ``azure_tenant_id`` / ``azure_workspace_resource_id``). The kernel
@@ -167,25 +173,27 @@ def kernel_auth_kwargs(
        rather than silently picking one flow (and failing later as a
        confusing 401 against the wrong principal):
        - a custom ``credentials_provider`` *and* M2M kwargs together;
-       - a U2M ``auth_type`` (``databricks-oauth``) *and*
+       - a U2M ``auth_type`` (``databricks-oauth`` / ``azure-oauth``) *and*
          ``oauth_client_secret`` together.
 
-    (The Azure Entra auth types are handled up front, before these guards:
-    ``azure-sp-m2m`` forwards to the kernel's Azure SP flow; ``azure-oauth``
-    is rejected with a pointer to ``databricks-oauth``. See the module
-    docstring.)
+    (The ``azure-sp-m2m`` Azure Entra auth type is handled up front, before
+    these guards, forwarding to the kernel's Azure SP flow. ``azure-oauth`` is
+    a U2M type and is handled by step 3 alongside ``databricks-oauth``. See the
+    module docstring.)
     1. **OAuth M2M** — ``oauth_client_id`` + ``oauth_client_secret``
        both present → forward raw creds to the kernel's ``oauth-m2m``.
     2. **PAT** — the built provider is (or wraps) an
        ``AccessTokenAuthProvider`` → extract the bearer token.
-    3. **OAuth U2M** — ``auth_type`` is ``databricks-oauth`` → forward the
-       connector's coupled ``databricks-sql-python`` bundle (``client_id``
-       + ``redirect_ports`` list, defaulting scopes to ``PYSQL_OAUTH_SCOPES``
-       when the caller supplies none) to the kernel's ``oauth-u2m``, so a
-       bare U2M connection authenticates as ``databricks-sql-python`` —
-       forwarding the connector's own OAuth app rather than the kernel's
-       ``databricks-sql-connector`` default (PECOBLR-4039/4040). Unlike the
-       Thrift path, a caller-supplied ``oauth_scopes`` is honored here.
+    3. **OAuth U2M** — ``auth_type`` is ``databricks-oauth`` or ``azure-oauth``
+       → forward the connector's coupled ``databricks-sql-python`` bundle
+       (``client_id`` + ``redirect_ports`` list, defaulting scopes to
+       ``PYSQL_OAUTH_SCOPES`` when the caller supplies none) to the kernel's
+       ``oauth-u2m``, so a bare U2M connection authenticates as
+       ``databricks-sql-python`` — forwarding the connector's own OAuth app
+       rather than the kernel's ``databricks-sql-connector`` default
+       (PECOBLR-4039/4040). ``azure-oauth`` resolves identically: the kernel's
+       workspace-federated flow serves Azure workspaces too (PECOBLR-4120).
+       Unlike the Thrift path, a caller-supplied ``oauth_scopes`` is honored here.
     4. **Custom credentials_provider** → ``NotSupportedError`` (opaque
        token source; no raw creds for the kernel to own).
     5. Anything else → ``NotSupportedError``.
@@ -212,20 +220,15 @@ def kernel_auth_kwargs(
     # creds in azure_* kwargs, not oauth_client_id/secret, so it would
     # otherwise fall through to the final "unsupported" error).
 
-    # azure-oauth (Azure AD U2M) is NOT supported on the kernel path. The
-    # in-house `databricks-oauth` browser flow works against Azure workspaces
-    # (the workspace federates the login to Entra), so it is the U2M path on the
-    # kernel — reject `azure-oauth` with a clear pointer rather than silently
-    # changing the user's selected flow. (A dedicated Azure U2M flow may return
-    # later; for now Azure U2M = `databricks-oauth`.)
-    if auth_type == "azure-oauth":
-        raise NotSupportedError(
-            "auth_type='azure-oauth' is not supported on use_kernel=True. Use "
-            "auth_type='databricks-oauth' instead — the in-house OAuth U2M "
-            "browser flow works against Azure Databricks workspaces (the "
-            "workspace federates the login to Microsoft Entra). Or use the "
-            "Thrift backend (default) for the dedicated Azure AD U2M flow."
-        )
+    # azure-oauth (Azure AD U2M) routes to the kernel's oauth-u2m, handled by the
+    # shared U2M branch below (step 3) alongside databricks-oauth. The kernel runs
+    # the in-house workspace-federated browser flow — the workspace federates the
+    # login to Microsoft Entra — which Azure Databricks workspaces support. It uses
+    # the connector's databricks-sql-python app bundle, NOT the Thrift Azure app
+    # (96eecda7 / redirect port 8030): that app is registered for Thrift's
+    # direct-Entra flow (login.microsoftonline.com config + {tenant}/user_impersonation
+    # scopes), which the kernel does not perform. So on the kernel path azure-oauth
+    # and databricks-oauth are the same flow. PECOBLR-4120.
 
     # azure-sp-m2m (Azure service principal, client-credentials): forward the
     # selector + Azure SP credentials; the KERNEL owns Azure resolution (it is
@@ -278,7 +281,7 @@ def kernel_auth_kwargs(
             "kernel-managed M2M, or use the Thrift backend (default) for "
             "credentials_provider."
         )
-    if client_secret and auth_type == "databricks-oauth":
+    if client_secret and auth_type in ("databricks-oauth", "azure-oauth"):
         raise NotSupportedError(
             f"Ambiguous auth on use_kernel=True: auth_type={auth_type!r} selects "
             "the U2M browser flow, but oauth_client_secret was also provided "
@@ -314,7 +317,10 @@ def kernel_auth_kwargs(
         return kwargs
 
     # 3. OAuth U2M — browser authorization-code flow; the kernel runs it.
-    #    Only databricks-oauth reaches here (azure-oauth rejected up front).
+    #    Both databricks-oauth and azure-oauth reach here: on the kernel path
+    #    they are the same in-house workspace-federated flow (Azure workspaces
+    #    federate the browser login to Entra), so azure-oauth is an alias for
+    #    databricks-oauth here — see the note near the top of this function.
     #    Forward the connector's own databricks-sql-python bundle instead of
     #    the kernel's databricks-sql-connector default, for parity with the
     #    Thrift path. client_id + redirect ports are coupled per app (each
@@ -326,7 +332,7 @@ def kernel_auth_kwargs(
     #    fallback), mirroring the Thrift DatabricksOAuthProvider which retries
     #    the next port when one is bound. A caller overriding client_id
     #    supplies its own single registered port.
-    if auth_type == "databricks-oauth":
+    if auth_type in ("databricks-oauth", "azure-oauth"):
         redirect_port = opts.get("oauth_redirect_port")
         # Honor a caller-supplied oauth_scopes (normalized to a list of
         # strings); fall back to the connector default when none is given.
@@ -367,8 +373,9 @@ def kernel_auth_kwargs(
     )
     raise NotSupportedError(
         f"use_kernel=True requires PAT (access_token), OAuth M2M "
-        f"(oauth_client_id + oauth_client_secret), or OAuth U2M "
-        f"(auth_type='databricks-oauth'), but got "
+        f"(oauth_client_id + oauth_client_secret), OAuth U2M "
+        f"(auth_type='databricks-oauth' or 'azure-oauth'), or Azure SP M2M "
+        f"(auth_type='azure-sp-m2m'), but got "
         f"{provider_desc} with auth_type={auth_type!r}. Use the Thrift "
         "backend (default) for other auth flows."
     )