From 3ef4e183bbfccf1a262916a687bf1df06f8ff986 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 00:35:21 +0800 Subject: [PATCH 01/22] [feat](authorization) add fe-authorization-api with neutral data-policy payloads First artifact of the FE authorization plugin work: a dependency-free API module that will hold the types an authorization plugin exchanges with the engine, starting with the two that have a consumer in this series - the row filter and column mask payloads. Both carry a SQL expression in Doris dialect rather than a parsed expression tree. That is the lossless form for every producer that exists: an internal row policy comes from CREATE ROW POLICY text, a Ranger filter is text a human typed into the Ranger UI, and a Ranger mask is a transformer template from the service definition - which is always a function call, a shape the neutral connector expression layer's reverse converter does not accept at all. Trino splits the same way: structured expressions engine-to-plugin (pushdown), SQL text plugin-to-engine (ViewExpression, masks). Both types are immutable values with real equality, which is load bearing rather than cosmetic: the SQL result cache decides "did the policies change?" by comparing the specs recorded at plan time with the specs evaluated now, so identity equality there evicts the cache on every lookup. The tests assert both directions - equal while the policy is untouched, unequal the moment its version or text changes. Verified: mvn -pl fe-authorization/fe-authorization-api -am test (7/7, build cache disabled) and checkstyle:check, both green. Co-Authored-By: Claude Opus 5 (1M context) --- .../fe-authorization-api/pom.xml | 57 +++++++++ .../doris/authorization/DataMaskSpec.java | 86 ++++++++++++++ .../authorization/RowFilterMergeType.java | 32 ++++++ .../doris/authorization/RowFilterSpec.java | 108 ++++++++++++++++++ .../doris/authorization/DataMaskSpecTest.java | 57 +++++++++ .../authorization/RowFilterSpecTest.java | 74 ++++++++++++ fe/fe-authorization/pom.xml | 41 +++++++ fe/fe-core/pom.xml | 5 + fe/pom.xml | 1 + 9 files changed, 461 insertions(+) create mode 100644 fe/fe-authorization/fe-authorization-api/pom.xml create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/DataMaskSpec.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterMergeType.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterSpec.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/DataMaskSpecTest.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/RowFilterSpecTest.java create mode 100644 fe/fe-authorization/pom.xml diff --git a/fe/fe-authorization/fe-authorization-api/pom.xml b/fe/fe-authorization/fe-authorization-api/pom.xml new file mode 100644 index 00000000000000..160c4da96bdd07 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/pom.xml @@ -0,0 +1,57 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization + ../pom.xml + + fe-authorization-api + jar + Doris FE Authorization API + + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + + **/*Test.java + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 17 + 17 + + + + + diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/DataMaskSpec.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/DataMaskSpec.java new file mode 100644 index 00000000000000..80fb7d500c71d8 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/DataMaskSpec.java @@ -0,0 +1,86 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Objects; + +/** + * How one column must be rewritten before a subject may read it. + * + *

The payload is a scalar SQL expression in Doris dialect that replaces the column in the projection + * ({@code CONCAT('XXXX', SUBSTR(phone, -4))}, {@code NULL}). Sources hold this as text natively - Ranger, for + * one, stores a transformer template in its service definition and substitutes the column name into it - so + * text is the lossless form; see {@link RowFilterSpec} for the full rationale.

+ * + *

Like {@link RowFilterSpec} this is an immutable value with real equality, because the SQL result cache + * uses spec equality to decide whether a column's masking changed.

+ */ +public final class DataMaskSpec { + + private final String policyIdent; + private final String maskSql; + + /** + * @param policyIdent identifies the policy that produced this mask, for auditing and for change + * detection; it must change when the policy changes (e.g. {@code ":"}) + * @param maskSql a scalar SQL expression in Doris dialect yielding the value the subject may see + */ + public DataMaskSpec(String policyIdent, String maskSql) { + this.policyIdent = requireNonBlank(policyIdent, "policyIdent"); + this.maskSql = requireNonBlank(maskSql, "maskSql"); + } + + public String getPolicyIdent() { + return policyIdent; + } + + public String getMaskSql() { + return maskSql; + } + + private static String requireNonBlank(String value, String field) { + Objects.requireNonNull(value, field + " is required"); + if (value.trim().isEmpty()) { + // A blank mask would parse to nothing and leave the raw column exposed. Fail at the source. + throw new IllegalArgumentException(field + " must not be blank"); + } + return value; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof DataMaskSpec)) { + return false; + } + DataMaskSpec that = (DataMaskSpec) o; + return policyIdent.equals(that.policyIdent) && maskSql.equals(that.maskSql); + } + + @Override + public int hashCode() { + return Objects.hash(policyIdent, maskSql); + } + + @Override + public String toString() { + return "DataMaskSpec{policyIdent='" + policyIdent + "', maskSql='" + maskSql + "'}"; + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterMergeType.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterMergeType.java new file mode 100644 index 00000000000000..43de03fd6019b7 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterMergeType.java @@ -0,0 +1,32 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +/** + * How the engine combines several {@link RowFilterSpec}s that apply to the same table for the same subject. + * + *

The engine owns the combination semantics (an authorization source never gets to define how its filters + * merge with another's): restrictive filters are ANDed, permissive filters are ORed, and when both kinds are + * present the result is {@code AND(restrictive...) AND OR(permissive...)}.

+ */ +public enum RowFilterMergeType { + /** The subject sees only rows matching this filter - ANDed with every other restrictive filter. */ + RESTRICTIVE, + /** The subject additionally sees rows matching this filter - ORed with every other permissive filter. */ + PERMISSIVE +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterSpec.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterSpec.java new file mode 100644 index 00000000000000..40d702348de9a4 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/RowFilterSpec.java @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Objects; + +/** + * One row-level security filter an authorization source applies to a table for one subject. + * + *

The payload is a SQL boolean expression in Doris dialect ({@code region = 'cn' AND dept IN + * ('a','b')}), not a parsed expression tree. The engine parses it, checks that it is boolean, and injects it + * as a filter above the scan. This is deliberate and mirrors Trino's {@code ViewExpression}: a row filter + * originates as SQL text authored by a human in an external system (a Ranger policy, a {@code CREATE ROW + * POLICY} statement), so text is its lossless native form. A structured tree would force every source to + * embed a SQL parser, and cannot express what real policies contain - function calls above all.

+ * + *

Value semantics are load bearing, not cosmetic: the SQL result cache decides "did the policies change?" + * by comparing the specs recorded at plan time against the specs evaluated now. Identity equality there means + * a freshly built - but identical - spec reads as a policy change and evicts the cache on every single + * lookup. Hence this type is an immutable value with {@link #equals}/{@link #hashCode}, and every producer + * must fold whatever version token it has into {@link #getPolicyIdent()} so that a genuinely updated policy + * does compare unequal.

+ */ +public final class RowFilterSpec { + + private final String policyIdent; + private final String filterSql; + private final RowFilterMergeType mergeType; + + /** + * @param policyIdent identifies the policy that produced this filter, for auditing and for change + * detection; it must change when the policy changes (e.g. {@code ":"}) + * @param filterSql a boolean SQL expression in Doris dialect over the table's columns + * @param mergeType how this filter combines with the table's other filters + */ + public RowFilterSpec(String policyIdent, String filterSql, RowFilterMergeType mergeType) { + this.policyIdent = requireNonBlank(policyIdent, "policyIdent"); + this.filterSql = requireNonBlank(filterSql, "filterSql"); + this.mergeType = Objects.requireNonNull(mergeType, "mergeType is required"); + } + + /** Creates a restrictive (ANDed) filter, the default for an authorization source. */ + public static RowFilterSpec restrictive(String policyIdent, String filterSql) { + return new RowFilterSpec(policyIdent, filterSql, RowFilterMergeType.RESTRICTIVE); + } + + public String getPolicyIdent() { + return policyIdent; + } + + public String getFilterSql() { + return filterSql; + } + + public RowFilterMergeType getMergeType() { + return mergeType; + } + + private static String requireNonBlank(String value, String field) { + Objects.requireNonNull(value, field + " is required"); + if (value.trim().isEmpty()) { + // A blank filter would silently degrade to "no restriction" once parsed, i.e. it would widen + // access. Reject at construction so a broken policy fails loudly at its source. + throw new IllegalArgumentException(field + " must not be blank"); + } + return value; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof RowFilterSpec)) { + return false; + } + RowFilterSpec that = (RowFilterSpec) o; + return policyIdent.equals(that.policyIdent) + && filterSql.equals(that.filterSql) + && mergeType == that.mergeType; + } + + @Override + public int hashCode() { + return Objects.hash(policyIdent, filterSql, mergeType); + } + + @Override + public String toString() { + return "RowFilterSpec{policyIdent='" + policyIdent + "', filterSql='" + filterSql + + "', mergeType=" + mergeType + '}'; + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/DataMaskSpecTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/DataMaskSpecTest.java new file mode 100644 index 00000000000000..eab9896e8042e9 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/DataMaskSpecTest.java @@ -0,0 +1,57 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +/** + * Same contract as {@link RowFilterSpecTest}, for column masking: the SQL result cache compares the mask + * recorded at plan time with the mask evaluated now, so equality must track the policy exactly - equal while + * the policy is untouched, unequal the moment it is edited. + */ +public class DataMaskSpecTest { + + @Test + public void testSpecsWithSameContentAreEqual() { + DataMaskSpec planned = new DataMaskSpec("7:3", "CONCAT('XXXX', SUBSTR(phone, -4))"); + DataMaskSpec reevaluated = new DataMaskSpec("7:3", "CONCAT('XXXX', SUBSTR(phone, -4))"); + + Assertions.assertEquals(planned, reevaluated, + "an unchanged mask policy must re-evaluate to an equal spec, or the cache is evicted every lookup"); + Assertions.assertEquals(planned.hashCode(), reevaluated.hashCode()); + } + + @Test + public void testUpdatedPolicyIsNotEqual() { + DataMaskSpec planned = new DataMaskSpec("7:3", "NULL"); + + Assertions.assertNotEquals(planned, new DataMaskSpec("7:4", "NULL"), + "a new policy version must compare unequal so the cache invalidates"); + Assertions.assertNotEquals(planned, new DataMaskSpec("7:3", "phone"), + "unmasking a column must compare unequal so the cache invalidates"); + } + + @Test + public void testBlankMaskIsRejected() { + // A blank mask expression would leave the raw column in the projection - i.e. no masking at all. + Assertions.assertThrows(IllegalArgumentException.class, () -> new DataMaskSpec("7:3", " ")); + Assertions.assertThrows(NullPointerException.class, () -> new DataMaskSpec("7:3", null)); + Assertions.assertThrows(NullPointerException.class, () -> new DataMaskSpec(null, "NULL")); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/RowFilterSpecTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/RowFilterSpecTest.java new file mode 100644 index 00000000000000..18af56bb527090 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/RowFilterSpecTest.java @@ -0,0 +1,74 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +/** + * The equality contract of {@link RowFilterSpec} is what the SQL result cache uses to answer "did this + * table's row policies change since we planned this query?". Both directions of that answer are a real + * defect when wrong, so both are asserted here: equal-when-unchanged (else every lookup evicts the cache and + * a Ranger user never gets a cache hit) and unequal-when-updated (else an edited policy keeps serving rows + * the subject may no longer see). + */ +public class RowFilterSpecTest { + + @Test + public void testSpecsWithSameContentAreEqual() { + RowFilterSpec planned = new RowFilterSpec("7:3", "region = 'cn'", RowFilterMergeType.RESTRICTIVE); + RowFilterSpec reevaluated = new RowFilterSpec("7:3", "region = 'cn'", RowFilterMergeType.RESTRICTIVE); + + Assertions.assertEquals(planned, reevaluated, + "an unchanged policy must re-evaluate to an equal spec, or the cache is evicted every lookup"); + Assertions.assertEquals(planned.hashCode(), reevaluated.hashCode()); + } + + @Test + public void testUpdatedPolicyIsNotEqual() { + RowFilterSpec planned = new RowFilterSpec("7:3", "region = 'cn'", RowFilterMergeType.RESTRICTIVE); + + Assertions.assertNotEquals(planned, + new RowFilterSpec("7:4", "region = 'cn'", RowFilterMergeType.RESTRICTIVE), + "a new policy version must compare unequal so the cache invalidates"); + Assertions.assertNotEquals(planned, + new RowFilterSpec("7:3", "region = 'us'", RowFilterMergeType.RESTRICTIVE), + "a rewritten predicate must compare unequal so the cache invalidates"); + Assertions.assertNotEquals(planned, + new RowFilterSpec("7:3", "region = 'cn'", RowFilterMergeType.PERMISSIVE), + "flipping restrictive to permissive changes which rows are visible"); + } + + @Test + public void testRestrictiveIsTheDefaultShorthand() { + Assertions.assertEquals(RowFilterMergeType.RESTRICTIVE, + RowFilterSpec.restrictive("7:3", "region = 'cn'").getMergeType()); + } + + @Test + public void testBlankFilterIsRejected() { + // A blank predicate parses to nothing, which would widen access to the whole table instead of + // restricting it - the failure mode that must never be silent. + Assertions.assertThrows(IllegalArgumentException.class, + () -> RowFilterSpec.restrictive("7:3", " ")); + Assertions.assertThrows(NullPointerException.class, + () -> RowFilterSpec.restrictive("7:3", null)); + Assertions.assertThrows(IllegalArgumentException.class, + () -> RowFilterSpec.restrictive("", "region = 'cn'")); + } +} diff --git a/fe/fe-authorization/pom.xml b/fe/fe-authorization/pom.xml new file mode 100644 index 00000000000000..138ee4ba32aed5 --- /dev/null +++ b/fe/fe-authorization/pom.xml @@ -0,0 +1,41 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe + ../pom.xml + + fe-authorization + pom + Doris FE Authorization + + + fe-authorization-api + + diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index 96d3d9ec22bcea..e7825f36633a50 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -390,6 +390,11 @@ under the License. fe-authentication-role-mapping ${project.version} + + ${project.groupId} + fe-authorization-api + ${project.version} + org.springframework.boot spring-boot-devtools diff --git a/fe/pom.xml b/fe/pom.xml index fa1ef0a650a524..60371c37bc9d08 100644 --- a/fe/pom.xml +++ b/fe/pom.xml @@ -235,6 +235,7 @@ under the License. hive-udf be-java-extensions fe-authentication + fe-authorization fe-thrift fe-type fe-grpc From a4ec9668b3c5553d4f83fe133eafbdfedb7c2998 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 00:48:13 +0800 Subject: [PATCH 02/22] [test](authorization) reproduce the SQL cache never hitting under Ranger data policies Pins the contract NereidsSqlCacheManager relies on before serving a cached result: it re-evaluates a query's row-filter and data-mask policies and compares them by value with what was recorded at plan time, so an untouched policy must re-evaluate to an equal object. The Ranger policy types do not hold up their end - neither implements equals, and RangerAccessController builds a fresh instance on every evaluation - so the comparison degrades to identity and always reports a change. A user with any row filter or masked column configured therefore never gets a SQL cache hit; the entry is evicted on every single lookup. Confirmed here rather than inferred: the two "untouched policy" cases fail and the five change-detection cases pass, which is the exact signature of missing value equality. This commit is deliberately RED - the payload rework in the next commit turns it green. The change-detection cases are pinned alongside so that fix cannot buy cache hits by weakening the comparison: an edited or revoked policy must still invalidate, or the cache keeps serving rows the user may no longer see. Verified: mvn -pl fe-core -am test -Dtest=NereidsSqlCacheDataPolicyTest (build cache disabled) - 7 run, 2 failures, both the untouched-policy cases. Co-Authored-By: Claude Opus 5 (1M context) --- .../cache/NereidsSqlCacheDataPolicyTest.java | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java diff --git a/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java b/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java new file mode 100644 index 00000000000000..3aba8399659211 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java @@ -0,0 +1,188 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.common.cache; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.catalog.Env; +import org.apache.doris.common.jmockit.Deencapsulation; +import org.apache.doris.mysql.privilege.AccessControllerManager; +import org.apache.doris.mysql.privilege.DataMaskPolicy; +import org.apache.doris.mysql.privilege.RangerDataMaskPolicy; +import org.apache.doris.mysql.privilege.RangerRowFilterPolicy; +import org.apache.doris.mysql.privilege.RowFilterPolicy; +import org.apache.doris.nereids.SqlCacheContext; + +import com.google.common.collect.ImmutableList; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; +import org.mockito.ArgumentMatchers; +import org.mockito.Mockito; + +import java.util.List; +import java.util.Optional; + +/** + * Before serving a cached result, {@link NereidsSqlCacheManager} re-evaluates the row-filter and data-mask + * policies of every table the query touched and compares them with what was recorded at plan time. The + * comparison is by value ({@code isEqualCollection} / {@code Objects.equals}), so it only answers the + * intended question - "has the administrator changed a policy?" - if an unchanged policy re-evaluates to an + * equal object. + * + *

An authorization source builds its policy objects fresh on every evaluation (it is answering a request, + * not handing out a cached object), so value equality is the source's obligation, and the "unchanged" tests + * below are what enforce it. They are RED for the Ranger policy types, which have no equals: the comparison + * degrades to identity, every lookup reads as a policy change, and a user with any row filter or column mask + * configured never gets a cache hit at all.

+ * + *

The "changed" tests pin the other half of the contract, so a fix cannot buy cache hits by weakening the + * comparison: an edited policy must still invalidate, or the cache keeps serving rows the user may no longer + * see.

+ */ +public class NereidsSqlCacheDataPolicyTest { + + private static final String CTL = "internal"; + private static final String DB = "test_db"; + private static final String TBL = "orders"; + private static final String COL = "phone"; + private static final UserIdentity USER = UserIdentity.ROOT; + + private RowFilterPolicy rowFilter(long policyVersion, String filterExpr) { + return new RangerRowFilterPolicy(USER, CTL, DB, TBL, 7L, policyVersion, filterExpr); + } + + private DataMaskPolicy dataMask(long policyVersion, String maskTypeDef) { + return new RangerDataMaskPolicy(USER, CTL, DB, TBL, COL, 7L, policyVersion, "CUSTOM", maskTypeDef); + } + + /** + * Wires an {@link Env} whose access manager re-evaluates to the supplied policies. The supplier is + * invoked per call so every evaluation yields a distinct object, exactly as a live authorization source + * behaves. + */ + private Env mockEnvEvaluating(java.util.function.Supplier rowFilterSupplier, + java.util.function.Supplier> dataMaskSupplier) { + AccessControllerManager accessManager = Mockito.mock(AccessControllerManager.class); + Mockito.doAnswer(invocation -> { + RowFilterPolicy evaluated = rowFilterSupplier.get(); + return evaluated == null ? ImmutableList.of() : ImmutableList.of(evaluated); + }) + .when(accessManager).evalRowFilterPolicies(ArgumentMatchers.any(UserIdentity.class), + ArgumentMatchers.anyString(), ArgumentMatchers.anyString(), ArgumentMatchers.anyString()); + Mockito.doAnswer(invocation -> dataMaskSupplier.get()) + .when(accessManager).evalDataMaskPolicy(ArgumentMatchers.any(UserIdentity.class), + ArgumentMatchers.anyString(), ArgumentMatchers.anyString(), ArgumentMatchers.anyString(), + ArgumentMatchers.anyString()); + + Env env = Mockito.mock(Env.class); + Mockito.when(env.getAccessManager()).thenReturn(accessManager); + return env; + } + + private boolean rowPoliciesChanged(Env env, SqlCacheContext context) { + return Deencapsulation.invoke(new NereidsSqlCacheManager(), "rowPoliciesChanged", USER, env, context); + } + + private boolean dataMaskPoliciesChanged(Env env, SqlCacheContext context) { + return Deencapsulation.invoke(new NereidsSqlCacheManager(), "dataMaskPoliciesChanged", USER, env, context); + } + + private SqlCacheContext contextWithRowFilter(RowFilterPolicy policy) { + SqlCacheContext context = new SqlCacheContext(USER); + List policies = policy == null ? ImmutableList.of() : ImmutableList.of(policy); + context.setRowFilterPolicy(CTL, DB, TBL, policies); + return context; + } + + private SqlCacheContext contextWithDataMask(Optional policy) { + SqlCacheContext context = new SqlCacheContext(USER); + context.addDataMaskPolicy(CTL, DB, TBL, COL, policy); + return context; + } + + /** The administrator touched nothing: the same filter must re-evaluate equal and keep the cache. */ + @Test + public void testUntouchedRowFilterKeepsCache() { + SqlCacheContext context = contextWithRowFilter(rowFilter(3L, "region = 'cn'")); + Env env = mockEnvEvaluating(() -> rowFilter(3L, "region = 'cn'"), Optional::empty); + + Assertions.assertFalse(rowPoliciesChanged(env, context), + "an unchanged row filter must not read as a policy change, otherwise every cache lookup for a " + + "user with row-level security evicts the entry and the SQL cache never serves anything"); + } + + /** The administrator edited the policy: the cache must not serve rows filtered by the old predicate. */ + @Test + public void testEditedRowFilterInvalidatesCache() { + SqlCacheContext context = contextWithRowFilter(rowFilter(3L, "region = 'cn'")); + Env env = mockEnvEvaluating(() -> rowFilter(4L, "region IN ('cn', 'us')"), Optional::empty); + + Assertions.assertTrue(rowPoliciesChanged(env, context), + "an edited row filter must invalidate the cache"); + } + + /** A revoked policy widens nothing but still changes the result set, so it must invalidate too. */ + @Test + public void testRemovedRowFilterInvalidatesCache() { + SqlCacheContext context = contextWithRowFilter(rowFilter(3L, "region = 'cn'")); + Env env = mockEnvEvaluating(() -> null, Optional::empty); + + Assertions.assertTrue(rowPoliciesChanged(env, context), + "dropping a row filter changes which rows the query returns"); + } + + /** Same contract for column masking. */ + @Test + public void testUntouchedDataMaskKeepsCache() { + SqlCacheContext context = contextWithDataMask(Optional.of(dataMask(3L, "CONCAT('XXXX', SUBSTR(phone, -4))"))); + Env env = mockEnvEvaluating(() -> null, + () -> Optional.of(dataMask(3L, "CONCAT('XXXX', SUBSTR(phone, -4))"))); + + Assertions.assertFalse(dataMaskPoliciesChanged(env, context), + "an unchanged column mask must not read as a policy change, otherwise every cache lookup for a " + + "user with a masked column evicts the entry"); + } + + @Test + public void testEditedDataMaskInvalidatesCache() { + SqlCacheContext context = contextWithDataMask(Optional.of(dataMask(3L, "CONCAT('XXXX', SUBSTR(phone, -4))"))); + Env env = mockEnvEvaluating(() -> null, () -> Optional.of(dataMask(4L, "NULL"))); + + Assertions.assertTrue(dataMaskPoliciesChanged(env, context), + "a strengthened column mask must invalidate the cache"); + } + + /** Unmasking a column exposes raw values: the cached masked result is stale, and so is the reverse. */ + @Test + public void testRemovedDataMaskInvalidatesCache() { + SqlCacheContext context = contextWithDataMask(Optional.of(dataMask(3L, "NULL"))); + Env env = mockEnvEvaluating(() -> null, Optional::empty); + + Assertions.assertTrue(dataMaskPoliciesChanged(env, context), + "dropping a column mask changes the values the query returns"); + } + + /** A column that never had a mask must not keep evicting the cache either. */ + @Test + public void testColumnWithoutMaskKeepsCache() { + SqlCacheContext context = contextWithDataMask(Optional.empty()); + Env env = mockEnvEvaluating(() -> null, Optional::empty); + + Assertions.assertFalse(dataMaskPoliciesChanged(env, context), + "an unmasked column must not read as a policy change"); + } +} From 57e2d278be63896d16ba18a9e1638de85268c104 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 01:35:59 +0800 Subject: [PATCH 03/22] [improvement](authorization) hand row filters and column masks to the planner as SQL text The two data-policy payloads an access controller returns were fe-core types that leaked the planner into the authorization surface: RowFilterPolicy handed back a Nereids Expression, so any third-party controller had to compile against fe-core and build an expression tree to express "region = 'cn'". They are now the neutral value types from fe-authorization-api, carrying the predicate and the mask as SQL text in Doris dialect, and the planner does the parsing - which is what the Ranger path already did. Fixes the SQL cache never hitting for anyone with a row filter or a masked column: the cache asks "did the policies change?" by comparing what it recorded at plan time with what evaluates now, the Ranger payloads had no equals, and each evaluation built a new instance, so the answer was always "changed". The new payloads are values with real equality, and their identity folds in the policy version so an edited policy still compares unequal. The repro test from the previous commit is green. Two things this turned up, both now pinned by tests: - The payload has to carry the merge type. The design sketch had the engine always AND row filters, but built-in policies can be permissive, which ORs them; without the field, two permissive policies would silently become an AND and the user would see nothing. CheckRowPolicyTest now covers a permissive pair and a restrictive pair - a single policy cannot tell the two merge modes apart, which is why no existing test caught this. - A built-in policy's predicate cannot be produced by rendering its parsed expression: CompoundPredicate.toSql() emits the diagnostic form AND[a,b], which does not parse back, so every policy combining two conditions would have broken. The text is instead recovered from the CREATE statement the policy already stores, captured at parse time - the text the administrator actually wrote. RowPolicyFilterSqlTest runs 25 representative predicates through create -> store -> hand to planner and asserts the predicate comes back identical; 7 of them failed against the rendering approach. An unparseable stored policy still fails the query with the same message as before rather than disappearing, since a vanished row filter exposes the whole table. Verified: 175 tests across every suite touching the changed surface (cache, policy, privilege, parser, Ranger), 0 failures, 0 skipped, build cache disabled. Mutation check: forcing permissive to merge as restrictive turns CheckRowPolicyTest red, and the assertion that fires is the new one. Co-Authored-By: Claude Opus 5 (1M context) --- .../ranger/RangerAccessController.java | 34 +++--- .../hive/RangerHiveAccessController.java | 8 +- .../common/cache/NereidsSqlCacheManager.java | 16 +-- .../privilege/AccessControllerManager.java | 10 +- .../privilege/CatalogAccessController.java | 14 ++- .../doris/mysql/privilege/DataMaskPolicy.java | 24 ---- .../privilege/InternalAccessController.java | 41 ++++++- .../mysql/privilege/RangerDataMaskPolicy.java | 103 ----------------- .../privilege/RangerRowFilterPolicy.java | 95 ---------------- .../mysql/privilege/RowFilterPolicy.java | 32 ------ .../apache/doris/nereids/SqlCacheContext.java | 24 ++-- .../nereids/parser/LogicalPlanBuilder.java | 7 +- .../plans/commands/CreatePolicyCommand.java | 9 +- .../plans/logical/LogicalCheckPolicy.java | 27 ++--- .../org/apache/doris/policy/RowPolicy.java | 44 ++++++- .../cache/NereidsSqlCacheDataPolicyTest.java | 33 +++--- .../CatalogAccessControllerTest.java | 6 +- .../doris/mysql/privilege/RangerTest.java | 7 +- .../privileges/TestCheckPrivileges.java | 57 +++------- .../rules/analysis/CheckRowPolicyTest.java | 72 ++++++++++-- .../doris/policy/RowPolicyFilterSqlTest.java | 107 ++++++++++++++++++ 21 files changed, 367 insertions(+), 403 deletions(-) delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/DataMaskPolicy.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDataMaskPolicy.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerRowFilterPolicy.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RowFilterPolicy.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/policy/RowPolicyFilterSqlTest.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java index 7a2779b43b1c6b..f26ea775081e10 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java @@ -18,13 +18,11 @@ package org.apache.doris.catalog.authorizer.ranger; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; import org.apache.doris.common.AuthorizationException; import org.apache.doris.mysql.privilege.CatalogAccessController; -import org.apache.doris.mysql.privilege.DataMaskPolicy; -import org.apache.doris.mysql.privilege.RangerDataMaskPolicy; -import org.apache.doris.mysql.privilege.RangerRowFilterPolicy; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import com.google.common.collect.Lists; import org.apache.commons.lang3.StringUtils; @@ -88,8 +86,17 @@ public static void checkRequestResults(Collection results, S } } + /** + * Identifies a Ranger policy for auditing and for the SQL cache's change detection. The version is part of + * it on purpose: an administrator editing a policy in place keeps its id, and without the version an + * updated policy would compare equal to the one the cached result was planned with. + */ + private static String policyIdent(RangerAccessResult policy) { + return policy.getPolicyId() + ":" + policy.getPolicyVersion(); + } + @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { RangerAccessResourceImpl resource = createResource(ctl, db, tbl); RangerAccessRequestImpl request = createRequest(currentUser); @@ -103,7 +110,7 @@ public List evalRowFilterPolicies(UserIdentity curren if (LOG.isDebugEnabled()) { LOG.debug("ranger request: {}", request); } - List res = Lists.newArrayList(); + List res = Lists.newArrayList(); RangerAccessResult policy = getPlugin().evalRowFilterPolicies(request, getAccessResultProcessor()); if (LOG.isDebugEnabled()) { LOG.debug("ranger response: {}", policy); @@ -115,13 +122,13 @@ public List evalRowFilterPolicies(UserIdentity curren if (StringUtils.isEmpty(filterExpr)) { return res; } - res.add(new RangerRowFilterPolicy(currentUser, ctl, db, tbl, policy.getPolicyId(), policy.getPolicyVersion(), - filterExpr)); + // Ranger row filters are always restrictive: it returns at most one expression and the row must match it. + res.add(RowFilterSpec.restrictive(policyIdent(policy), filterExpr)); return res; } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { RangerAccessResourceImpl resource = createResource(ctl, db, tbl, col); RangerAccessRequestImpl request = createRequest(currentUser); @@ -144,8 +151,7 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Str } switch (maskType) { case "MASK_NULL": - return Optional.of(new RangerDataMaskPolicy(currentUser, ctl, db, tbl, col, policy.getPolicyId(), - policy.getPolicyVersion(), maskType, "NULL")); + return Optional.of(new DataMaskSpec(policyIdent(policy), "NULL")); case "MASK_NONE": return Optional.empty(); case "CUSTOM": @@ -153,15 +159,13 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Str if (StringUtils.isEmpty(maskedValue)) { return Optional.empty(); } - return Optional.of(new RangerDataMaskPolicy(currentUser, ctl, db, tbl, col, policy.getPolicyId(), - policy.getPolicyVersion(), maskType, maskedValue.replace("{col}", col))); + return Optional.of(new DataMaskSpec(policyIdent(policy), maskedValue.replace("{col}", col))); default: String transformer = policy.getMaskTypeDef().getTransformer(); if (StringUtils.isEmpty(transformer)) { return Optional.empty(); } - return Optional.of(new RangerDataMaskPolicy(currentUser, ctl, db, tbl, col, policy.getPolicyId(), - policy.getPolicyVersion(), maskType, transformer.replace("{col}", col))); + return Optional.of(new DataMaskSpec(policyIdent(policy), transformer.replace("{col}", col))); } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java index 6f862ceade1f1f..11cbdf20f1b943 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java @@ -19,14 +19,14 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; import org.apache.doris.common.AuthorizationException; import org.apache.doris.common.ThreadPoolManager; import org.apache.doris.datasource.InternalCatalog; -import org.apache.doris.mysql.privilege.DataMaskPolicy; import org.apache.doris.mysql.privilege.PrivPredicate; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import com.google.common.collect.Maps; import org.apache.logging.log4j.LogManager; @@ -259,7 +259,7 @@ public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadG } @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { lifecycleLock.readLock().lock(); try { @@ -270,7 +270,7 @@ public List evalRowFilterPolicies(UserIdentity curren } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { lifecycleLock.readLock().lock(); try { diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/cache/NereidsSqlCacheManager.java b/fe/fe-core/src/main/java/org/apache/doris/common/cache/NereidsSqlCacheManager.java index bee844a9349bb9..4e38efab2781c1 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/common/cache/NereidsSqlCacheManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/common/cache/NereidsSqlCacheManager.java @@ -18,6 +18,8 @@ package org.apache.doris.common.cache; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.DatabaseIf; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.MTMV; @@ -34,8 +36,6 @@ import org.apache.doris.datasource.CatalogIf; import org.apache.doris.datasource.CatalogMgr; import org.apache.doris.mtmv.MTMVRelatedTableIf; -import org.apache.doris.mysql.privilege.DataMaskPolicy; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.CascadesContext; import org.apache.doris.nereids.SqlCacheContext; import org.apache.doris.nereids.SqlCacheContext.CacheKeyType; @@ -535,11 +535,11 @@ private IsChanged viewsChanged(Env env, SqlCacheContext sqlCacheContext) { } private boolean rowPoliciesChanged(UserIdentity currentUserIdentity, Env env, SqlCacheContext sqlCacheContext) { - for (Entry> kv : sqlCacheContext.getRowPolicies().entrySet()) { + for (Entry> kv : sqlCacheContext.getRowPolicies().entrySet()) { FullTableName qualifiedTable = kv.getKey(); - List cachedPolicies = kv.getValue(); + List cachedPolicies = kv.getValue(); - List rowPolicies = env.getAccessManager().evalRowFilterPolicies( + List rowPolicies = env.getAccessManager().evalRowFilterPolicies( currentUserIdentity, qualifiedTable.catalog, qualifiedTable.db, qualifiedTable.table); if (!CollectionUtils.isEqualCollection(cachedPolicies, rowPolicies)) { return true; @@ -550,11 +550,11 @@ private boolean rowPoliciesChanged(UserIdentity currentUserIdentity, Env env, Sq private boolean dataMaskPoliciesChanged( UserIdentity currentUserIdentity, Env env, SqlCacheContext sqlCacheContext) { - for (Entry> kv : sqlCacheContext.getDataMaskPolicies().entrySet()) { + for (Entry> kv : sqlCacheContext.getDataMaskPolicies().entrySet()) { FullColumnName qualifiedColumn = kv.getKey(); - Optional cachedPolicy = kv.getValue(); + Optional cachedPolicy = kv.getValue(); - Optional dataMaskPolicy = env.getAccessManager() + Optional dataMaskPolicy = env.getAccessManager() .evalDataMaskPolicy(currentUserIdentity, qualifiedColumn.catalog, qualifiedColumn.db, qualifiedColumn.table, qualifiedColumn.column); if (!Objects.equals(cachedPolicy, dataMaskPolicy)) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index 1268341600db45..07ff5b46854393 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -19,6 +19,8 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.AuthorizationInfo; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.info.TableNameInfo; @@ -450,16 +452,16 @@ public boolean checkPrivByAuthInfo(ConnectContext ctx, AuthorizationInfo authInf return true; } - public Map> evalDataMaskPolicies(UserIdentity currentUser, String + public Map> evalDataMaskPolicies(UserIdentity currentUser, String ctl, String db, String tbl, Set cols) { - Map> res = Maps.newHashMap(); + Map> res = Maps.newHashMap(); for (String col : cols) { res.put(col, evalDataMaskPolicy(currentUser, ctl, db, tbl, col)); } return res; } - public Optional evalDataMaskPolicy(UserIdentity currentUser, String + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { Objects.requireNonNull(currentUser, "require currentUser object"); Objects.requireNonNull(ctl, "require ctl object"); @@ -469,7 +471,7 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Str return getAccessControllerOrDefault(ctl).evalDataMaskPolicy(currentUser, ctl, db, tbl, col.toLowerCase()); } - public List evalRowFilterPolicies(UserIdentity currentUser, String + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { Objects.requireNonNull(currentUser, "require currentUser object"); Objects.requireNonNull(ctl, "require ctl object"); diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java index 8e548b7463f297..deb144722b2c91 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java @@ -19,6 +19,8 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.common.AuthorizationException; import java.util.List; @@ -88,8 +90,16 @@ void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, PrivPredicate wanted); - Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + /** + * How {@code col} must be rewritten before {@code currentUser} may read it, or empty when it is not masked. + * The returned payload carries a SQL expression, never a parsed one: see {@link DataMaskSpec}. + */ + Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col); - List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl); + /** + * The row-level filters that apply to {@code tbl} for {@code currentUser}, empty when there are none. + * The engine combines them per {@link org.apache.doris.authorization.RowFilterMergeType}. + */ + List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/DataMaskPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/DataMaskPolicy.java deleted file mode 100644 index ca22129628e445..00000000000000 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/DataMaskPolicy.java +++ /dev/null @@ -1,24 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -public interface DataMaskPolicy { - String getMaskTypeDef(); - - String getPolicyIdent(); -} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java index 65a40ae136199c..51c6ae9f1203e9 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java @@ -19,8 +19,16 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterMergeType; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.Env; +import org.apache.doris.common.AnalysisException; import org.apache.doris.common.AuthorizationException; +import org.apache.doris.policy.FilterType; +import org.apache.doris.policy.RowPolicy; + +import com.google.common.collect.ImmutableList; import java.util.List; import java.util.Optional; @@ -81,14 +89,41 @@ public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVau } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { + // The built-in privilege model has no column masking: there is no DDL to define one. return Optional.empty(); } @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { - return Env.getCurrentEnv().getPolicyMgr().getUserPolicies(ctl, db, tbl, currentUser); + List policies = Env.getCurrentEnv().getPolicyMgr().getUserPolicies(ctl, db, tbl, currentUser); + ImmutableList.Builder specs = ImmutableList.builderWithExpectedSize(policies.size()); + for (RowPolicy policy : policies) { + try { + specs.add(new RowFilterSpec(policy.getPolicyIdent(), policy.getFilterSql(), + mergeTypeOf(policy.getFilterType()))); + } catch (AnalysisException e) { + // A policy whose statement no longer parses cannot be turned into a filter, and dropping it + // would silently widen access to the whole table. Fail the query with the same message the + // planner used to raise when it asked the policy for its expression. + throw new org.apache.doris.nereids.exceptions.AnalysisException(e.getMessage(), e); + } + } + return specs.build(); + } + + private static RowFilterMergeType mergeTypeOf(FilterType filterType) { + switch (filterType) { + case PERMISSIVE: + return RowFilterMergeType.PERMISSIVE; + case RESTRICTIVE: + return RowFilterMergeType.RESTRICTIVE; + default: + // Same shape as the planner's merge switch used to have: an unmapped filter type is a bug, + // and guessing either way would change which rows the user sees. + throw new IllegalStateException("Invalid operator"); + } } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDataMaskPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDataMaskPolicy.java deleted file mode 100644 index 91010f80cb378f..00000000000000 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDataMaskPolicy.java +++ /dev/null @@ -1,103 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.analysis.UserIdentity; - -public class RangerDataMaskPolicy implements DataMaskPolicy { - private UserIdentity userIdentity; - private String ctl; - private String db; - private String tbl; - private String col; - private long policyId; - private long policyVersion; - private String maskType; - private String maskTypeDef; - - public RangerDataMaskPolicy(UserIdentity userIdentity, String ctl, String db, String tbl, String col, - long policyId, - long policyVersion, String maskType, String maskTypeDef) { - this.userIdentity = userIdentity; - this.ctl = ctl; - this.db = db; - this.tbl = tbl; - this.col = col; - this.policyId = policyId; - this.policyVersion = policyVersion; - this.maskType = maskType; - this.maskTypeDef = maskTypeDef; - } - - public UserIdentity getUserIdentity() { - return userIdentity; - } - - public String getCtl() { - return ctl; - } - - public String getDb() { - return db; - } - - public String getTbl() { - return tbl; - } - - public String getCol() { - return col; - } - - public long getPolicyId() { - return policyId; - } - - public long getPolicyVersion() { - return policyVersion; - } - - public String getMaskType() { - return maskType; - } - - @Override - public String getMaskTypeDef() { - return maskTypeDef; - } - - @Override - public String getPolicyIdent() { - return getPolicyId() + ":" + getPolicyVersion(); - } - - @Override - public String toString() { - return "RangerDataMaskPolicy{" - + "userIdentity=" + userIdentity - + ", ctl='" + ctl + '\'' - + ", db='" + db + '\'' - + ", tbl='" + tbl + '\'' - + ", col='" + col + '\'' - + ", policyId=" + policyId - + ", policyVersion=" + policyVersion - + ", maskType='" + maskType + '\'' - + ", maskTypeDef='" + maskTypeDef + '\'' - + '}'; - } -} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerRowFilterPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerRowFilterPolicy.java deleted file mode 100644 index 661efcf8a4a852..00000000000000 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerRowFilterPolicy.java +++ /dev/null @@ -1,95 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.analysis.UserIdentity; -import org.apache.doris.nereids.parser.NereidsParser; -import org.apache.doris.nereids.trees.expressions.Expression; - -public class RangerRowFilterPolicy implements RowFilterPolicy { - private UserIdentity userIdentity; - private String ctl; - private String db; - private String tbl; - private long policyId; - private long policyVersion; - private String filterExpr; - - public RangerRowFilterPolicy(UserIdentity userIdentity, String ctl, String db, String tbl, long policyId, - long policyVersion, String filterExpr) { - this.userIdentity = userIdentity; - this.ctl = ctl; - this.db = db; - this.tbl = tbl; - this.policyId = policyId; - this.policyVersion = policyVersion; - this.filterExpr = filterExpr; - } - - public UserIdentity getUserIdentity() { - return userIdentity; - } - - public String getCtl() { - return ctl; - } - - public String getDb() { - return db; - } - - public String getTbl() { - return tbl; - } - - public long getPolicyId() { - return policyId; - } - - public long getPolicyVersion() { - return policyVersion; - } - - public String getFilterExpr() { - return filterExpr; - } - - @Override - public Expression getFilterExpression() { - NereidsParser nereidsParser = new NereidsParser(); - return nereidsParser.parseExpression(filterExpr); - } - - @Override - public String getPolicyIdent() { - return getPolicyId() + ":" + getPolicyVersion(); - } - - @Override - public String toString() { - return "RangerRowFilterPolicy{" - + "userIdentity=" + userIdentity - + ", ctl='" + ctl + '\'' - + ", db='" + db + '\'' - + ", tbl='" + tbl + '\'' - + ", policyId=" + policyId - + ", policyVersion=" + policyVersion - + ", filterExpr='" + filterExpr + '\'' - + '}'; - } -} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RowFilterPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RowFilterPolicy.java deleted file mode 100644 index 678a1927e243dc..00000000000000 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RowFilterPolicy.java +++ /dev/null @@ -1,32 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.common.AnalysisException; -import org.apache.doris.nereids.trees.expressions.Expression; -import org.apache.doris.policy.FilterType; - -public interface RowFilterPolicy { - default FilterType getFilterType() { - return FilterType.RESTRICTIVE; - } - - Expression getFilterExpression() throws AnalysisException; - - String getPolicyIdent(); -} diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/SqlCacheContext.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/SqlCacheContext.java index aeb58b771c6bbf..028c5b7535c42d 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/SqlCacheContext.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/SqlCacheContext.java @@ -19,6 +19,8 @@ import org.apache.doris.analysis.Expr; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.DatabaseIf; import org.apache.doris.catalog.MTMV; import org.apache.doris.catalog.OlapTable; @@ -29,8 +31,6 @@ import org.apache.doris.mtmv.MTMVRelatedTableIf; import org.apache.doris.mysql.FieldInfo; import org.apache.doris.mysql.privilege.Auth; -import org.apache.doris.mysql.privilege.DataMaskPolicy; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.parser.NereidsParser; import org.apache.doris.nereids.trees.expressions.Expression; import org.apache.doris.nereids.trees.expressions.Variable; @@ -79,8 +79,8 @@ public class SqlCacheContext { private final Map usedViews = Maps.newLinkedHashMap(); // value: usedColumns private final Map> checkPrivilegeTablesOrViews = Maps.newLinkedHashMap(); - private final Map> rowPolicies = Maps.newLinkedHashMap(); - private final Map> dataMaskPolicies = Maps.newLinkedHashMap(); + private final Map> rowPolicies = Maps.newLinkedHashMap(); + private final Map> dataMaskPolicies = Maps.newLinkedHashMap(); private final Set usedVariables = Sets.newLinkedHashSet(); // key: the expression which **contains** nondeterministic function, e.g. date_add(date_column, date(now())) // value: the expression which already try to fold nondeterministic function, @@ -275,22 +275,22 @@ public synchronized void addCheckPrivilegeTablesOrViews(TableIf tableIf, Set rowFilterPolicy) { + String catalog, String db, String table, List rowFilterPolicy) { rowPolicies.put(new FullTableName(catalog, db, table), Utils.fastToImmutableList(rowFilterPolicy)); } - public synchronized Map> getRowFilterPolicies() { + public synchronized Map> getRowFilterPolicies() { return ImmutableMap.copyOf(rowPolicies); } public synchronized void addDataMaskPolicy( - String catalog, String db, String table, String columnName, Optional dataMaskPolicy) { + String catalog, String db, String table, String columnName, Optional dataMaskPolicy) { dataMaskPolicies.put( new FullColumnName(catalog, db, table, columnName.toLowerCase(Locale.ROOT)), dataMaskPolicy ); } - public synchronized Map> getDataMaskPolicies() { + public synchronized Map> getDataMaskPolicies() { return ImmutableMap.copyOf(dataMaskPolicies); } @@ -378,7 +378,7 @@ public synchronized Map> getCheckPrivilegeTablesOrVie return ImmutableMap.copyOf(checkPrivilegeTablesOrViews); } - public synchronized Map> getRowPolicies() { + public synchronized Map> getRowPolicies() { return ImmutableMap.copyOf(rowPolicies); } @@ -480,8 +480,8 @@ public synchronized PUniqueId doComputeCacheKeyMd5( .append("=") .append(pair.value().toSql()); } - for (Entry> entry : rowPolicies.entrySet()) { - List policy = entry.getValue(); + for (Entry> entry : rowPolicies.entrySet()) { + List policy = entry.getValue(); if (policy.isEmpty()) { continue; } @@ -490,7 +490,7 @@ public synchronized PUniqueId doComputeCacheKeyMd5( .append("=") .append(policy); } - for (Entry> entry : dataMaskPolicies.entrySet()) { + for (Entry> entry : dataMaskPolicies.entrySet()) { if (!entry.getValue().isPresent()) { continue; } diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java index 05336a7771e784..ec0e275c69f810 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java @@ -2538,7 +2538,10 @@ public Command visitCreateRowPolicy(CreateRowPolicyContext ctx) { ctx.EXISTS() != null, new TableNameInfo(nameParts), Optional.of(filterType), ctx.user == null ? null : visitUserIdentify(ctx.user), ctx.roleName == null ? null : ctx.roleName.getText(), - Optional.of(getExpression(ctx.booleanExpression())), ImmutableMap.of()); + Optional.of(getExpression(ctx.booleanExpression())), + // The predicate is kept as the user wrote it, because that text - not a rendering of the + // parsed tree - is what the authorization layer hands back to the planner. + getOriginSql(ctx.booleanExpression()), ImmutableMap.of()); } @Override @@ -2548,7 +2551,7 @@ public Command visitCreateStoragePolicy(CreateStoragePolicyContext ctx) { : Maps.newHashMap(); return new CreatePolicyCommand(PolicyTypeEnum.STORAGE, ctx.name.getText(), ctx.EXISTS() != null, null, Optional.empty(), - null, null, Optional.empty(), properties); + null, null, Optional.empty(), null, properties); } @Override diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreatePolicyCommand.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreatePolicyCommand.java index 8dca6d83a12e44..16aa21ec91bf9e 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreatePolicyCommand.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreatePolicyCommand.java @@ -61,6 +61,8 @@ public class CreatePolicyCommand extends Command implements ForwardWithSync { private final UserIdentity user; private final String roleName; private final Optional wherePredicate; + // The row predicate exactly as written, null for a storage policy. + private final String wherePredicateSql; private final Map properties; /** @@ -68,7 +70,7 @@ public class CreatePolicyCommand extends Command implements ForwardWithSync { */ public CreatePolicyCommand(PolicyTypeEnum policyType, String policyName, boolean ifNotExists, TableNameInfo tableNameInfo, Optional filterType, UserIdentity user, String roleName, - Optional wherePredicate, Map properties) { + Optional wherePredicate, String wherePredicateSql, Map properties) { super(PlanType.CREATE_POLICY_COMMAND); this.policyType = policyType; this.policyName = policyName; @@ -78,6 +80,7 @@ public CreatePolicyCommand(PolicyTypeEnum policyType, String policyName, boolean this.user = user; this.roleName = roleName; this.wherePredicate = wherePredicate; + this.wherePredicateSql = wherePredicateSql; this.properties = properties; } @@ -85,6 +88,10 @@ public Optional getWherePredicate() { return wherePredicate; } + public String getWherePredicateSql() { + return wherePredicateSql; + } + public Map getProperties() { return properties; } diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCheckPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCheckPolicy.java index 6d204304cb18c0..a40127df7bfe85 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCheckPolicy.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCheckPolicy.java @@ -18,17 +18,16 @@ package org.apache.doris.nereids.trees.plans.logical; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.DatabaseIf; import org.apache.doris.catalog.TableIf; import org.apache.doris.datasource.CatalogIf; import org.apache.doris.mysql.privilege.AccessControllerManager; -import org.apache.doris.mysql.privilege.DataMaskPolicy; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.CascadesContext; import org.apache.doris.nereids.SqlCacheContext; import org.apache.doris.nereids.StatementContext; import org.apache.doris.nereids.analyzer.UnboundAlias; -import org.apache.doris.nereids.exceptions.AnalysisException; import org.apache.doris.nereids.memo.GroupExpression; import org.apache.doris.nereids.parser.NereidsParser; import org.apache.doris.nereids.properties.LogicalProperties; @@ -180,10 +179,10 @@ public RelatedPolicy findPolicy(LogicalPlan logicalPlan, CascadesContext cascade Optional sqlCacheContext = statementContext.getSqlCacheContext(); boolean hasDataMask = false; for (Slot slot : logicalPlan.getOutput()) { - Optional dataMaskPolicy = accessManager.evalDataMaskPolicy( + Optional dataMaskPolicy = accessManager.evalDataMaskPolicy( currentUserIdentity, ctlName, dbName, tableName, slot.getName()); if (dataMaskPolicy.isPresent()) { - Expression unboundExpr = nereidsParser.parseExpression(dataMaskPolicy.get().getMaskTypeDef()); + Expression unboundExpr = nereidsParser.parseExpression(dataMaskPolicy.get().getMaskSql()); Expression childOfAlias = unboundExpr instanceof UnboundAlias ? unboundExpr.child(0) : unboundExpr; Alias alias = new Alias( @@ -201,7 +200,7 @@ public RelatedPolicy findPolicy(LogicalPlan logicalPlan, CascadesContext cascade } } - List rowPolicies = accessManager.evalRowFilterPolicies( + List rowPolicies = accessManager.evalRowFilterPolicies( currentUserIdentity, ctlName, dbName, tableName); if (sqlCacheContext.isPresent()) { sqlCacheContext.get().setRowFilterPolicy(ctlName, dbName, tableName, rowPolicies); @@ -223,17 +222,15 @@ private RelatedPolicy findPolicyByMvRefresh(Map> mvRefr return RelatedPolicy.NO_POLICY; } - private Expression mergeRowPolicy(List policies) { + private Expression mergeRowPolicy(List policies) { List orList = new ArrayList<>(); List andList = new ArrayList<>(); - for (RowFilterPolicy policy : policies) { - Expression wherePredicate = null; - try { - wherePredicate = policy.getFilterExpression(); - } catch (org.apache.doris.common.AnalysisException e) { - throw new AnalysisException(e.getMessage(), e); - } - switch (policy.getFilterType()) { + NereidsParser nereidsParser = new NereidsParser(); + for (RowFilterSpec policy : policies) { + // The authorization source hands us the predicate as SQL text - the form both a Ranger policy and + // a CREATE ROW POLICY statement natively have - and parsing it is the engine's job. + Expression wherePredicate = nereidsParser.parseExpression(policy.getFilterSql()); + switch (policy.getMergeType()) { case PERMISSIVE: orList.add(wherePredicate); break; diff --git a/fe/fe-core/src/main/java/org/apache/doris/policy/RowPolicy.java b/fe/fe-core/src/main/java/org/apache/doris/policy/RowPolicy.java index e83a1191c0b47d..c25dd713758759 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/policy/RowPolicy.java +++ b/fe/fe-core/src/main/java/org/apache/doris/policy/RowPolicy.java @@ -21,7 +21,6 @@ import org.apache.doris.catalog.Column; import org.apache.doris.catalog.ScalarType; import org.apache.doris.common.AnalysisException; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.parser.NereidsParser; import org.apache.doris.nereids.trees.expressions.Expression; import org.apache.doris.nereids.trees.plans.commands.CreatePolicyCommand; @@ -30,6 +29,8 @@ import com.google.common.collect.Lists; import com.google.gson.annotations.SerializedName; import lombok.Data; +import lombok.EqualsAndHashCode; +import lombok.ToString; import org.apache.commons.lang3.StringUtils; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -43,7 +44,7 @@ * Save policy for filtering data. **/ @Data -public class RowPolicy extends Policy implements RowFilterPolicy { +public class RowPolicy extends Policy { public static final ShowResultSetMetaData ROW_META_DATA = ShowResultSetMetaData.builder() @@ -101,6 +102,13 @@ public class RowPolicy extends Policy implements RowFilterPolicy { private Expression wherePredicate = null; + // Derived from originStmt on first use, never persisted: see getFilterSql(). Excluded from equality and + // toString because it is a lazily filled cache - two policies created from the same statement must not + // compare differently depending on whether a query has already asked for the predicate text. + @EqualsAndHashCode.Exclude + @ToString.Exclude + private volatile String wherePredicateSql = null; + public RowPolicy() { super(PolicyTypeEnum.ROW); } @@ -221,15 +229,39 @@ public boolean isInvalid() { return (wherePredicate == null); } - @Override - public Expression getFilterExpression() throws AnalysisException { + /** + * The predicate as SQL text, which is the form the authorization layer hands to the planner. + * + *

It is the text the administrator wrote, recovered from the stored statement - not a rendering of + * the parsed predicate. Rendering would not survive the round trip: {@code toSql()} on a compound + * predicate produces the diagnostic form {@code AND[a,b]}, which does not parse back, so any policy + * combining two conditions would break.

+ */ + public String getFilterSql() throws AnalysisException { if (wherePredicate == null) { throw new AnalysisException("Invalid row policy [" + getPolicyIdent() + "], " + getOriginStmt()); } - return wherePredicate; + if (wherePredicateSql == null) { + wherePredicateSql = parseWherePredicateSql(); + } + return wherePredicateSql; + } + + private String parseWherePredicateSql() throws AnalysisException { + try { + CreatePolicyCommand command = (CreatePolicyCommand) new NereidsParser().parseSingle(getOriginStmt()); + if (!StringUtils.isEmpty(command.getWherePredicateSql())) { + return command.getWherePredicateSql(); + } + } catch (Exception e) { + LOG.warn("failed to recover the predicate text of row policy [{}]", getPolicyIdent(), e); + } + // The statement parsed once already (that is where wherePredicate came from), so reaching here means + // the stored statement no longer matches what the parser produces. Refuse the query rather than let + // the table be read unfiltered. + throw new AnalysisException("Invalid row policy [" + getPolicyIdent() + "], " + getOriginStmt()); } - @Override public String getPolicyIdent() { return getPolicyName(); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java b/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java index 3aba8399659211..23b80781635bdf 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/common/cache/NereidsSqlCacheDataPolicyTest.java @@ -18,13 +18,11 @@ package org.apache.doris.common.cache; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.Env; import org.apache.doris.common.jmockit.Deencapsulation; import org.apache.doris.mysql.privilege.AccessControllerManager; -import org.apache.doris.mysql.privilege.DataMaskPolicy; -import org.apache.doris.mysql.privilege.RangerDataMaskPolicy; -import org.apache.doris.mysql.privilege.RangerRowFilterPolicy; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.SqlCacheContext; import com.google.common.collect.ImmutableList; @@ -45,9 +43,9 @@ * *

An authorization source builds its policy objects fresh on every evaluation (it is answering a request, * not handing out a cached object), so value equality is the source's obligation, and the "unchanged" tests - * below are what enforce it. They are RED for the Ranger policy types, which have no equals: the comparison - * degrades to identity, every lookup reads as a policy change, and a user with any row filter or column mask - * configured never gets a cache hit at all.

+ * below are what enforce it. They were RED before the payload rework: the Ranger policy types carried no + * equals, so the comparison degraded to identity, every lookup read as a policy change, and a user with any + * row filter or column mask configured never got a cache hit at all.

* *

The "changed" tests pin the other half of the contract, so a fix cannot buy cache hits by weakening the * comparison: an edited policy must still invalidate, or the cache keeps serving rows the user may no longer @@ -61,12 +59,13 @@ public class NereidsSqlCacheDataPolicyTest { private static final String COL = "phone"; private static final UserIdentity USER = UserIdentity.ROOT; - private RowFilterPolicy rowFilter(long policyVersion, String filterExpr) { - return new RangerRowFilterPolicy(USER, CTL, DB, TBL, 7L, policyVersion, filterExpr); + /** Shaped like what a Ranger controller returns: a fresh object per evaluation, identified by id:version. */ + private RowFilterSpec rowFilter(long policyVersion, String filterExpr) { + return RowFilterSpec.restrictive("7:" + policyVersion, filterExpr); } - private DataMaskPolicy dataMask(long policyVersion, String maskTypeDef) { - return new RangerDataMaskPolicy(USER, CTL, DB, TBL, COL, 7L, policyVersion, "CUSTOM", maskTypeDef); + private DataMaskSpec dataMask(long policyVersion, String maskSql) { + return new DataMaskSpec("7:" + policyVersion, maskSql); } /** @@ -74,11 +73,11 @@ private DataMaskPolicy dataMask(long policyVersion, String maskTypeDef) { * invoked per call so every evaluation yields a distinct object, exactly as a live authorization source * behaves. */ - private Env mockEnvEvaluating(java.util.function.Supplier rowFilterSupplier, - java.util.function.Supplier> dataMaskSupplier) { + private Env mockEnvEvaluating(java.util.function.Supplier rowFilterSupplier, + java.util.function.Supplier> dataMaskSupplier) { AccessControllerManager accessManager = Mockito.mock(AccessControllerManager.class); Mockito.doAnswer(invocation -> { - RowFilterPolicy evaluated = rowFilterSupplier.get(); + RowFilterSpec evaluated = rowFilterSupplier.get(); return evaluated == null ? ImmutableList.of() : ImmutableList.of(evaluated); }) .when(accessManager).evalRowFilterPolicies(ArgumentMatchers.any(UserIdentity.class), @@ -101,14 +100,14 @@ private boolean dataMaskPoliciesChanged(Env env, SqlCacheContext context) { return Deencapsulation.invoke(new NereidsSqlCacheManager(), "dataMaskPoliciesChanged", USER, env, context); } - private SqlCacheContext contextWithRowFilter(RowFilterPolicy policy) { + private SqlCacheContext contextWithRowFilter(RowFilterSpec policy) { SqlCacheContext context = new SqlCacheContext(USER); - List policies = policy == null ? ImmutableList.of() : ImmutableList.of(policy); + List policies = policy == null ? ImmutableList.of() : ImmutableList.of(policy); context.setRowFilterPolicy(CTL, DB, TBL, policies); return context; } - private SqlCacheContext contextWithDataMask(Optional policy) { + private SqlCacheContext contextWithDataMask(Optional policy) { SqlCacheContext context = new SqlCacheContext(USER); context.addDataMaskPolicy(CTL, DB, TBL, COL, policy); return context; diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java index 163518b33a594c..58a18566f405cd 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java @@ -19,6 +19,8 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.common.AuthorizationException; import com.google.common.collect.ImmutableList; @@ -115,13 +117,13 @@ public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVau } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { return Optional.empty(); } @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { return ImmutableList.of(); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java index e0e1dd36b2a4d6..9a3f1af03b447c 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java @@ -19,6 +19,7 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisResource; import org.apache.doris.common.AuthorizationException; @@ -222,14 +223,14 @@ public void testDataMask() { RangerDorisAccessController ac = new RangerDorisAccessController(plugin); UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); // MASK_NULL - Optional policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col1"); - Assertions.assertEquals("NULL", policy.get().getMaskTypeDef()); + Optional policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col1"); + Assertions.assertEquals("NULL", policy.get().getMaskSql()); // MASK_NONE policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col2"); Assertions.assertTrue(!policy.isPresent()); // CUSTOM policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col3"); - Assertions.assertEquals("hex(col3)", policy.get().getMaskTypeDef()); + Assertions.assertEquals("hex(col3)", policy.get().getMaskSql()); // Others policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col4"); Assertions.assertTrue(!policy.isPresent()); diff --git a/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java b/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java index 0473bce3250b13..76c7d3d91eb11f 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java +++ b/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java @@ -19,6 +19,8 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.Column; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.PrimitiveType; @@ -30,21 +32,16 @@ import org.apache.doris.datasource.test.TestExternalCatalog.TestCatalogProvider; import org.apache.doris.mysql.privilege.AccessControllerManager; import org.apache.doris.mysql.privilege.CatalogAccessController; -import org.apache.doris.mysql.privilege.DataMaskPolicy; import org.apache.doris.mysql.privilege.PrivPredicate; -import org.apache.doris.mysql.privilege.RowFilterPolicy; import org.apache.doris.nereids.exceptions.AnalysisException; -import org.apache.doris.nereids.parser.NereidsParser; import org.apache.doris.nereids.pattern.GeneratedMemoPatterns; import org.apache.doris.nereids.rules.RulePromise; import org.apache.doris.nereids.trees.expressions.Alias; import org.apache.doris.nereids.trees.expressions.EqualTo; -import org.apache.doris.nereids.trees.expressions.Expression; import org.apache.doris.nereids.trees.expressions.NamedExpression; import org.apache.doris.nereids.trees.expressions.Slot; import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral; import org.apache.doris.nereids.util.PlanChecker; -import org.apache.doris.policy.FilterType; import org.apache.doris.utframe.TestWithFeService; import com.google.common.collect.ImmutableList; @@ -388,7 +385,7 @@ public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVau } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { List dataMaskingPolicies = dataMaskings.get(); if (dataMaskingPolicies == null) { @@ -397,32 +394,21 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Str for (CustomDataMaskingPolicy dataMaskingPolicy : dataMaskingPolicies) { if (dataMaskingPolicy.column.equalsIgnoreCase(col)) { - return Optional.of(dataMaskingPolicy); + return Optional.of(dataMaskingPolicy.toSpec()); } } return Optional.empty(); } @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, + public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, String tbl) { List customRowPolicies = rowPolicies.get(); if (customRowPolicies == null) { return ImmutableList.of(); } - NereidsParser nereidsParser = new NereidsParser(); return customRowPolicies.stream() - .map(p -> new RowFilterPolicy() { - @Override - public Expression getFilterExpression() { - return nereidsParser.parseExpression(p.filter); - } - - @Override - public String getPolicyIdent() { - return "custom policy: " + p.filter; - } - }) + .map(CustomRowPolicy::toSpec) .collect(Collectors.toList()); } } @@ -529,7 +515,7 @@ public boolean isSameTable(String catalog, String db, String tbl) { } } - private static class CustomRowPolicy implements RowFilterPolicy { + private static class CustomRowPolicy { private final String user; private final String filter; @@ -542,23 +528,14 @@ public String getUser() { return user; } - @Override - public Expression getFilterExpression() { - return new NereidsParser().parseExpression(filter); - } - - @Override - public String getPolicyIdent() { - return "custom policy: " + filter; - } - - @Override - public FilterType getFilterType() { - return FilterType.PERMISSIVE; + // Restrictive, which is what this fixture has always produced: the policy object the controller used + // to return took the interface default and never carried the PERMISSIVE the fixture declared. + public RowFilterSpec toSpec() { + return RowFilterSpec.restrictive("custom policy: " + filter, filter); } } - private static class CustomDataMaskingPolicy implements DataMaskPolicy { + private static class CustomDataMaskingPolicy { private final String user; private final String column; private final String project; @@ -573,14 +550,8 @@ public String getUser() { return user; } - @Override - public String getMaskTypeDef() { - return project; - } - - @Override - public String getPolicyIdent() { - return "custom policy: " + project; + public DataMaskSpec toSpec() { + return new DataMaskSpec("custom policy: " + project, project); } } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/CheckRowPolicyTest.java b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/CheckRowPolicyTest.java index a67de805825c06..5b71ef543fe459 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/CheckRowPolicyTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/CheckRowPolicyTest.java @@ -20,6 +20,7 @@ import org.apache.doris.analysis.TablePattern; import org.apache.doris.analysis.UserDesc; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.catalog.AccessPrivilege; import org.apache.doris.catalog.AccessPrivilegeWithCols; import org.apache.doris.catalog.Database; @@ -28,11 +29,11 @@ import org.apache.doris.common.FeConstants; import org.apache.doris.common.jmockit.Deencapsulation; import org.apache.doris.mysql.privilege.AccessControllerManager; -import org.apache.doris.mysql.privilege.DataMaskPolicy; import org.apache.doris.nereids.StatementContext; import org.apache.doris.nereids.analyzer.UnboundRelation; import org.apache.doris.nereids.exceptions.AnalysisException; import org.apache.doris.nereids.trees.expressions.EqualTo; +import org.apache.doris.nereids.trees.expressions.Or; import org.apache.doris.nereids.trees.expressions.StatementScopeIdGenerator; import org.apache.doris.nereids.trees.plans.Plan; import org.apache.doris.nereids.trees.plans.commands.CreateUserCommand; @@ -105,17 +106,9 @@ protected void runBeforeAll() throws Exception { String tbl = invocation.getArgument(3); String col = invocation.getArgument(4); return tbl.equalsIgnoreCase(tableNameRanddomDist) - ? Optional.of(new DataMaskPolicy() { - @Override - public String getMaskTypeDef() { - return String.format("concat(%s, '_****_', %s)", col, col); - } - - @Override - public String getPolicyIdent() { - return String.format("custom policy: concat(%s, '_****_', %s)", col, col); - } - }) + ? Optional.of(new DataMaskSpec( + String.format("custom policy: concat(%s, '_****_', %s)", col, col), + String.format("concat(%s, '_****_', %s)", col, col))) : Optional.empty(); }).when(spyAcm).evalDataMaskPolicy( Mockito.any(UserIdentity.class), Mockito.anyString(), @@ -205,6 +198,61 @@ public void checkOnePolicy() throws Exception { + tableName); } + /** + * Two permissive policies widen each other: the user sees rows matching either one, so they must be + * ORed into a single conjunct. A single policy cannot tell OR from AND, so this is what actually pins + * the merge type the authorization layer carries alongside each filter - drop it and the two predicates + * become an AND, which lets the user see nothing at all. + */ + @Test + public void checkTwoPermissivePoliciesAreOred() throws Exception { + useUser(userName); + LogicalRelation relation = new LogicalOlapScan(StatementScopeIdGenerator.newRelationId(), olapTable, + Arrays.asList(fullDbName)); + LogicalCheckPolicy checkPolicy = new LogicalCheckPolicy<>(relation); + createPolicy("CREATE ROW POLICY " + policyName + " ON " + tableName + + " AS PERMISSIVE TO " + userName + " USING (k1 = 1)"); + createPolicy("CREATE ROW POLICY " + policyName + "_second ON " + tableName + + " AS PERMISSIVE TO " + userName + " USING (k1 = 2)"); + try { + Plan plan = PlanRewriter.bottomUpRewrite(checkPolicy, connectContext, new CheckPolicy()); + + LogicalFilter filter = (LogicalFilter) plan; + Assertions.assertEquals(1, filter.getConjuncts().size(), + "permissive policies must merge into one disjunction, not into separate conjuncts"); + Assertions.assertTrue(ImmutableList.copyOf(filter.getConjuncts()).get(0) instanceof Or, + "the user must see rows matching either permissive policy"); + } finally { + // A leaked policy would change what every later test in this class plans. + dropPolicy("DROP ROW POLICY " + policyName + " ON " + tableName); + dropPolicy("DROP ROW POLICY " + policyName + "_second ON " + tableName); + } + } + + /** Restrictive policies narrow each other, so they stay separate conjuncts (ANDed). */ + @Test + public void checkTwoRestrictivePoliciesAreAnded() throws Exception { + useUser(userName); + LogicalRelation relation = new LogicalOlapScan(StatementScopeIdGenerator.newRelationId(), olapTable, + Arrays.asList(fullDbName)); + LogicalCheckPolicy checkPolicy = new LogicalCheckPolicy<>(relation); + createPolicy("CREATE ROW POLICY " + policyName + " ON " + tableName + + " AS RESTRICTIVE TO " + userName + " USING (k1 = 1)"); + createPolicy("CREATE ROW POLICY " + policyName + "_second ON " + tableName + + " AS RESTRICTIVE TO " + userName + " USING (k2 = 2)"); + try { + Plan plan = PlanRewriter.bottomUpRewrite(checkPolicy, connectContext, new CheckPolicy()); + + LogicalFilter filter = (LogicalFilter) plan; + Assertions.assertEquals(2, filter.getConjuncts().size(), + "restrictive policies must all hold, so each stays its own conjunct"); + filter.getConjuncts().forEach(conjunct -> Assertions.assertTrue(conjunct instanceof EqualTo)); + } finally { + dropPolicy("DROP ROW POLICY " + policyName + " ON " + tableName); + dropPolicy("DROP ROW POLICY " + policyName + "_second ON " + tableName); + } + } + @Test public void checkOnePolicyRandomDist() throws Exception { useUser(userName); diff --git a/fe/fe-core/src/test/java/org/apache/doris/policy/RowPolicyFilterSqlTest.java b/fe/fe-core/src/test/java/org/apache/doris/policy/RowPolicyFilterSqlTest.java new file mode 100644 index 00000000000000..79dbf419c50d98 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/policy/RowPolicyFilterSqlTest.java @@ -0,0 +1,107 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.policy; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.common.AnalysisException; +import org.apache.doris.nereids.parser.NereidsParser; +import org.apache.doris.nereids.trees.expressions.Expression; +import org.apache.doris.nereids.trees.plans.commands.CreatePolicyCommand; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +/** + * A built-in row policy is stored as a statement but handed to the planner as the SQL text of its predicate. + * The recovered text has to be faithful: whatever the planner parses back must be the same predicate the + * administrator wrote, or the policy silently filters different rows than it says it does. + * + *

The corpus below is what row policies actually contain - comparisons, set membership, pattern matches, + * null tests, boolean structure with mixed precedence, function calls and the literal shapes that are easiest + * to mangle (quotes inside strings, negative numbers, dates). Boolean structure is the case that matters + * most: {@code toSql()} on a compound predicate yields the diagnostic form {@code AND[a,b]}, so recovering + * the text from the stored statement rather than re-rendering the tree is what keeps these working.

+ */ +public class RowPolicyFilterSqlTest { + + private static final NereidsParser PARSER = new NereidsParser(); + + @ParameterizedTest + @ValueSource(strings = { + "k1 = 1", + "k1 <> 1", + "k1 >= 1 and k2 < 10", + "k1 = 1 or k2 = 2", + "not (k1 = 1)", + "k1 = 1 and (k2 = 2 or k2 = 3)", + "(k1 = 1 or k2 = 2) and k3 = 3", + "k1 in (1, 2, 3)", + "k1 not in (1, 2)", + "region = 'cn'", + "region = 'it''s'", + "name like 'a%'", + "name not like '%b'", + "k1 is null", + "k1 is not null", + "k1 between 1 and 10", + "k1 = -1", + "amount > 1.5", + "dt = date '2024-01-01'", + "dt > '2024-01-01 10:00:00'", + "upper(region) = 'CN'", + "concat(a, b) = 'ab'", + "substr(phone, 1, 3) = '138'", + "k1 + 1 > 2", + "k1 = 1 and region in ('cn', 'us') and name like 'x%'" + }) + public void testPredicateSurvivesTheRoundTripToThePlanner(String original) throws AnalysisException { + RowPolicy policy = policyOver(original); + + Expression asHandedToThePlanner = PARSER.parseExpression(policy.getFilterSql()); + + Assertions.assertEquals(PARSER.parseExpression(original), asHandedToThePlanner, + "the predicate the planner receives is not the one the policy was created with: " + original + + " became " + policy.getFilterSql()); + } + + /** + * A policy whose statement no longer parses (an upgrade dropped a function, say) has no predicate to + * render. It must fail the query, never quietly disappear - a vanished row filter exposes the whole + * table. + */ + @Test + public void testUnparseablePolicyFailsInsteadOfVanishing() { + RowPolicy broken = new RowPolicy(1L, "p1", "internal", "db1", "t1", UserIdentity.ROOT, null, + "CREATE ROW POLICY p1 ON db1.t1 AS RESTRICTIVE TO root USING (gone_function(k1))", 0, + FilterType.RESTRICTIVE, null); + + AnalysisException thrown = Assertions.assertThrows(AnalysisException.class, broken::getFilterSql); + Assertions.assertTrue(thrown.getMessage().contains("Invalid row policy"), + "the error must name the broken policy so an operator can find it: " + thrown.getMessage()); + } + + /** Builds the policy the way CREATE ROW POLICY does: statement text plus the predicate parsed from it. */ + private RowPolicy policyOver(String predicate) { + String statement = "CREATE ROW POLICY p1 ON db1.t1 AS RESTRICTIVE TO root USING (" + predicate + ")"; + CreatePolicyCommand command = (CreatePolicyCommand) PARSER.parseSingle(statement); + return new RowPolicy(1L, "p1", "internal", "db1", "t1", UserIdentity.ROOT, null, + statement, 0, FilterType.RESTRICTIVE, command.getWherePredicate().get()); + } +} From bdcd7648aa888dfc612fa5bfc96150f8610aa176 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 09:18:38 +0800 Subject: [PATCH 04/22] [test](authorization) freeze today's access-control decisions in a golden matrix Two reworks are queued behind this: dropping the hasGlobal argument from the controller interface, and collapsing AccessControllerManager into pure routing with no cross-plugin OR. Both are meant to change structure and nothing else, and "nothing else" is only checkable against a recording made beforehand. The matrix crosses the default controller (built-in / Ranger) with the catalog kind (internal, unbound external, Ranger-governed external), five callers whose privilege shapes each pin one invariant, every PrivPredicate constant found by reflection, and every check the manager exposes. The built-in half runs against a real FE with real grants and a real row policy; only Ranger's policy engine is stubbed, and it denies the built-in admin everywhere so that wherever that user still passes on a Ranger-governed resource, the verdict provably came from the engine's OR rather than from Ranger. Each row lists the actions that were allowed, so a flipped decision shows up as one action name appearing or disappearing on one line. Recording it surfaced three things worth writing down: - checkCloudPriv cannot be probed at all on a non-cloud FE: the built-in path casts the system info service to its cloud subclass unconditionally, so every caller that does not already hold a global privilege gets a ClassCastException. That probe is excluded rather than frozen. - "the default workload group is always allowed" is implemented twice, in Auth and again in Role, so removing either one alone changes nothing. - a column check rejects any privilege carrying neither SELECT nor LOAD with an IllegalStateException, and whether a caller hits it depends on whether the manager's global short circuit answered first - which is exactly the ordering the rework moves, so the baseline records that outcome as its own column. Co-Authored-By: Claude Opus 5 (1M context) --- .../AccessControlBehaviorBaselineTest.java | 405 ++++++++++++++++++ .../StubRangerAccessControllerFactory.java | 38 ++ .../privilege/StubRangerPolicyEngine.java | 172 ++++++++ ...is.mysql.privilege.AccessControllerFactory | 3 +- .../access-control-behavior-baseline.txt | 323 ++++++++++++++ 5 files changed, 940 insertions(+), 1 deletion(-) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java create mode 100644 fe/fe-core/src/test/resources/access-control-behavior-baseline.txt diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java new file mode 100644 index 00000000000000..4abeee55ee7c75 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java @@ -0,0 +1,405 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.catalog.Column; +import org.apache.doris.catalog.Env; +import org.apache.doris.catalog.PrimitiveType; +import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; +import org.apache.doris.common.Config; +import org.apache.doris.common.FeConstants; +import org.apache.doris.common.UserException; +import org.apache.doris.common.jmockit.Deencapsulation; +import org.apache.doris.datasource.CatalogMgr; +import org.apache.doris.datasource.InternalCatalog; +import org.apache.doris.datasource.test.TestExternalCatalog.TestCatalogProvider; +import org.apache.doris.nereids.parser.NereidsParser; +import org.apache.doris.nereids.trees.plans.commands.GrantResourcePrivilegeCommand; +import org.apache.doris.nereids.trees.plans.logical.LogicalPlan; +import org.apache.doris.qe.StmtExecutor; +import org.apache.doris.resource.workloadgroup.WorkloadGroupMgr; +import org.apache.doris.utframe.TestWithFeService; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.ImmutableSet; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.TreeMap; +import java.util.function.BooleanSupplier; +import java.util.stream.Collectors; + +/** + * Records, in one golden file, every decision {@link AccessControllerManager} makes over the matrix + * (resource kind x action x built-in/Ranger x privilege level of the caller). + * + *

Why this exists. Two reworks are queued behind it: dropping the {@code hasGlobal} argument from + * the controller interface (moving "global first, then fine grained" inside the built-in implementation), and + * collapsing the manager into pure routing with no cross-plugin OR. Both are supposed to change structure and + * nothing else. "Nothing else" is only checkable against a recording made before the change, so this file is + * that recording: after either rework, {@code git diff} on the baseline must be empty. + * + *

What is real and what is faked. The built-in half is entirely real - a real FE, real users, real + * {@code GRANT} statements, a real row policy - because a baseline built on hand-poked internal state would + * freeze this test's assumptions rather than the product's behaviour. The Ranger half runs the production + * {@link org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController} over the deterministic + * {@link StubRangerPolicyEngine}; only the policy engine is a stub. + * + *

Reading a row. {@code | | | | }. For the + * check probes the verdict lists exactly the actions that were allowed, so a single flipped cell shows up as + * one action name appearing or disappearing on one line. {@code -} means "nothing allowed" / "no policy". + * Scope {@code -} marks the system-level resources, which are always answered by the default controller. + * + *

Deliberately frozen oddities. Some rows record behaviour that looks wrong and is kept anyway, + * because Phase 0's contract is structure-only: + *

    + *
  • {@code node_user} holds only global NODE_PRIV, and {@link Auth} refuses NODE privileges at + * catalog/database/table level - yet the manager's global short circuit lets OPERATOR through there;
    • + *
  • {@code admin_user} is denied by Ranger everywhere, yet passes on the Ranger-governed catalog: that is + * the cross-plugin OR the rework must reproduce from inside the plugin;
    • + *
  • a built-in per-table GRANT on a Ranger-governed catalog is ignored ({@code local_user} on + * {@code ext_ranger}), because the OR only exists at the global level;
    • + *
  • the workload group named {@code normal} is allowed unconditionally, in both implementations;
    • + *
  • with {@code skip_catalog_priv_check} on, a catalog bound to an external plugin answers SELECT/SHOW + * with a flat yes.
    • + *
+ * + *

Regenerating. Run this test; on mismatch it writes the full current matrix next to the build + * output and names the path. Copy it over {@code src/test/resources/access-control-behavior-baseline.txt} + * only together with a justification for every changed line - each one is a user-visible privilege change. + */ +public class AccessControlBehaviorBaselineTest extends TestWithFeService { + private static final String BASELINE_RESOURCE = "/access-control-behavior-baseline.txt"; + private static final Path ACTUAL_DUMP = Paths.get("target", "access-control-behavior-baseline.actual.txt"); + + private static final String DB = StubRangerPolicyEngine.ALLOWED_DB; + private static final String TBL = StubRangerPolicyEngine.ALLOWED_TABLE; + /** Ranger grants this column to {@code ranger_user}; the built-in grants are table wide. */ + private static final String COL_ALLOWED = StubRangerPolicyEngine.ALLOWED_COLUMN; + private static final String COL_DENIED = "pcol2"; + + private static final String CTL_PLAIN = "ext_plain"; + private static final String CTL_RANGER = "ext_ranger"; + private static final List CATALOGS = + ImmutableList.of(InternalCatalog.INTERNAL_CATALOG_NAME, CTL_PLAIN, CTL_RANGER); + + private static final String ADMIN_USER = "admin_user"; + private static final String NODE_USER = "node_user"; + private static final String LOCAL_USER = "local_user"; + private static final String RANGER_USER = StubRangerPolicyEngine.ALLOWED_USER; + private static final String NOBODY = "nobody"; + private static final List USERS = + ImmutableList.of(ADMIN_USER, NODE_USER, LOCAL_USER, RANGER_USER, NOBODY); + + /** Every {@link PrivPredicate} constant, by name. A new constant lands here on its own and turns red. */ + private static final Map ACTIONS = allPrivPredicates(); + + /** Metadata for the two external catalogs: only the names matter, nothing reads a row. */ + public static class BaselineCatalogProvider implements TestCatalogProvider { + @Override + public Map>> getMetadata() { + return ImmutableMap.of(DB, ImmutableMap.of(TBL, ImmutableList.of( + new Column(COL_ALLOWED, PrimitiveType.INT), + new Column(COL_DENIED, PrimitiveType.INT)))); + } + } + + @Override + protected void runBeforeAll() throws Exception { + FeConstants.runningUnitTest = true; + + createDatabase(DB); + useDatabase(DB); + createTable("create table " + TBL + " (" + COL_ALLOWED + " int, " + COL_DENIED + " int)" + + " distributed by hash(" + COL_ALLOWED + ") buckets 1" + + " properties(\"replication_num\" = \"1\");"); + + String provider = BaselineCatalogProvider.class.getName(); + createCatalog("create catalog " + CTL_PLAIN + " properties(" + + "\"type\"=\"test\", \"catalog_provider.class\"=\"" + provider + "\")"); + createCatalog("create catalog " + CTL_RANGER + " properties(" + + "\"type\"=\"test\", \"catalog_provider.class\"=\"" + provider + "\"," + + "\"" + CatalogMgr.ACCESS_CONTROLLER_CLASS_PROP + "\"=\"" + + StubRangerAccessControllerFactory.class.getName() + "\")"); + + for (String user : USERS) { + addUser(user, true); + } + grantPriv("GRANT ADMIN_PRIV ON *.*.* TO '" + ADMIN_USER + "'@'%'"); + grantPriv("GRANT NODE_PRIV ON *.*.* TO '" + NODE_USER + "'@'%'"); + // The same table-level grant on all three catalogs: on the Ranger-governed one it is ignored today, + // and that asymmetry is one of the things the baseline is here to hold still. + for (String catalog : CATALOGS) { + grantPriv("GRANT SELECT_PRIV ON " + catalog + "." + DB + "." + TBL + + " TO '" + LOCAL_USER + "'@'%'"); + } + grantResourcePriv("GRANT USAGE_PRIV ON RESOURCE '" + StubRangerPolicyEngine.ALLOWED_RESOURCE + + "' TO '" + LOCAL_USER + "'@'%'"); + grantResourcePriv("GRANT USAGE_PRIV ON WORKLOAD GROUP '" + + StubRangerPolicyEngine.ALLOWED_WORKLOAD_GROUP + "' TO '" + LOCAL_USER + "'@'%'"); + + createPolicy("CREATE ROW POLICY builtin_row_policy ON " + DB + "." + TBL + + " AS RESTRICTIVE TO " + LOCAL_USER + " USING (" + COL_ALLOWED + " = 1)"); + } + + @Test + public void accessDecisionsMatchRecordedBaseline() throws Exception { + List actual = renderSnapshot(); + List expected = readBaseline(); + + if (!actual.equals(expected)) { + Files.createDirectories(ACTUAL_DUMP.getParent()); + Files.write(ACTUAL_DUMP, actual, StandardCharsets.UTF_8); + } + Assertions.assertEquals(expected.size(), actual.size(), + "The access-control behaviour matrix changed shape.\n" + regenerationHint()); + for (int i = 0; i < expected.size(); i++) { + Assertions.assertEquals(expected.get(i), actual.get(i), + "Access-control behaviour changed at line " + (i + 1) + ".\n" + regenerationHint()); + } + } + + private String regenerationHint() { + return "Every differing line is a privilege decision that flipped for some user - it is a behaviour" + + " change, not a test artifact. The current matrix was written to " + ACTUAL_DUMP.toAbsolutePath() + + "; only copy it over src/test/resources" + BASELINE_RESOURCE + " once each changed line is" + + " explained and intended."; + } + + // ------------------------------------------------------------------------------------------------ + // rendering + // ------------------------------------------------------------------------------------------------ + + private List renderSnapshot() { + List lines = new ArrayList<>(); + lines.add("ACTIONS: " + String.join(",", ACTIONS.keySet())); + lines.add("USERS: " + String.join(",", USERS)); + lines.add(""); + AccessControllerManager manager = Env.getCurrentEnv().getAccessManager(); + CatalogAccessController builtin = Deencapsulation.getField(manager, "defaultAccessController"); + + renderWithDefaultController(lines, manager, "builtin", builtin); + renderWithDefaultController(lines, manager, "ranger", + new RangerDorisAccessController(new StubRangerPolicyEngine())); + return lines; + } + + /** + * Renders the whole matrix under one default controller, i.e. under one value of + * {@code fe.conf: access_controller_type}. + */ + private void renderWithDefaultController(List lines, AccessControllerManager manager, + String label, CatalogAccessController defaultController) { + CatalogAccessController original = Deencapsulation.getField(manager, "defaultAccessController"); + Map routes = Deencapsulation.getField(manager, "ctlToCtlAccessController"); + Map savedRoutes = new HashMap<>(routes); + try { + Deencapsulation.setField(manager, "defaultAccessController", defaultController); + // Catalogs that fall back to the default keep a cached reference to the previous one; drop those + // so they resolve against the controller under test. The bound catalog keeps its own plugin. + routes.keySet().removeIf(ctl -> !CTL_RANGER.equals(ctl)); + + renderSystemScope(lines, manager, label); + for (String catalog : CATALOGS) { + renderCatalogScope(lines, manager, label, catalog); + } + } finally { + Deencapsulation.setField(manager, "defaultAccessController", original); + routes.clear(); + routes.putAll(savedRoutes); + } + } + + /** Global / resource / workload group / storage vault / compute group: always the default controller. */ + private void renderSystemScope(List lines, AccessControllerManager manager, String label) { + addRows(lines, label, "-", "global", + (user, action) -> outcome(manager.checkGlobalPriv(user, action))); + addRows(lines, label, "-", "resource:res1", + (user, action) -> outcome(manager.checkResourcePriv(user, "res1", action))); + addRows(lines, label, "-", "workloadgrp:wg1", + (user, action) -> outcome(manager.checkWorkloadGroupPriv(user, "wg1", action))); + addRows(lines, label, "-", "workloadgrp:default", + (user, action) -> outcome( + manager.checkWorkloadGroupPriv(user, WorkloadGroupMgr.DEFAULT_GROUP_NAME, action))); + addRows(lines, label, "-", "vault:sv1", + (user, action) -> outcome(manager.checkStorageVaultPriv(user, "sv1", action))); + // checkCloudPriv is deliberately absent. Its built-in implementation reaches + // Role.checkCloudVirtualComputeGroup, which casts the system info service to the cloud subclass + // unconditionally, so on a non-cloud FE it throws for every caller that does not already hold a + // global privilege. Recording that would freeze a crash, not a decision. + } + + private void renderCatalogScope(List lines, AccessControllerManager manager, String label, + String catalog) { + addRows(lines, label, catalog, "catalog", + (user, action) -> outcome(manager.checkCtlPriv(user, catalog, action))); + addRows(lines, label, catalog, "catalog:skipchk", + (user, action) -> outcome( + withSkipCatalogPrivCheck(() -> manager.checkCtlPriv(user, catalog, action)))); + addRows(lines, label, catalog, "database", + (user, action) -> outcome(manager.checkDbPriv(user, catalog, DB, action))); + addRows(lines, label, catalog, "table", + (user, action) -> outcome(manager.checkTblPriv(user, catalog, DB, TBL, action))); + addRows(lines, label, catalog, "column:" + COL_ALLOWED, + (user, action) -> checkColumn(manager, user, catalog, COL_ALLOWED, action)); + addRows(lines, label, catalog, "column:" + COL_DENIED, + (user, action) -> checkColumn(manager, user, catalog, COL_DENIED, action)); + + for (String user : USERS) { + List filters = manager.evalRowFilterPolicies(identity(user), catalog, DB, TBL); + lines.add(row(label, catalog, user, "rowfilter", filters.isEmpty() ? "-" + : filters.stream() + .map(spec -> "[" + spec.getPolicyIdent() + " | " + spec.getFilterSql() + + " | " + spec.getMergeType() + "]") + .collect(Collectors.joining(",")))); + } + for (String column : ImmutableList.of(COL_ALLOWED, COL_DENIED)) { + for (String user : USERS) { + Optional mask = + manager.evalDataMaskPolicy(identity(user), catalog, DB, TBL, column); + lines.add(row(label, catalog, user, "mask:" + column, + mask.map(spec -> spec.getPolicyIdent() + " | " + spec.getMaskSql()).orElse("-"))); + } + } + } + + private void addRows(List lines, String label, String scope, String probe, Probe decision) { + for (String user : USERS) { + lines.add(row(label, scope, user, probe, verdict(identity(user), decision))); + } + } + + /** + * {@code }, plus {@code / unsupported=} when the check refused to answer at all. + * The third outcome is not decoration: a column check only accepts privileges that carry SELECT or LOAD, + * so whether a caller gets a verdict or an IllegalStateException today depends on whether the manager's + * global short circuit answered before the built-in column check was ever reached. + */ + private String verdict(UserIdentity user, Probe probe) { + List allowed = new ArrayList<>(); + List unsupported = new ArrayList<>(); + for (Map.Entry action : ACTIONS.entrySet()) { + switch (probe.evaluate(user, action.getValue())) { + case ALLOW: + allowed.add(action.getKey()); + break; + case UNSUPPORTED: + unsupported.add(action.getKey()); + break; + default: + break; + } + } + String verdict = allowed.isEmpty() ? "-" : String.join(",", allowed); + return unsupported.isEmpty() ? verdict : verdict + " / unsupported=" + String.join(",", unsupported); + } + + private static String row(String label, String scope, String user, String probe, String verdict) { + return String.format("%-7s | %-10s | %-11s | %-17s | %s", label, scope, user, probe, verdict); + } + + private enum Outcome { ALLOW, DENY, UNSUPPORTED } + + private interface Probe { + Outcome evaluate(UserIdentity user, PrivPredicate action); + } + + private static Outcome outcome(boolean allowed) { + return allowed ? Outcome.ALLOW : Outcome.DENY; + } + + private Outcome checkColumn(AccessControllerManager manager, UserIdentity user, String catalog, String column, + PrivPredicate action) { + try { + manager.checkColumnsPriv(user, catalog, DB, TBL, ImmutableSet.of(column), action); + return Outcome.ALLOW; + } catch (UserException e) { + return Outcome.DENY; + } catch (IllegalStateException e) { + // Auth.checkColPriv rejects any predicate that carries neither SELECT nor LOAD. + return Outcome.UNSUPPORTED; + } + } + + private boolean withSkipCatalogPrivCheck(BooleanSupplier body) { + boolean original = Config.skip_catalog_priv_check; + Config.skip_catalog_priv_check = true; + try { + return body.getAsBoolean(); + } finally { + Config.skip_catalog_priv_check = original; + } + } + + private static UserIdentity identity(String user) { + return UserIdentity.createAnalyzedUserIdentWithIp(user, "%"); + } + + private static Map allPrivPredicates() { + Map sorted = new TreeMap<>(); + for (Field field : PrivPredicate.class.getDeclaredFields()) { + if (Modifier.isStatic(field.getModifiers()) && field.getType() == PrivPredicate.class) { + try { + sorted.put(field.getName(), (PrivPredicate) field.get(null)); + } catch (IllegalAccessException e) { + throw new IllegalStateException("cannot read PrivPredicate." + field.getName(), e); + } + } + } + return new LinkedHashMap<>(sorted); + } + + /** {@code GRANT ... ON RESOURCE/WORKLOAD GROUP}, which the shared helper does not run. */ + private void grantResourcePriv(String sql) throws Exception { + LogicalPlan parsed = new NereidsParser().parseSingle(sql); + ((GrantResourcePrivilegeCommand) parsed).run(connectContext, new StmtExecutor(connectContext, sql)); + } + + private List readBaseline() throws IOException { + List lines = new ArrayList<>(); + try (InputStream in = getClass().getResourceAsStream(BASELINE_RESOURCE)) { + Assertions.assertNotNull(in, "missing baseline resource " + BASELINE_RESOURCE); + BufferedReader reader = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8)); + String line; + while ((line = reader.readLine()) != null) { + lines.add(line); + } + } + return lines; + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java new file mode 100644 index 00000000000000..20ad286b412d93 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java @@ -0,0 +1,38 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; + +import java.util.Map; + +/** + * Binds a catalog to the production Ranger controller driven by {@link StubRangerPolicyEngine}, so a test can + * create a Ranger-governed catalog with {@code "access_controller.class"} without a Ranger server. + */ +public class StubRangerAccessControllerFactory implements AccessControllerFactory { + @Override + public String factoryIdentifier() { + return "stub-ranger-doris"; + } + + @Override + public CatalogAccessController createAccessController(Map prop) { + return new RangerDorisAccessController(new StubRangerPolicyEngine()); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java new file mode 100644 index 00000000000000..52549ee09be31e --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java @@ -0,0 +1,172 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisResource; + +import com.google.common.collect.Lists; +import org.apache.ranger.plugin.policyengine.RangerAccessRequest; +import org.apache.ranger.plugin.policyengine.RangerAccessRequest.ResourceMatchingScope; +import org.apache.ranger.plugin.policyengine.RangerAccessResource; +import org.apache.ranger.plugin.policyengine.RangerAccessResult; +import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor; +import org.apache.ranger.plugin.service.RangerBasePlugin; + +import java.util.Collection; +import java.util.List; + +/** + * A Ranger policy engine whose answers are a pure function of (user, resource, access type), so that a + * behaviour baseline recorded against it stays byte-identical across runs and machines. + * + *

Only the policy engine is faked. The controller above it is the production + * {@link org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController}, so its own + * global-then-catalog-then-database-then-table cascade, its SHOW-as-subtree probe and its privilege-bit + * decomposition are all exercised for real. + * + *

The whole policy set, and nothing else, is: + *

    + *
  • {@code ranger_user} may SELECT {@code .pdb.ptbl} and its column {@code pcol1};
    • + *
  • {@code ranger_user} may USAGE the resource {@code res1}, workload group {@code wg1}, + * storage vault {@code sv1} and compute group {@code cg1};
    • + *
  • a subtree probe (the request Ranger sends for SHOW: no access type, SELF_OR_DESCENDANTS) succeeds for + * {@code ranger_user} on any ancestor of {@code pdb.ptbl};
    • + *
  • {@code ranger_user} reading {@code pdb.ptbl} gets the row filter {@code pcol1 = 1} and a MASK_NULL + * mask on {@code pcol1};
    • + *
  • every other user - including the ones holding built-in ADMIN_PRIV and NODE_PRIV - is denied + * everywhere, and gets no filter and no mask.
    • + *
+ * + *

Denying the built-in admin is the point: wherever the baseline still records ALLOW for that user on a + * Ranger-governed resource, the decision provably came from the engine's cross-plugin OR rather than from + * Ranger, which is exactly the behaviour the routing rework has to reproduce. + */ +public class StubRangerPolicyEngine extends RangerBasePlugin { + public static final String ALLOWED_USER = "ranger_user"; + public static final String ALLOWED_DB = "pdb"; + public static final String ALLOWED_TABLE = "ptbl"; + public static final String ALLOWED_COLUMN = "pcol1"; + public static final String ALLOWED_RESOURCE = "res1"; + public static final String ALLOWED_WORKLOAD_GROUP = "wg1"; + public static final String ALLOWED_STORAGE_VAULT = "sv1"; + public static final String ALLOWED_COMPUTE_GROUP = "cg1"; + + public static final String ROW_FILTER_EXPR = "pcol1 = 1"; + public static final long ROW_FILTER_POLICY_ID = 101L; + public static final long ROW_FILTER_POLICY_VERSION = 7L; + public static final long DATA_MASK_POLICY_ID = 202L; + public static final long DATA_MASK_POLICY_VERSION = 9L; + + private static final String SELECT_ACCESS_TYPE = "SELECT"; + private static final String USAGE_ACCESS_TYPE = "USAGE"; + + public StubRangerPolicyEngine() { + super("stub-ranger-doris", null, null); + } + + @Override + public RangerAccessResult isAccessAllowed(RangerAccessRequest request) { + RangerAccessResult result = new RangerAccessResult(1, "stub", null, request); + result.setPolicyVersion(1L); + result.setIsAllowed(isAllowed(request)); + return result; + } + + @Override + public Collection isAccessAllowed(Collection requests) { + List results = Lists.newArrayList(); + for (RangerAccessRequest request : requests) { + results.add(isAccessAllowed(request)); + } + return results; + } + + @Override + public RangerAccessResult evalRowFilterPolicies(RangerAccessRequest request, + RangerAccessResultProcessor resultProcessor) { + RangerAccessResult result = new RangerAccessResult(3, "stub", null, request); + result.setPolicyId(ROW_FILTER_POLICY_ID); + result.setPolicyVersion(ROW_FILTER_POLICY_VERSION); + RangerAccessResource resource = request.getResource(); + if (ALLOWED_USER.equals(request.getUser()) + && ALLOWED_DB.equals(value(resource, RangerDorisResource.KEY_DATABASE)) + && ALLOWED_TABLE.equals(value(resource, RangerDorisResource.KEY_TABLE))) { + result.setFilterExpr(ROW_FILTER_EXPR); + } + return result; + } + + @Override + public RangerAccessResult evalDataMaskPolicies(RangerAccessRequest request, + RangerAccessResultProcessor resultProcessor) { + RangerAccessResult result = new RangerAccessResult(2, "stub", null, request); + result.setPolicyId(DATA_MASK_POLICY_ID); + result.setPolicyVersion(DATA_MASK_POLICY_VERSION); + RangerAccessResource resource = request.getResource(); + if (ALLOWED_USER.equals(request.getUser()) + && ALLOWED_DB.equals(value(resource, RangerDorisResource.KEY_DATABASE)) + && ALLOWED_TABLE.equals(value(resource, RangerDorisResource.KEY_TABLE)) + && ALLOWED_COLUMN.equals(value(resource, RangerDorisResource.KEY_COLUMN))) { + result.setMaskType("MASK_NULL"); + } + return result; + } + + private boolean isAllowed(RangerAccessRequest request) { + if (!ALLOWED_USER.equals(request.getUser())) { + return false; + } + RangerAccessResource resource = request.getResource(); + String db = value(resource, RangerDorisResource.KEY_DATABASE); + String tbl = value(resource, RangerDorisResource.KEY_TABLE); + String col = value(resource, RangerDorisResource.KEY_COLUMN); + String accessType = request.getAccessType(); + + if (ResourceMatchingScope.SELF_OR_DESCENDANTS == request.getResourceMatchingScope()) { + // The subtree probe Ranger sends for SHOW: "does this user hold anything at or below here?". + // It carries no access type of its own, so it must be recognised by its matching scope. + return isPrefixOfAllowedTable(resource, db, tbl); + } + if (SELECT_ACCESS_TYPE.equals(accessType)) { + return ALLOWED_DB.equals(db) && ALLOWED_TABLE.equals(tbl) + && (col == null || ALLOWED_COLUMN.equals(col)); + } + if (USAGE_ACCESS_TYPE.equals(accessType)) { + return ALLOWED_RESOURCE.equals(value(resource, RangerDorisResource.KEY_RESOURCE)) + || ALLOWED_WORKLOAD_GROUP.equals(value(resource, RangerDorisResource.KEY_WORKLOAD_GROUP)) + || ALLOWED_STORAGE_VAULT.equals(value(resource, RangerDorisResource.KEY_STORAGE_VAULT)) + || ALLOWED_COMPUTE_GROUP.equals(value(resource, RangerDorisResource.KEY_COMPUTE_GROUP)); + } + return false; + } + + private boolean isPrefixOfAllowedTable(RangerAccessResource resource, String db, String tbl) { + if (value(resource, RangerDorisResource.KEY_CATALOG) == null) { + // A global / resource / workload group probe is never an ancestor of the one allowed table. + return false; + } + return (db == null || ALLOWED_DB.equals(db)) + && (tbl == null || ALLOWED_TABLE.equals(tbl)) + && value(resource, RangerDorisResource.KEY_COLUMN) == null; + } + + private static String value(RangerAccessResource resource, String key) { + Object value = resource.getValue(key); + return value == null ? null : value.toString(); + } +} diff --git a/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory index 83924e7e0f6073..776fc40c3f77a0 100644 --- a/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory +++ b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory @@ -15,4 +15,5 @@ # limitations under the License. # # -org.apache.doris.nereids.privileges.CustomAccessControllerFactory \ No newline at end of file +org.apache.doris.nereids.privileges.CustomAccessControllerFactory +org.apache.doris.mysql.privilege.StubRangerAccessControllerFactory \ No newline at end of file diff --git a/fe/fe-core/src/test/resources/access-control-behavior-baseline.txt b/fe/fe-core/src/test/resources/access-control-behavior-baseline.txt new file mode 100644 index 00000000000000..c36390cc9a02ab --- /dev/null +++ b/fe/fe-core/src/test/resources/access-control-behavior-baseline.txt @@ -0,0 +1,323 @@ +ACTIONS: ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +USERS: admin_user,node_user,local_user,ranger_user,nobody + +builtin | - | admin_user | global | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | node_user | global | ADMIN_OR_NODE,ALL,OPERATOR +builtin | - | local_user | global | - +builtin | - | ranger_user | global | - +builtin | - | nobody | global | - +builtin | - | admin_user | resource:res1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | node_user | resource:res1 | ADMIN_OR_NODE,ALL,OPERATOR +builtin | - | local_user | resource:res1 | ALL,SHOW_RESOURCES,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | ranger_user | resource:res1 | - +builtin | - | nobody | resource:res1 | - +builtin | - | admin_user | workloadgrp:wg1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | node_user | workloadgrp:wg1 | ADMIN_OR_NODE,ALL,OPERATOR +builtin | - | local_user | workloadgrp:wg1 | ALL,SHOW_RESOURCES,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | ranger_user | workloadgrp:wg1 | - +builtin | - | nobody | workloadgrp:wg1 | - +builtin | - | admin_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | node_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | local_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | ranger_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | nobody | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | admin_user | vault:sv1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | - | node_user | vault:sv1 | ADMIN_OR_NODE,ALL,OPERATOR +builtin | - | local_user | vault:sv1 | - +builtin | - | ranger_user | vault:sv1 | - +builtin | - | nobody | vault:sv1 | - +builtin | internal | admin_user | catalog | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | node_user | catalog | ADMIN_OR_NODE,ALL,OPERATOR,SHOW +builtin | internal | local_user | catalog | SHOW +builtin | internal | ranger_user | catalog | SHOW +builtin | internal | nobody | catalog | SHOW +builtin | internal | admin_user | catalog:skipchk | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | node_user | catalog:skipchk | ADMIN_OR_NODE,ALL,OPERATOR,SHOW +builtin | internal | local_user | catalog:skipchk | SHOW +builtin | internal | ranger_user | catalog:skipchk | SHOW +builtin | internal | nobody | catalog:skipchk | SHOW +builtin | internal | admin_user | database | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | node_user | database | ADMIN_OR_NODE,ALL,OPERATOR +builtin | internal | local_user | database | SHOW +builtin | internal | ranger_user | database | - +builtin | internal | nobody | database | - +builtin | internal | admin_user | table | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | node_user | table | ADMIN_OR_NODE,ALL,OPERATOR +builtin | internal | local_user | table | SELECT,SHOW +builtin | internal | ranger_user | table | - +builtin | internal | nobody | table | - +builtin | internal | admin_user | column:pcol1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE / unsupported=OPERATOR +builtin | internal | node_user | column:pcol1 | ADMIN_OR_NODE,ALL,OPERATOR / unsupported=ADMIN,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | local_user | column:pcol1 | ALL,SELECT,SHOW / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | ranger_user | column:pcol1 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | nobody | column:pcol1 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | admin_user | column:pcol2 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE / unsupported=OPERATOR +builtin | internal | node_user | column:pcol2 | ADMIN_OR_NODE,ALL,OPERATOR / unsupported=ADMIN,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | local_user | column:pcol2 | ALL,SELECT,SHOW / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | ranger_user | column:pcol2 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | nobody | column:pcol2 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | internal | admin_user | rowfilter | - +builtin | internal | node_user | rowfilter | - +builtin | internal | local_user | rowfilter | [builtin_row_policy | pcol1 = 1 | RESTRICTIVE] +builtin | internal | ranger_user | rowfilter | - +builtin | internal | nobody | rowfilter | - +builtin | internal | admin_user | mask:pcol1 | - +builtin | internal | node_user | mask:pcol1 | - +builtin | internal | local_user | mask:pcol1 | - +builtin | internal | ranger_user | mask:pcol1 | - +builtin | internal | nobody | mask:pcol1 | - +builtin | internal | admin_user | mask:pcol2 | - +builtin | internal | node_user | mask:pcol2 | - +builtin | internal | local_user | mask:pcol2 | - +builtin | internal | ranger_user | mask:pcol2 | - +builtin | internal | nobody | mask:pcol2 | - +builtin | ext_plain | admin_user | catalog | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | node_user | catalog | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_plain | local_user | catalog | SHOW +builtin | ext_plain | ranger_user | catalog | - +builtin | ext_plain | nobody | catalog | - +builtin | ext_plain | admin_user | catalog:skipchk | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | node_user | catalog:skipchk | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_plain | local_user | catalog:skipchk | SHOW +builtin | ext_plain | ranger_user | catalog:skipchk | - +builtin | ext_plain | nobody | catalog:skipchk | - +builtin | ext_plain | admin_user | database | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | node_user | database | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_plain | local_user | database | SHOW +builtin | ext_plain | ranger_user | database | - +builtin | ext_plain | nobody | database | - +builtin | ext_plain | admin_user | table | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | node_user | table | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_plain | local_user | table | SELECT,SHOW +builtin | ext_plain | ranger_user | table | - +builtin | ext_plain | nobody | table | - +builtin | ext_plain | admin_user | column:pcol1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE / unsupported=OPERATOR +builtin | ext_plain | node_user | column:pcol1 | ADMIN_OR_NODE,ALL,OPERATOR / unsupported=ADMIN,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | local_user | column:pcol1 | ALL,SELECT,SHOW / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | ranger_user | column:pcol1 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | nobody | column:pcol1 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | admin_user | column:pcol2 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE / unsupported=OPERATOR +builtin | ext_plain | node_user | column:pcol2 | ADMIN_OR_NODE,ALL,OPERATOR / unsupported=ADMIN,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | local_user | column:pcol2 | ALL,SELECT,SHOW / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | ranger_user | column:pcol2 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | nobody | column:pcol2 | - / unsupported=ADMIN,ADMIN_OR_NODE,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,OPERATOR,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_plain | admin_user | rowfilter | - +builtin | ext_plain | node_user | rowfilter | - +builtin | ext_plain | local_user | rowfilter | - +builtin | ext_plain | ranger_user | rowfilter | - +builtin | ext_plain | nobody | rowfilter | - +builtin | ext_plain | admin_user | mask:pcol1 | - +builtin | ext_plain | node_user | mask:pcol1 | - +builtin | ext_plain | local_user | mask:pcol1 | - +builtin | ext_plain | ranger_user | mask:pcol1 | - +builtin | ext_plain | nobody | mask:pcol1 | - +builtin | ext_plain | admin_user | mask:pcol2 | - +builtin | ext_plain | node_user | mask:pcol2 | - +builtin | ext_plain | local_user | mask:pcol2 | - +builtin | ext_plain | ranger_user | mask:pcol2 | - +builtin | ext_plain | nobody | mask:pcol2 | - +builtin | ext_ranger | admin_user | catalog | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | catalog | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_ranger | local_user | catalog | SHOW +builtin | ext_ranger | ranger_user | catalog | - +builtin | ext_ranger | nobody | catalog | - +builtin | ext_ranger | admin_user | catalog:skipchk | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | catalog:skipchk | ADMIN_OR_NODE,ALL,OPERATOR,SELECT,SHOW +builtin | ext_ranger | local_user | catalog:skipchk | SELECT,SHOW +builtin | ext_ranger | ranger_user | catalog:skipchk | SELECT,SHOW +builtin | ext_ranger | nobody | catalog:skipchk | SELECT,SHOW +builtin | ext_ranger | admin_user | database | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | database | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_ranger | local_user | database | - +builtin | ext_ranger | ranger_user | database | SHOW +builtin | ext_ranger | nobody | database | - +builtin | ext_ranger | admin_user | table | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | table | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_ranger | local_user | table | - +builtin | ext_ranger | ranger_user | table | ALL,SELECT,SHOW +builtin | ext_ranger | nobody | table | - +builtin | ext_ranger | admin_user | column:pcol1 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | column:pcol1 | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_ranger | local_user | column:pcol1 | - +builtin | ext_ranger | ranger_user | column:pcol1 | ALL,SELECT,SHOW +builtin | ext_ranger | nobody | column:pcol1 | - +builtin | ext_ranger | admin_user | column:pcol2 | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +builtin | ext_ranger | node_user | column:pcol2 | ADMIN_OR_NODE,ALL,OPERATOR +builtin | ext_ranger | local_user | column:pcol2 | - +builtin | ext_ranger | ranger_user | column:pcol2 | ALL,SELECT,SHOW +builtin | ext_ranger | nobody | column:pcol2 | - +builtin | ext_ranger | admin_user | rowfilter | - +builtin | ext_ranger | node_user | rowfilter | - +builtin | ext_ranger | local_user | rowfilter | - +builtin | ext_ranger | ranger_user | rowfilter | [101:7 | pcol1 = 1 | RESTRICTIVE] +builtin | ext_ranger | nobody | rowfilter | - +builtin | ext_ranger | admin_user | mask:pcol1 | - +builtin | ext_ranger | node_user | mask:pcol1 | - +builtin | ext_ranger | local_user | mask:pcol1 | - +builtin | ext_ranger | ranger_user | mask:pcol1 | 202:9 | NULL +builtin | ext_ranger | nobody | mask:pcol1 | - +builtin | ext_ranger | admin_user | mask:pcol2 | - +builtin | ext_ranger | node_user | mask:pcol2 | - +builtin | ext_ranger | local_user | mask:pcol2 | - +builtin | ext_ranger | ranger_user | mask:pcol2 | - +builtin | ext_ranger | nobody | mask:pcol2 | - +ranger | - | admin_user | global | - +ranger | - | node_user | global | - +ranger | - | local_user | global | - +ranger | - | ranger_user | global | - +ranger | - | nobody | global | - +ranger | - | admin_user | resource:res1 | - +ranger | - | node_user | resource:res1 | - +ranger | - | local_user | resource:res1 | - +ranger | - | ranger_user | resource:res1 | ALL,SHOW_RESOURCES,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | nobody | resource:res1 | - +ranger | - | admin_user | workloadgrp:wg1 | - +ranger | - | node_user | workloadgrp:wg1 | - +ranger | - | local_user | workloadgrp:wg1 | - +ranger | - | ranger_user | workloadgrp:wg1 | ALL,SHOW_RESOURCES,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | nobody | workloadgrp:wg1 | - +ranger | - | admin_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | node_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | local_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | ranger_user | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | nobody | workloadgrp:default | ADMIN,ADMIN_OR_NODE,ALL,ALTER,ALTER_CREATE,ALTER_CREATE_DROP,CREATE,DROP,GRANT,LOAD,OPERATOR,SELECT,SHOW,SHOW_RESOURCES,SHOW_VIEW,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | admin_user | vault:sv1 | - +ranger | - | node_user | vault:sv1 | - +ranger | - | local_user | vault:sv1 | - +ranger | - | ranger_user | vault:sv1 | ALL,SHOW_RESOURCES,SHOW_WORKLOAD_GROUP,USAGE +ranger | - | nobody | vault:sv1 | - +ranger | internal | admin_user | catalog | - +ranger | internal | node_user | catalog | - +ranger | internal | local_user | catalog | - +ranger | internal | ranger_user | catalog | SHOW +ranger | internal | nobody | catalog | - +ranger | internal | admin_user | catalog:skipchk | - +ranger | internal | node_user | catalog:skipchk | - +ranger | internal | local_user | catalog:skipchk | - +ranger | internal | ranger_user | catalog:skipchk | SHOW +ranger | internal | nobody | catalog:skipchk | - +ranger | internal | admin_user | database | - +ranger | internal | node_user | database | - +ranger | internal | local_user | database | - +ranger | internal | ranger_user | database | SHOW +ranger | internal | nobody | database | - +ranger | internal | admin_user | table | - +ranger | internal | node_user | table | - +ranger | internal | local_user | table | - +ranger | internal | ranger_user | table | ALL,SELECT,SHOW +ranger | internal | nobody | table | - +ranger | internal | admin_user | column:pcol1 | - +ranger | internal | node_user | column:pcol1 | - +ranger | internal | local_user | column:pcol1 | - +ranger | internal | ranger_user | column:pcol1 | ALL,SELECT,SHOW +ranger | internal | nobody | column:pcol1 | - +ranger | internal | admin_user | column:pcol2 | - +ranger | internal | node_user | column:pcol2 | - +ranger | internal | local_user | column:pcol2 | - +ranger | internal | ranger_user | column:pcol2 | ALL,SELECT,SHOW +ranger | internal | nobody | column:pcol2 | - +ranger | internal | admin_user | rowfilter | - +ranger | internal | node_user | rowfilter | - +ranger | internal | local_user | rowfilter | - +ranger | internal | ranger_user | rowfilter | [101:7 | pcol1 = 1 | RESTRICTIVE] +ranger | internal | nobody | rowfilter | - +ranger | internal | admin_user | mask:pcol1 | - +ranger | internal | node_user | mask:pcol1 | - +ranger | internal | local_user | mask:pcol1 | - +ranger | internal | ranger_user | mask:pcol1 | 202:9 | NULL +ranger | internal | nobody | mask:pcol1 | - +ranger | internal | admin_user | mask:pcol2 | - +ranger | internal | node_user | mask:pcol2 | - +ranger | internal | local_user | mask:pcol2 | - +ranger | internal | ranger_user | mask:pcol2 | - +ranger | internal | nobody | mask:pcol2 | - +ranger | ext_plain | admin_user | catalog | - +ranger | ext_plain | node_user | catalog | - +ranger | ext_plain | local_user | catalog | - +ranger | ext_plain | ranger_user | catalog | SHOW +ranger | ext_plain | nobody | catalog | - +ranger | ext_plain | admin_user | catalog:skipchk | - +ranger | ext_plain | node_user | catalog:skipchk | - +ranger | ext_plain | local_user | catalog:skipchk | - +ranger | ext_plain | ranger_user | catalog:skipchk | SHOW +ranger | ext_plain | nobody | catalog:skipchk | - +ranger | ext_plain | admin_user | database | - +ranger | ext_plain | node_user | database | - +ranger | ext_plain | local_user | database | - +ranger | ext_plain | ranger_user | database | SHOW +ranger | ext_plain | nobody | database | - +ranger | ext_plain | admin_user | table | - +ranger | ext_plain | node_user | table | - +ranger | ext_plain | local_user | table | - +ranger | ext_plain | ranger_user | table | ALL,SELECT,SHOW +ranger | ext_plain | nobody | table | - +ranger | ext_plain | admin_user | column:pcol1 | - +ranger | ext_plain | node_user | column:pcol1 | - +ranger | ext_plain | local_user | column:pcol1 | - +ranger | ext_plain | ranger_user | column:pcol1 | ALL,SELECT,SHOW +ranger | ext_plain | nobody | column:pcol1 | - +ranger | ext_plain | admin_user | column:pcol2 | - +ranger | ext_plain | node_user | column:pcol2 | - +ranger | ext_plain | local_user | column:pcol2 | - +ranger | ext_plain | ranger_user | column:pcol2 | ALL,SELECT,SHOW +ranger | ext_plain | nobody | column:pcol2 | - +ranger | ext_plain | admin_user | rowfilter | - +ranger | ext_plain | node_user | rowfilter | - +ranger | ext_plain | local_user | rowfilter | - +ranger | ext_plain | ranger_user | rowfilter | [101:7 | pcol1 = 1 | RESTRICTIVE] +ranger | ext_plain | nobody | rowfilter | - +ranger | ext_plain | admin_user | mask:pcol1 | - +ranger | ext_plain | node_user | mask:pcol1 | - +ranger | ext_plain | local_user | mask:pcol1 | - +ranger | ext_plain | ranger_user | mask:pcol1 | 202:9 | NULL +ranger | ext_plain | nobody | mask:pcol1 | - +ranger | ext_plain | admin_user | mask:pcol2 | - +ranger | ext_plain | node_user | mask:pcol2 | - +ranger | ext_plain | local_user | mask:pcol2 | - +ranger | ext_plain | ranger_user | mask:pcol2 | - +ranger | ext_plain | nobody | mask:pcol2 | - +ranger | ext_ranger | admin_user | catalog | - +ranger | ext_ranger | node_user | catalog | - +ranger | ext_ranger | local_user | catalog | - +ranger | ext_ranger | ranger_user | catalog | SHOW +ranger | ext_ranger | nobody | catalog | - +ranger | ext_ranger | admin_user | catalog:skipchk | SELECT,SHOW +ranger | ext_ranger | node_user | catalog:skipchk | SELECT,SHOW +ranger | ext_ranger | local_user | catalog:skipchk | SELECT,SHOW +ranger | ext_ranger | ranger_user | catalog:skipchk | SELECT,SHOW +ranger | ext_ranger | nobody | catalog:skipchk | SELECT,SHOW +ranger | ext_ranger | admin_user | database | - +ranger | ext_ranger | node_user | database | - +ranger | ext_ranger | local_user | database | - +ranger | ext_ranger | ranger_user | database | SHOW +ranger | ext_ranger | nobody | database | - +ranger | ext_ranger | admin_user | table | - +ranger | ext_ranger | node_user | table | - +ranger | ext_ranger | local_user | table | - +ranger | ext_ranger | ranger_user | table | ALL,SELECT,SHOW +ranger | ext_ranger | nobody | table | - +ranger | ext_ranger | admin_user | column:pcol1 | - +ranger | ext_ranger | node_user | column:pcol1 | - +ranger | ext_ranger | local_user | column:pcol1 | - +ranger | ext_ranger | ranger_user | column:pcol1 | ALL,SELECT,SHOW +ranger | ext_ranger | nobody | column:pcol1 | - +ranger | ext_ranger | admin_user | column:pcol2 | - +ranger | ext_ranger | node_user | column:pcol2 | - +ranger | ext_ranger | local_user | column:pcol2 | - +ranger | ext_ranger | ranger_user | column:pcol2 | ALL,SELECT,SHOW +ranger | ext_ranger | nobody | column:pcol2 | - +ranger | ext_ranger | admin_user | rowfilter | - +ranger | ext_ranger | node_user | rowfilter | - +ranger | ext_ranger | local_user | rowfilter | - +ranger | ext_ranger | ranger_user | rowfilter | [101:7 | pcol1 = 1 | RESTRICTIVE] +ranger | ext_ranger | nobody | rowfilter | - +ranger | ext_ranger | admin_user | mask:pcol1 | - +ranger | ext_ranger | node_user | mask:pcol1 | - +ranger | ext_ranger | local_user | mask:pcol1 | - +ranger | ext_ranger | ranger_user | mask:pcol1 | 202:9 | NULL +ranger | ext_ranger | nobody | mask:pcol1 | - +ranger | ext_ranger | admin_user | mask:pcol2 | - +ranger | ext_ranger | node_user | mask:pcol2 | - +ranger | ext_ranger | local_user | mask:pcol2 | - +ranger | ext_ranger | ranger_user | mask:pcol2 | - +ranger | ext_ranger | nobody | mask:pcol2 | - From ab342b9c3d29588c0daa73908397d0469e421a08 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 13:06:49 +0800 Subject: [PATCH 05/22] [improvement](authorization) let each controller decide its own global exemption The manager used to establish a global verdict on every controller's behalf and hand it down as the hasGlobal argument, which the interface turned into "if (hasGlobal) return true" for catalog, database, table and column checks. Two things were tangled in that one line. For the built-in controller it was its own privilege model talking to itself. For a catalog bound to Ranger it was something else entirely: a second controller's verdict granting access inside a range Ranger governs, decided by the engine, invisible from the plugin. Which policies apply to a resource was therefore not readable from which controller the catalog is bound to. The argument is gone and the manager only routes now. Each controller says for itself what a global privilege buys: - the built-in one checks global privileges ahead of the fine grained ones. Not redundant with the per-role checks underneath: Auth refuses NODE privileges below global level, so a caller holding only global NODE_PRIV is granted by that line and by nothing else. - the Ranger controllers ask whichever controller owns global scope - the one access_controller_type installs - and honour its grant. That is the same question the engine used to ask, so the answer is the same in every deployment: with the built-in controller installed globally an administrator still reaches a Ranger-governed catalog, and with Ranger installed globally Ranger keeps deciding its own exemptions. Asking the built-in model directly instead would have handed built-in ADMIN_PRIV a way into a Ranger-only deployment, which is a policy change and not this commit's business. A controller that is itself the global-scope authority skips the question: its own global check answers it one line later, and asking through the manager would evaluate the same Ranger policies twice per database, table and column check. The behaviour baseline is unchanged, byte for byte. The interface no longer hands a third-party controller the exemption for free, which is the point - it is a policy decision, and a plugin that refuses it is a legal plugin. The tests that covered the deleted default methods move to the built-in implementation, where that policy now lives. Co-Authored-By: Claude Opus 5 (1M context) --- .../ranger/RangerAccessController.java | 22 ++ .../doris/RangerDorisAccessController.java | 9 + .../hive/RangerHiveAccessController.java | 9 + .../privilege/AccessControllerManager.java | 22 +- .../privilege/CatalogAccessController.java | 43 +-- .../privilege/InternalAccessController.java | 17 +- .../RangerGlobalScopeDeferenceTest.java | 126 +++++++++ .../AccessControlBehaviorBaselineTest.java | 20 +- .../AccessControllerManagerTest.java | 12 +- .../CatalogAccessControllerTest.java | 260 ------------------ .../InternalAccessControllerTest.java | 139 ++++++++++ .../privilege/StubRangerPolicyEngine.java | 7 +- 12 files changed, 358 insertions(+), 328 deletions(-) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java delete mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java index f26ea775081e10..10bb4a1137c657 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java @@ -20,9 +20,12 @@ import org.apache.doris.analysis.UserIdentity; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.catalog.Env; import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; import org.apache.doris.common.AuthorizationException; +import org.apache.doris.datasource.InternalCatalog; import org.apache.doris.mysql.privilege.CatalogAccessController; +import org.apache.doris.mysql.privilege.PrivPredicate; import com.google.common.collect.Lists; import org.apache.commons.lang3.StringUtils; @@ -43,6 +46,25 @@ public abstract class RangerAccessController implements CatalogAccessController protected static final String CLIENT_TYPE_DORIS = "doris"; + /** + * Whether the privilege is already held at global scope, which the Ranger plugins honour as a grant on + * everything they govern. + * + *

Global scope is not a Ranger catalog: it belongs to whichever controller {@code access_controller_type} + * installs, so that is who gets asked. With the built-in controller there this reproduces "an administrator + * of the cluster can reach a Ranger-governed catalog"; with Ranger installed globally, Ranger decides its + * own exemptions and the built-in grants stay out of it. Deciding this here rather than in the engine is + * what lets a third-party controller refuse the exemption outright. + * + *

Returns false without asking when this controller is itself the global-scope authority: the caller's + * own global check answers the same question one line later. + */ + protected boolean grantedByGlobalScopeAuthority(UserIdentity currentUser, PrivPredicate wanted) { + CatalogAccessController authority = Env.getCurrentEnv().getAccessManager() + .getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME); + return authority != this && authority.checkGlobalPriv(currentUser, wanted); + } + protected static boolean checkRequestResult(RangerAccessRequestImpl request, RangerAccessResult result, String name) { if (result == null) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java index 02420d2341cc3c..d02c9c7f2eb1a9 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java @@ -160,6 +160,9 @@ private boolean checkAnyPrivWithinCtl(UserIdentity currentUser, String ctl) { @Override public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return true; + } PrivBitSet checkedPrivs = PrivBitSet.of(); if (checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) @@ -187,6 +190,9 @@ private boolean checkAnyPrivWithinDb(UserIdentity currentUser, String ctl, Strin @Override public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return true; + } PrivBitSet checkedPrivs = PrivBitSet.of(); if (checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) @@ -216,6 +222,9 @@ private boolean checkAnyPrivWithinTbl(UserIdentity currentUser, String ctl, Stri @Override public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, PrivPredicate wanted) throws AuthorizationException { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return; + } PrivBitSet checkedPrivs = PrivBitSet.of(); boolean hasTablePriv = checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java index 11cbdf20f1b943..3f2973ba678ad4 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java @@ -210,6 +210,9 @@ public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate @Override public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return true; + } RangerHiveResource resource = new RangerHiveResource(HiveObjectType.DATABASE, db); return checkPrivilege(currentUser, convertToAccessType(wanted), resource); @@ -217,6 +220,9 @@ public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, Priv @Override public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return true; + } RangerHiveResource resource = new RangerHiveResource(HiveObjectType.TABLE, db, tbl); return checkPrivilege(currentUser, convertToAccessType(wanted), resource); @@ -225,6 +231,9 @@ public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, Str @Override public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, PrivPredicate wanted) throws AuthorizationException { + if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + return; + } List resources = new ArrayList<>(); for (String col : cols) { RangerHiveResource resource = new RangerHiveResource(HiveObjectType.COLUMN, diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index 07ff5b46854393..347b91cf314be5 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -56,6 +56,11 @@ * SystemAccessController: for global level priv, resource priv and other Doris internal priv checking * CatalogAccessController: for specified catalog's priv checking, can be customized. * And using InternalCatalogAccessController as default. + * + *

It routes and nothing more: each check goes to the single controller that governs the resource, and that + * controller's answer is the answer. The manager establishes no privilege of its own beforehand and never + * combines two controllers' verdicts, so which policies apply to a resource is readable from which controller + * the catalog is bound to. */ public class AccessControllerManager { private static final Logger LOG = LogManager.getLogger(AccessControllerManager.class); @@ -320,14 +325,13 @@ private boolean shouldSkipCatalogPrivCheck(PrivPredicate wanted) { } public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - boolean hasGlobal = checkGlobalPriv(currentUser, wanted); if (shouldSkipCatalogPrivCheck(wanted)) { CatalogIf catalog = Env.getCurrentEnv().getCatalogMgr().getCatalog(ctl); if (catalog == null) { return false; } if (catalog.isInternalCatalog()) { - return defaultAccessController.checkCtlPriv(hasGlobal, currentUser, ctl, wanted); + return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); } // If catalog not set access controller, use internal access controller // otherwise, skip catalog priv check @@ -335,13 +339,13 @@ public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate ""); if (Strings.isNullOrEmpty(className)) { // not set access controller, use internal access controller - return defaultAccessController.checkCtlPriv(hasGlobal, currentUser, ctl, wanted); + return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); } return true; } // for checking catalog priv, always use InternalAccessController. // because catalog priv is only saved in InternalAccessController. - return defaultAccessController.checkCtlPriv(hasGlobal, currentUser, ctl, wanted); + return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); } // ==== Database ==== @@ -350,8 +354,7 @@ public boolean checkDbPriv(ConnectContext ctx, String ctl, String db, PrivPredic } public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - boolean hasGlobal = checkGlobalPriv(currentUser, wanted); - return getAccessControllerOrDefault(ctl).checkDbPriv(hasGlobal, currentUser, ctl, db, wanted); + return getAccessControllerOrDefault(ctl).checkDbPriv(currentUser, ctl, db, wanted); } // ==== Table ==== @@ -369,8 +372,7 @@ public boolean checkTblPriv(ConnectContext ctx, String qualifiedCtl, } public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - boolean hasGlobal = checkGlobalPriv(currentUser, wanted); - return getAccessControllerOrDefault(ctl).checkTblPriv(hasGlobal, currentUser, ctl, db, tbl, wanted); + return getAccessControllerOrDefault(ctl).checkTblPriv(currentUser, ctl, db, tbl, wanted); } // ==== Column ==== @@ -386,11 +388,9 @@ public void checkColumnsPriv(ConnectContext ctx, String ctl, String qualifiedDb, public void checkColumnsPriv(UserIdentity currentUser, String ctl, String qualifiedDb, String tbl, Set cols, PrivPredicate wanted) throws UserException { - boolean hasGlobal = checkGlobalPriv(currentUser, wanted); CatalogAccessController accessController = getAccessControllerOrDefault(ctl); long start = System.currentTimeMillis(); - accessController.checkColsPriv(hasGlobal, currentUser, ctl, qualifiedDb, - tbl, cols, wanted); + accessController.checkColsPriv(currentUser, ctl, qualifiedDb, tbl, cols, wanted); if (LOG.isDebugEnabled()) { LOG.debug("checkColumnsPriv use {} mills, user: {}, ctl: {}, db: {}, table: {}, cols: {}", System.currentTimeMillis() - start, currentUser, ctl, qualifiedDb, tbl, cols); diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java index deb144722b2c91..00ce013746e0ca 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java @@ -27,18 +27,20 @@ import java.util.Optional; import java.util.Set; +/** + * Decides access to the resources of one catalog. + * + *

A controller is asked only about the resources it governs, and its answer is final: nothing outside it + * grants first. In particular the engine no longer establishes a global privilege before routing, so an + * implementation that wants "holding the privilege globally is enough" has to say so itself - see + * {@link InternalAccessController}, which checks global privileges ahead of the fine grained ones, and + * {@link org.apache.doris.catalog.authorizer.ranger.RangerAccessController}, which defers to whichever + * controller owns global scope. + */ public interface CatalogAccessController { default void close() { } - // ==== Catalog ==== - default boolean checkCtlPriv(boolean hasGlobal, UserIdentity currentUser, String ctl, PrivPredicate wanted) { - if (hasGlobal) { - return true; - } - return checkCtlPriv(currentUser, ctl, wanted); - } - // ==== Global ==== boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted); @@ -46,36 +48,11 @@ default boolean checkCtlPriv(boolean hasGlobal, UserIdentity currentUser, String boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted); // ==== Database ==== - default boolean checkDbPriv(boolean hasGlobal, UserIdentity currentUser, String ctl, String db, - PrivPredicate wanted) { - if (hasGlobal) { - return true; - } - return checkDbPriv(currentUser, ctl, db, wanted); - } - boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted); // ==== Table ==== - default boolean checkTblPriv(boolean hasGlobal, UserIdentity currentUser, String ctl, String db, String tbl, - PrivPredicate wanted) { - if (hasGlobal) { - return true; - } - return checkTblPriv(currentUser, ctl, db, tbl, wanted); - } - boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted); - // ==== Column ==== - default void checkColsPriv(boolean hasGlobal, UserIdentity currentUser, String ctl, String db, String tbl, - Set cols, PrivPredicate wanted) throws AuthorizationException { - if (hasGlobal) { - return; - } - checkColsPriv(currentUser, ctl, db, tbl, cols, wanted); - } - // ==== Resource ==== boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted); diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java index 51c6ae9f1203e9..333cb8e6983fb3 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java @@ -34,6 +34,14 @@ import java.util.Optional; import java.util.Set; +/** + * The privilege model Doris ships with: users, roles and {@code GRANT} statements. + * + *

Every scoped check answers "globally, then at this scope". That order is this implementation's own + * decision, not something the engine arranges for it, and it is not redundant with the checks {@link Auth} + * runs per role: {@link Auth} refuses NODE privileges below global level, so a caller holding only global + * NODE_PRIV is granted by the global check here and by nothing else. + */ public class InternalAccessController implements CatalogAccessController { private Auth auth; @@ -48,22 +56,25 @@ public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { @Override public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - return auth.checkCtlPriv(currentUser, ctl, wanted); + return checkGlobalPriv(currentUser, wanted) || auth.checkCtlPriv(currentUser, ctl, wanted); } @Override public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - return auth.checkDbPriv(currentUser, ctl, db, wanted); + return checkGlobalPriv(currentUser, wanted) || auth.checkDbPriv(currentUser, ctl, db, wanted); } @Override public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - return auth.checkTblPriv(currentUser, ctl, db, tbl, wanted); + return checkGlobalPriv(currentUser, wanted) || auth.checkTblPriv(currentUser, ctl, db, tbl, wanted); } @Override public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, PrivPredicate wanted) throws AuthorizationException { + if (checkGlobalPriv(currentUser, wanted)) { + return; + } auth.checkColsPriv(currentUser, ctl, db, tbl, cols, wanted); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java new file mode 100644 index 00000000000000..a47e38e8d40372 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java @@ -0,0 +1,126 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.catalog.authorizer.ranger; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.catalog.Env; +import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; +import org.apache.doris.common.jmockit.Deencapsulation; +import org.apache.doris.mysql.privilege.AccessControllerManager; +import org.apache.doris.mysql.privilege.Auth; +import org.apache.doris.mysql.privilege.CatalogAccessController; +import org.apache.doris.mysql.privilege.PrivPredicate; +import org.apache.doris.mysql.privilege.StubRangerPolicyEngine; + +import org.apache.ranger.plugin.policyengine.RangerAccessRequest; +import org.apache.ranger.plugin.policyengine.RangerAccessResult; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.MockedStatic; +import org.mockito.Mockito; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.BooleanSupplier; + +/** + * A catalog bound to Ranger still lets through whoever holds the privilege at global scope. + * + *

Global scope is not something a Ranger service knows about; it belongs to the controller named by + * {@code access_controller_type}. Honouring it is why a cluster administrator can still reach a Ranger-governed + * catalog after the engine stopped establishing privileges on a plugin's behalf. It is the plugin's own choice, + * so it is tested on the plugin, and a plugin that declines it stays a legal plugin. + */ +public class RangerGlobalScopeDeferenceTest { + private static final UserIdentity ADMIN = UserIdentity.createAnalyzedUserIdentWithIp("admin_user", "%"); + private static final UserIdentity RANGER_USER = + UserIdentity.createAnalyzedUserIdentWithIp(StubRangerPolicyEngine.ALLOWED_USER, "%"); + + /** Counts what actually reached the policy engine, so "answered without asking Ranger" is observable. */ + private static final class CountingPolicyEngine extends StubRangerPolicyEngine { + private final AtomicInteger requests = new AtomicInteger(); + + @Override + public RangerAccessResult isAccessAllowed(RangerAccessRequest request) { + requests.incrementAndGet(); + return super.isAccessAllowed(request); + } + } + + private final CountingPolicyEngine engine = new CountingPolicyEngine(); + private final RangerDorisAccessController controller = new RangerDorisAccessController(engine); + + @Test + public void testGlobalScopeAuthorityGrantsWithoutConsultingRanger() { + CatalogAccessController authority = Mockito.mock(CatalogAccessController.class); + Mockito.when(authority.checkGlobalPriv(ADMIN, PrivPredicate.SELECT)).thenReturn(true); + + boolean allowed = withGlobalScopeAuthority(authority, + () -> controller.checkTblPriv(ADMIN, "ctl", StubRangerPolicyEngine.ALLOWED_DB, + StubRangerPolicyEngine.ALLOWED_TABLE, PrivPredicate.SELECT)); + + Assert.assertTrue(allowed); + // Not merely an optimisation: the Ranger policy set denies this user everywhere, so had the request + // reached the engine the answer would have been the opposite one. + Assert.assertEquals(0, engine.requests.get()); + } + + @Test + public void testRangerDecidesWhenTheAuthorityGrantsNothingGlobally() { + CatalogAccessController authority = Mockito.mock(CatalogAccessController.class); + + Assert.assertTrue(withGlobalScopeAuthority(authority, + () -> controller.checkTblPriv(RANGER_USER, "ctl", StubRangerPolicyEngine.ALLOWED_DB, + StubRangerPolicyEngine.ALLOWED_TABLE, PrivPredicate.SELECT))); + Assert.assertFalse(withGlobalScopeAuthority(authority, + () -> controller.checkTblPriv(RANGER_USER, "ctl", StubRangerPolicyEngine.ALLOWED_DB, + "other_tbl", PrivPredicate.SELECT))); + } + + /** + * With Ranger installed globally, the controller is itself the authority and its own global check already + * answers the question - asking through the manager would evaluate the same Ranger policies a second time + * on every database, table and column check. + */ + @Test + public void testBeingTheAuthorityCostsNoExtraPolicyEvaluation() { + int asPlugin = requestsWhile(Mockito.mock(CatalogAccessController.class), + () -> controller.checkDbPriv(RANGER_USER, "ctl", "other_db", PrivPredicate.SELECT)); + int asAuthority = requestsWhile(controller, + () -> controller.checkDbPriv(RANGER_USER, "ctl", "other_db", PrivPredicate.SELECT)); + + Assert.assertEquals(asPlugin, asAuthority); + } + + private int requestsWhile(CatalogAccessController authority, BooleanSupplier check) { + engine.requests.set(0); + withGlobalScopeAuthority(authority, check); + return engine.requests.get(); + } + + /** Runs {@code check} against an FE whose {@code access_controller_type} resolves to {@code authority}. */ + private boolean withGlobalScopeAuthority(CatalogAccessController authority, BooleanSupplier check) { + AccessControllerManager manager = new AccessControllerManager(new Auth()); + Deencapsulation.setField(manager, "defaultAccessController", authority); + try (MockedStatic mockedEnv = Mockito.mockStatic(Env.class)) { + Env env = Mockito.mock(Env.class); + mockedEnv.when(Env::getCurrentEnv).thenReturn(env); + Mockito.when(env.getAccessManager()).thenReturn(manager); + return check.getAsBoolean(); + } + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java index 4abeee55ee7c75..618cd120fbfbda 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java @@ -68,11 +68,12 @@ * Records, in one golden file, every decision {@link AccessControllerManager} makes over the matrix * (resource kind x action x built-in/Ranger x privilege level of the caller). * - *

Why this exists. Two reworks are queued behind it: dropping the {@code hasGlobal} argument from - * the controller interface (moving "global first, then fine grained" inside the built-in implementation), and - * collapsing the manager into pure routing with no cross-plugin OR. Both are supposed to change structure and - * nothing else. "Nothing else" is only checkable against a recording made before the change, so this file is - * that recording: after either rework, {@code git diff} on the baseline must be empty. + *

Why this exists. It was recorded ahead of two reworks that were meant to change structure and + * nothing else - dropping the {@code hasGlobal} argument from the controller interface, and collapsing the + * manager into pure routing with no cross-plugin OR - because "nothing else" is only checkable against a + * recording made beforehand. Both have since landed against it unchanged. The rest of the plugin work moves + * the same decisions around further, so the rule stands: after a structural change, {@code git diff} on the + * baseline must be empty. * *

What is real and what is faked. The built-in half is entirely real - a real FE, real users, real * {@code GRANT} statements, a real row policy - because a baseline built on hand-poked internal state would @@ -89,11 +90,12 @@ * because Phase 0's contract is structure-only: *

    *
  • {@code node_user} holds only global NODE_PRIV, and {@link Auth} refuses NODE privileges at - * catalog/database/table level - yet the manager's global short circuit lets OPERATOR through there;
    • - *
  • {@code admin_user} is denied by Ranger everywhere, yet passes on the Ranger-governed catalog: that is - * the cross-plugin OR the rework must reproduce from inside the plugin;
    • + * catalog/database/table level - yet the built-in controller's global check lets OPERATOR through + * there; + *
  • {@code admin_user} is denied by Ranger everywhere, yet passes on the Ranger-governed catalog, because + * the Ranger controller defers to whichever controller owns global scope;
    • *
  • a built-in per-table GRANT on a Ranger-governed catalog is ignored ({@code local_user} on - * {@code ext_ranger}), because the OR only exists at the global level;
    • + * {@code ext_ranger}), because that deference exists only at the global level; *
  • the workload group named {@code normal} is allowed unconditionally, in both implementations;
    • *
  • with {@code skip_catalog_priv_check} on, a catalog bound to an external plugin answers SELECT/SHOW * with a flat yes.
    • diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java index a6b2514b43aa45..06d8ca6d56653d 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java @@ -62,7 +62,6 @@ public void testCheckCtlPrivSkipCatalogPrivCheckWithCustomAccessControllerForSel UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("custom_catalog")).thenReturn(catalog); @@ -87,7 +86,6 @@ public void testCheckCtlPrivSkipCatalogPrivCheckWithCustomAccessControllerForSho UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("custom_catalog")).thenReturn(catalog); @@ -112,9 +110,8 @@ public void testCheckCtlPrivSkipCatalogPrivCheckWithoutCustomAccessController() UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); Mockito.when(defaultAccessController.checkCtlPriv( - Mockito.anyBoolean(), Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(false); + Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(false); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("custom_catalog")).thenReturn(catalog); @@ -137,9 +134,8 @@ public void testCheckCtlPrivCreateMustCheckDefaultAccessController() { UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); Mockito.when(defaultAccessController.checkCtlPriv( - Mockito.anyBoolean(), Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(true); + Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(true); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("not_exist_catalog")).thenReturn(null); @@ -161,9 +157,8 @@ public void testCheckCtlPrivLoadMustCheckDefaultAccessController() { UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); Mockito.when(defaultAccessController.checkCtlPriv( - Mockito.anyBoolean(), Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(false); + Mockito.any(), Mockito.anyString(), Mockito.any())).thenReturn(false); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("custom_catalog")).thenReturn(catalog); @@ -187,7 +182,6 @@ public void testCheckCtlPrivSkipCatalogPrivCheckWhenCatalogNotExist() { UserIdentity userIdentity = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); Config.skip_catalog_priv_check = true; - Mockito.when(defaultAccessController.checkGlobalPriv(Mockito.any(), Mockito.any())).thenReturn(false); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("not_exist_catalog")).thenReturn(null); diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java deleted file mode 100644 index 58a18566f405cd..00000000000000 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CatalogAccessControllerTest.java +++ /dev/null @@ -1,260 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.analysis.ResourceTypeEnum; -import org.apache.doris.analysis.UserIdentity; -import org.apache.doris.authorization.DataMaskSpec; -import org.apache.doris.authorization.RowFilterSpec; -import org.apache.doris.common.AuthorizationException; - -import com.google.common.collect.ImmutableList; -import com.google.common.collect.ImmutableSet; -import org.junit.Assert; -import org.junit.Test; - -import java.util.List; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.atomic.AtomicBoolean; - -public class CatalogAccessControllerTest { - - private static class StubAccessController implements CatalogAccessController { - final AtomicBoolean ctlPrivCalled = new AtomicBoolean(false); - final AtomicBoolean dbPrivCalled = new AtomicBoolean(false); - final AtomicBoolean tblPrivCalled = new AtomicBoolean(false); - final AtomicBoolean colsPrivCalled = new AtomicBoolean(false); - - private final boolean ctlResult; - private final boolean dbResult; - private final boolean tblResult; - private final boolean colsResult; - - StubAccessController() { - this(false, false, false, false); - } - - StubAccessController(boolean ctlResult, boolean dbResult, boolean tblResult) { - this(ctlResult, dbResult, tblResult, false); - } - - StubAccessController(boolean ctlResult, boolean dbResult, boolean tblResult, boolean colsResult) { - this.ctlResult = ctlResult; - this.dbResult = dbResult; - this.tblResult = tblResult; - this.colsResult = colsResult; - } - - @Override - public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { - return false; - } - - @Override - public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - ctlPrivCalled.set(true); - return ctlResult; - } - - @Override - public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - dbPrivCalled.set(true); - return dbResult; - } - - @Override - public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - tblPrivCalled.set(true); - return tblResult; - } - - @Override - public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, - Set cols, PrivPredicate wanted) throws AuthorizationException { - colsPrivCalled.set(true); - if (!colsResult) { - throw new AuthorizationException("denied"); - } - } - - @Override - public boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted) { - return false; - } - - @Override - public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadGroupName, - PrivPredicate wanted) { - return false; - } - - @Override - public boolean checkCloudPriv(UserIdentity currentUser, String cloudName, - PrivPredicate wanted, ResourceTypeEnum type) { - return false; - } - - @Override - public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, - PrivPredicate wanted) { - return false; - } - - @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, - String tbl, String col) { - return Optional.empty(); - } - - @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, - String db, String tbl) { - return ImmutableList.of(); - } - } - - @Test - public void testCheckCtlPrivShortCircuitOnHasGlobal() { - StubAccessController controller = new StubAccessController(); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkCtlPriv(true, user, "ctl", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertFalse(controller.ctlPrivCalled.get()); - } - - @Test - public void testCheckCtlPrivFallsThroughWithoutHasGlobal() { - StubAccessController controller = new StubAccessController(true, false, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkCtlPriv(false, user, "ctl", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertTrue(controller.ctlPrivCalled.get()); - } - - @Test - public void testCheckCtlPrivFallsThroughAndReturnsFalse() { - StubAccessController controller = new StubAccessController(false, false, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkCtlPriv(false, user, "ctl", PrivPredicate.SELECT); - - Assert.assertFalse(result); - Assert.assertTrue(controller.ctlPrivCalled.get()); - } - - @Test - public void testCheckDbPrivShortCircuitOnHasGlobal() { - StubAccessController controller = new StubAccessController(); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkDbPriv(true, user, "ctl", "db", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertFalse(controller.dbPrivCalled.get()); - } - - @Test - public void testCheckDbPrivFallsThroughWithoutHasGlobal() { - StubAccessController controller = new StubAccessController(false, true, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkDbPriv(false, user, "ctl", "db", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertTrue(controller.dbPrivCalled.get()); - } - - @Test - public void testCheckDbPrivFallsThroughAndReturnsFalse() { - StubAccessController controller = new StubAccessController(false, false, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkDbPriv(false, user, "ctl", "db", PrivPredicate.SELECT); - - Assert.assertFalse(result); - Assert.assertTrue(controller.dbPrivCalled.get()); - } - - @Test - public void testCheckTblPrivShortCircuitOnHasGlobal() { - StubAccessController controller = new StubAccessController(); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkTblPriv(true, user, "ctl", "db", "tbl", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertFalse(controller.tblPrivCalled.get()); - } - - @Test - public void testCheckTblPrivFallsThroughWithoutHasGlobal() { - StubAccessController controller = new StubAccessController(false, false, true); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkTblPriv(false, user, "ctl", "db", "tbl", PrivPredicate.SELECT); - - Assert.assertTrue(result); - Assert.assertTrue(controller.tblPrivCalled.get()); - } - - @Test - public void testCheckTblPrivFallsThroughAndReturnsFalse() { - StubAccessController controller = new StubAccessController(false, false, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - boolean result = controller.checkTblPriv(false, user, "ctl", "db", "tbl", PrivPredicate.SELECT); - - Assert.assertFalse(result); - Assert.assertTrue(controller.tblPrivCalled.get()); - } - - @Test - public void testCheckColsPrivShortCircuitOnHasGlobal() throws AuthorizationException { - StubAccessController controller = new StubAccessController(); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - controller.checkColsPriv(true, user, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - - Assert.assertFalse(controller.colsPrivCalled.get()); - } - - @Test - public void testCheckColsPrivFallsThroughWithoutHasGlobal() throws AuthorizationException { - StubAccessController controller = new StubAccessController(false, false, false, true); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - controller.checkColsPriv(false, user, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - - Assert.assertTrue(controller.colsPrivCalled.get()); - } - - @Test - public void testCheckColsPrivFallsThroughAndThrows() { - StubAccessController controller = new StubAccessController(false, false, false, false); - UserIdentity user = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - Assert.assertThrows(AuthorizationException.class, () -> - controller.checkColsPriv(false, user, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT)); - Assert.assertTrue(controller.colsPrivCalled.get()); - } -} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java new file mode 100644 index 00000000000000..49d0d19f7368d5 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java @@ -0,0 +1,139 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.common.AuthorizationException; + +import com.google.common.collect.ImmutableSet; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; + +/** + * The built-in controller answers "globally, then at this scope". + * + *

    Both halves of that sentence are load bearing. Answering globally first is what lets an administrator + * reach a resource no grant names, and it must happen before the scoped lookup rather than instead of + * it - the scoped lookups are the expensive ones, and one of them refuses to answer at all for privileges that + * only exist globally. The engine used to arrange this order on every controller's behalf; now each controller + * owns it, so these tests watch the built-in one keep it. + */ +public class InternalAccessControllerTest { + private static final UserIdentity USER = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); + + private final Auth auth = Mockito.mock(Auth.class); + private final InternalAccessController controller = new InternalAccessController(auth); + + private void holdsGlobally(boolean granted) { + Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.SELECT)).thenReturn(granted); + } + + @Test + public void testCatalogCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(controller.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)); + Mockito.verify(auth, Mockito.never()).checkCtlPriv(Mockito.any(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testCatalogCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(controller.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)); + Assert.assertFalse(controller.checkCtlPriv(USER, "other_ctl", PrivPredicate.SELECT)); + } + + @Test + public void testDatabaseCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)); + Mockito.verify(auth, Mockito.never()) + .checkDbPriv(Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testDatabaseCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)); + Assert.assertFalse(controller.checkDbPriv(USER, "ctl", "other_db", PrivPredicate.SELECT)); + } + + @Test + public void testTableCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(controller.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)); + Mockito.verify(auth, Mockito.never()).checkTblPriv( + Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testTableCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(controller.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)); + Assert.assertFalse(controller.checkTblPriv(USER, "ctl", "db", "other_tbl", PrivPredicate.SELECT)); + } + + @Test + public void testColumnCheckIsSkippedWhenPrivilegeIsHeldGlobally() throws Exception { + holdsGlobally(true); + + controller.checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + Mockito.verify(auth, Mockito.never()).checkColsPriv(Mockito.any(), Mockito.anyString(), + Mockito.anyString(), Mockito.anyString(), Mockito.any(), Mockito.any()); + } + + @Test + public void testColumnCheckDecidesWhenPrivilegeIsNotHeldGlobally() throws Exception { + holdsGlobally(false); + + controller.checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + Mockito.verify(auth).checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + } + + @Test + public void testColumnDenialIsReportedWhenPrivilegeIsNotHeldGlobally() throws Exception { + holdsGlobally(false); + Mockito.doThrow(new AuthorizationException("denied")).when(auth).checkColsPriv( + USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + + Assert.assertThrows(AuthorizationException.class, () -> controller.checkColsPriv( + USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT)); + } + + /** + * A caller holding only global NODE_PRIV is why the global check has to run first instead of being folded + * into the scoped one: {@link Auth} refuses NODE privileges below global level, so the scoped lookup would + * turn the administrator away. + */ + @Test + public void testGloballyHeldNodePrivilegeIsNotRefusedByTheScopedCheck() { + Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.OPERATOR)).thenReturn(true); + Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.OPERATOR)).thenReturn(false); + + Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.OPERATOR)); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java index 52549ee09be31e..57b64c2e1d6ca6 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerPolicyEngine.java @@ -52,9 +52,10 @@ * everywhere, and gets no filter and no mask. *

* - *

Denying the built-in admin is the point: wherever the baseline still records ALLOW for that user on a - * Ranger-governed resource, the decision provably came from the engine's cross-plugin OR rather than from - * Ranger, which is exactly the behaviour the routing rework has to reproduce. + *

Denying the built-in admin is the point: wherever the baseline records ALLOW for that user on a + * Ranger-governed resource, the decision provably did not come from Ranger. It comes from the controller + * deferring to whoever owns global scope, and pinning it here is what keeps that deference from being dropped + * or widened unnoticed. */ public class StubRangerPolicyEngine extends RangerBasePlugin { public static final String ALLOWED_USER = "ranger_user"; From fdd035ca47126a28f5b26456464b6158a741e20b Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 16:04:31 +0800 Subject: [PATCH 06/22] [improvement](authorization) route every access check through one decision The manager offered nine ways to ask the same question, each naming its own resource in loose strings and each picking a controller on its own. Which controller answers for what was therefore spread across nine method bodies, and the resource was described differently in every one of them. There is one routing point now. decide(subject, resource, requirement) picks the controller from the resource alone and returns its answer; the nine entry points build a resource and call it. Adding a chain of sources later, or an audit record of what was asked, is a change to that one function. The resource and the requirement are neutral types, in fe-authorization-api, so they can later cross into a plugin that must not see fe-core. Three things about their shape were decided by what the code actually does: - an action stands one-to-one with a privilege Doris grants. Folding cluster usage and stage usage into plain usage would leave the built-in model - one of the sources being asked - unable to tell which of its three privilege bits the caller meant. - a requirement carries a set of actions and whether one or all are needed, rather than a single action. PrivPredicate is not a closed set of constants: a GRANT statement builds one on the spot out of the privileges it names, and those are the "all of" ones. Asking one action at a time would also undo the Ranger controller's habit of remembering which privileges an outer resource level already granted, turning one walk of the hierarchy into several. - translating back returns the very constant the requirement came from. The engine tells questions apart by comparing predicates against those constants with ==, in Role and in both Ranger controllers, so an equal-but-different object silently drops "may see this catalog" for every user who had no explicit grant on it. The behaviour baseline caught exactly that, twelve cells of it, before this was rewritten to canonicalize. Columns keep an entry point of their own. Their answer has a different shape - which column was refused, carried by an exception - and merging it into a yes-or-no now would mean swallowing that. Verified: behaviour baseline unchanged; 67 tests in 10 classes, no failures; checkstyle clean. Mutation: making a GRANT statement's "all of these privileges" read as "any of them" leaves the baseline green - it only enumerates the OR-shaped constants - and turns the round trip tests red, which is why they exist. Co-Authored-By: Claude Opus 5 (1M context) --- .../doris/authorization/AccessAction.java | 58 +++ .../authorization/AccessRequirement.java | 131 +++++++ .../doris/authorization/ActionMatch.java | 32 ++ .../authorization/AuthorizedResource.java | 329 ++++++++++++++++++ .../doris/authorization/ResourceKind.java | 50 +++ .../authorization/AccessRequirementTest.java | 83 +++++ .../authorization/AuthorizedResourceTest.java | 108 ++++++ .../privilege/AccessControllerManager.java | 126 +++++-- .../mysql/privilege/AccessTranslation.java | 210 +++++++++++ .../privilege/AccessTranslationTest.java | 227 ++++++++++++ 10 files changed, 1331 insertions(+), 23 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessAction.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirement.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ActionMatch.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedResource.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ResourceKind.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessRequirementTest.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedResourceTest.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessAction.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessAction.java new file mode 100644 index 00000000000000..f6e776b5b7048d --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessAction.java @@ -0,0 +1,58 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +/** + * One privilege an authorization source can grant on a resource. + * + *

The constants stand one-to-one with the privileges Doris itself grants, and that is a requirement + * rather than a coincidence: the built-in privilege model is one of the authorization sources, so any + * folding here - "cluster usage and stage usage are both just usage", say - would leave it unable to tell + * which privilege it was actually asked about, and it would answer a different question than the caller + * asked.

+ * + *

There is deliberately no {@code SHOW} action. "May this subject see the object at all" is not a + * privilege anyone grants; it is the question "does the subject hold any of the privileges that + * imply visibility", which {@link AccessRequirement#anyOf} expresses.

+ */ +public enum AccessAction { + /** Cluster node operations. Only ever granted globally. */ + NODE, + /** Administration of the cluster. */ + ADMIN, + /** Granting privileges to others. */ + GRANT, + /** Reading data. */ + SELECT, + /** Writing data. */ + LOAD, + /** Altering an object's definition. */ + ALTER, + /** Creating an object. */ + CREATE, + /** Dropping an object. */ + DROP, + /** Using a resource or a workload group. */ + USAGE, + /** Using a compute group. Distinct from {@link #USAGE}: it is a privilege of its own. */ + CLUSTER_USAGE, + /** Using a stage. Distinct from {@link #USAGE}: it is a privilege of its own. */ + STAGE_USAGE, + /** Reading a view's definition. */ + SHOW_VIEW +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirement.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirement.java new file mode 100644 index 00000000000000..83a828f761e7e2 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirement.java @@ -0,0 +1,131 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Collection; +import java.util.Collections; +import java.util.EnumSet; +import java.util.Objects; +import java.util.Set; + +/** + * What a caller needs on a resource: a set of actions and how many of them must be held. + * + *

A requirement is asked as a whole rather than one action at a time, because the two sources of + * authorization decisions answer it as a whole. The built-in model tests a bit set in a single pass; the + * Ranger plugin walks the resource hierarchy asking about one privilege per request and remembers which ones + * an outer level already granted, so an already-answered privilege is never asked about twice. Handing them + * one action at a time would replace both with N independent evaluations of the same policies - the same + * answer, several times the cost. A plugin that has no such structure can still be written one action at a + * time and let the SPI's default implementation take this apart.

+ * + *

The action set is never empty. An empty requirement is not a harmless edge case: read as {@link + * ActionMatch#ALL} it is satisfied by everyone, which is a silent grant of everything the caller was + * checking.

+ */ +public final class AccessRequirement { + + private final Set actions; + private final ActionMatch match; + + private AccessRequirement(Set actions, ActionMatch match) { + this.actions = actions; + this.match = match; + } + + /** A requirement satisfied by holding at least one of {@code actions}. */ + public static AccessRequirement anyOf(AccessAction... actions) { + return of(toSet(actions), ActionMatch.ANY); + } + + /** A requirement satisfied only by holding every one of {@code actions}. */ + public static AccessRequirement allOf(AccessAction... actions) { + return of(toSet(actions), ActionMatch.ALL); + } + + /** A requirement for a single action; the match type is irrelevant for one action. */ + public static AccessRequirement of(AccessAction action) { + return of(EnumSet.of(Objects.requireNonNull(action, "action is required")), ActionMatch.ANY); + } + + /** + * @param actions the actions in question, at least one + * @param match whether one of them suffices or all of them are needed + */ + public static AccessRequirement of(Collection actions, ActionMatch match) { + Objects.requireNonNull(actions, "actions is required"); + Objects.requireNonNull(match, "match is required"); + if (actions.isEmpty()) { + throw new IllegalArgumentException("an access requirement must name at least one action"); + } + EnumSet copy = EnumSet.noneOf(AccessAction.class); + for (AccessAction action : actions) { + copy.add(Objects.requireNonNull(action, "actions must not contain null")); + } + return new AccessRequirement(Collections.unmodifiableSet(copy), match); + } + + public Set getActions() { + return actions; + } + + public ActionMatch getMatch() { + return match; + } + + /** + * Whether {@code granted} - everything the subject holds on the resource - satisfies this requirement. + * Provided so that every implementation reads the match type the same way instead of writing its own + * {@code containsAll} versus {@code disjoint} by hand. + */ + public boolean isSatisfiedBy(Set granted) { + Objects.requireNonNull(granted, "granted is required"); + return match == ActionMatch.ALL ? granted.containsAll(actions) : !Collections.disjoint(granted, actions); + } + + private static EnumSet toSet(AccessAction... actions) { + Objects.requireNonNull(actions, "actions is required"); + EnumSet set = EnumSet.noneOf(AccessAction.class); + for (AccessAction action : actions) { + set.add(Objects.requireNonNull(action, "actions must not contain null")); + } + return set; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof AccessRequirement)) { + return false; + } + AccessRequirement that = (AccessRequirement) o; + return match == that.match && actions.equals(that.actions); + } + + @Override + public int hashCode() { + return Objects.hash(actions, match); + } + + @Override + public String toString() { + return match + actions.toString(); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ActionMatch.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ActionMatch.java new file mode 100644 index 00000000000000..f12bafc87a988a --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ActionMatch.java @@ -0,0 +1,32 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +/** + * How many of an {@link AccessRequirement}'s actions the subject must hold. + * + *

Both are in daily use and they are not interchangeable: reading a table requires + * {@link #ANY} of several privileges, whereas granting a privilege to someone else requires + * {@link #ALL} of "the privilege being granted" and "the right to grant".

+ */ +public enum ActionMatch { + /** The subject must hold at least one of the actions. */ + ANY, + /** The subject must hold every one of the actions. */ + ALL +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedResource.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedResource.java new file mode 100644 index 00000000000000..f8e8f03657af18 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedResource.java @@ -0,0 +1,329 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Collections; +import java.util.LinkedHashSet; +import java.util.Objects; +import java.util.Set; + +/** + * The object an authorization decision is about. + * + *

Every variant is nested here and can only be built through the factory methods, so the set of things + * that can be asked about is closed: an implementation may switch on {@link #getKind()} and treat an + * unknown kind as a defect rather than as something to guess about. Names are carried as given - no + * lower-casing, no qualification - because whoever asks already resolved them and a second normalization + * here would decide access on a name nobody used.

+ */ +public abstract class AuthorizedResource { + + private final ResourceKind kind; + + private AuthorizedResource(ResourceKind kind) { + this.kind = kind; + } + + public final ResourceKind getKind() { + return kind; + } + + /** The cluster as a whole. */ + public static Global global() { + return Global.INSTANCE; + } + + public static Catalog catalog(String catalog) { + return new Catalog(catalog); + } + + public static Database database(String catalog, String database) { + return new Database(catalog, database); + } + + public static Table table(String catalog, String database, String table) { + return new Table(catalog, database, table); + } + + public static Columns columns(String catalog, String database, String table, Set columns) { + return new Columns(catalog, database, table, columns); + } + + public static Named resource(String name) { + return new Named(ResourceKind.RESOURCE, name); + } + + public static Named workloadGroup(String name) { + return new Named(ResourceKind.WORKLOAD_GROUP, name); + } + + public static Named storageVault(String name) { + return new Named(ResourceKind.STORAGE_VAULT, name); + } + + /** + * A cloud object, reached through the cloud privilege path. + * + * @param kind one of the {@code CLOUD_*} kinds + */ + public static Named cloud(ResourceKind kind, String name) { + Objects.requireNonNull(kind, "kind is required"); + switch (kind) { + case CLOUD_GENERAL: + case CLOUD_COMPUTE_GROUP: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + return new Named(kind, name); + default: + throw new IllegalArgumentException(kind + " is not a cloud resource kind"); + } + } + + /** The cluster as a whole; global privileges are the ones held on it. */ + public static final class Global extends AuthorizedResource { + private static final Global INSTANCE = new Global(); + + private Global() { + super(ResourceKind.GLOBAL); + } + + @Override + public String toString() { + return "global"; + } + } + + /** One catalog. */ + public static final class Catalog extends AuthorizedResource { + private final String catalog; + + private Catalog(String catalog) { + super(ResourceKind.CATALOG); + this.catalog = Objects.requireNonNull(catalog, "catalog is required"); + } + + public String getCatalog() { + return catalog; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof Catalog)) { + return false; + } + return catalog.equals(((Catalog) o).catalog); + } + + @Override + public int hashCode() { + return catalog.hashCode(); + } + + @Override + public String toString() { + return "catalog " + catalog; + } + } + + /** One database of one catalog. */ + public static final class Database extends AuthorizedResource { + private final String catalog; + private final String database; + + private Database(String catalog, String database) { + super(ResourceKind.DATABASE); + this.catalog = Objects.requireNonNull(catalog, "catalog is required"); + this.database = Objects.requireNonNull(database, "database is required"); + } + + public String getCatalog() { + return catalog; + } + + public String getDatabase() { + return database; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof Database)) { + return false; + } + Database that = (Database) o; + return catalog.equals(that.catalog) && database.equals(that.database); + } + + @Override + public int hashCode() { + return Objects.hash(catalog, database); + } + + @Override + public String toString() { + return "database " + catalog + "." + database; + } + } + + /** One table, view or any other relation. */ + public static final class Table extends AuthorizedResource { + private final String catalog; + private final String database; + private final String table; + + private Table(String catalog, String database, String table) { + super(ResourceKind.TABLE); + this.catalog = Objects.requireNonNull(catalog, "catalog is required"); + this.database = Objects.requireNonNull(database, "database is required"); + this.table = Objects.requireNonNull(table, "table is required"); + } + + public String getCatalog() { + return catalog; + } + + public String getDatabase() { + return database; + } + + public String getTable() { + return table; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof Table)) { + return false; + } + Table that = (Table) o; + return catalog.equals(that.catalog) && database.equals(that.database) && table.equals(that.table); + } + + @Override + public int hashCode() { + return Objects.hash(catalog, database, table); + } + + @Override + public String toString() { + return "table " + catalog + "." + database + "." + table; + } + } + + /** Named columns of one table, asked about together. */ + public static final class Columns extends AuthorizedResource { + private final String catalog; + private final String database; + private final String table; + private final Set columns; + + private Columns(String catalog, String database, String table, Set columns) { + super(ResourceKind.COLUMNS); + this.catalog = Objects.requireNonNull(catalog, "catalog is required"); + this.database = Objects.requireNonNull(database, "database is required"); + this.table = Objects.requireNonNull(table, "table is required"); + Objects.requireNonNull(columns, "columns is required"); + // Insertion order is kept: a denial names the first column that failed, and a set that reorders + // would make which column is reported depend on hashing. + this.columns = Collections.unmodifiableSet(new LinkedHashSet<>(columns)); + } + + public String getCatalog() { + return catalog; + } + + public String getDatabase() { + return database; + } + + public String getTable() { + return table; + } + + public Set getColumns() { + return columns; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof Columns)) { + return false; + } + Columns that = (Columns) o; + return catalog.equals(that.catalog) && database.equals(that.database) + && table.equals(that.table) && columns.equals(that.columns); + } + + @Override + public int hashCode() { + return Objects.hash(catalog, database, table, columns); + } + + @Override + public String toString() { + return "columns " + columns + " of " + catalog + "." + database + "." + table; + } + } + + /** A system-wide object addressed by a single name: a resource, a workload group, a vault, a stage. */ + public static final class Named extends AuthorizedResource { + private final String name; + + private Named(ResourceKind kind, String name) { + super(kind); + this.name = Objects.requireNonNull(name, "name is required"); + } + + public String getName() { + return name; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof Named)) { + return false; + } + Named that = (Named) o; + return getKind() == that.getKind() && name.equals(that.name); + } + + @Override + public int hashCode() { + return Objects.hash(getKind(), name); + } + + @Override + public String toString() { + return getKind() + " " + name; + } + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ResourceKind.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ResourceKind.java new file mode 100644 index 00000000000000..c93152a89a6551 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/ResourceKind.java @@ -0,0 +1,50 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +/** + * The kinds of object an authorization decision can be about. + * + *

The list is exhaustive by construction: it has one constant per question the engine can ask, so an + * implementation that switches over it and rejects the default branch cannot be silently bypassed by a new + * kind of object appearing.

+ * + *

The cloud kinds are separate from {@link #RESOURCE} and {@link #STORAGE_VAULT} rather than folded into + * them because they are separate questions today, answered against different privilege tables and with an + * extra fallback of their own. Folding them would change who is allowed in.

+ */ +public enum ResourceKind { + /** The cluster as a whole. */ + GLOBAL, + CATALOG, + DATABASE, + TABLE, + /** Named columns of one table. */ + COLUMNS, + /** A resource, e.g. an ODBC or S3 resource. */ + RESOURCE, + WORKLOAD_GROUP, + STORAGE_VAULT, + /** A cloud resource that is none of the kinds below. */ + CLOUD_GENERAL, + /** A compute group, historically named "cluster". */ + CLOUD_COMPUTE_GROUP, + CLOUD_STAGE, + /** A storage vault reached through the cloud privilege path, distinct from {@link #STORAGE_VAULT}. */ + CLOUD_STORAGE_VAULT +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessRequirementTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessRequirementTest.java new file mode 100644 index 00000000000000..8296aeb18a668a --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessRequirementTest.java @@ -0,0 +1,83 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.util.Collections; +import java.util.EnumSet; + +public class AccessRequirementTest { + + @Test + public void testAnyIsSatisfiedByOneOfTheActions() { + AccessRequirement requirement = AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.LOAD); + + Assertions.assertTrue(requirement.isSatisfiedBy(EnumSet.of(AccessAction.LOAD))); + Assertions.assertFalse(requirement.isSatisfiedBy(EnumSet.of(AccessAction.DROP))); + Assertions.assertFalse(requirement.isSatisfiedBy(EnumSet.noneOf(AccessAction.class))); + } + + @Test + public void testAllNeedsEveryAction() { + // The shape of the check a GRANT statement makes: holding one of the two is not enough. + AccessRequirement requirement = AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT); + + Assertions.assertTrue(requirement.isSatisfiedBy(EnumSet.of(AccessAction.SELECT, AccessAction.GRANT))); + Assertions.assertFalse(requirement.isSatisfiedBy(EnumSet.of(AccessAction.SELECT))); + Assertions.assertFalse(requirement.isSatisfiedBy(EnumSet.of(AccessAction.GRANT))); + } + + @Test + public void testRequirementWithoutActionsIsRefused() { + // Read as "all of", an empty requirement is satisfied by holding nothing at all, so a caller that + // built one by accident would be granting whatever it was checking. + Assertions.assertThrows(IllegalArgumentException.class, AccessRequirement::anyOf); + Assertions.assertThrows(IllegalArgumentException.class, + () -> AccessRequirement.of(Collections.emptySet(), ActionMatch.ALL)); + } + + @Test + public void testActionsCannotBeChangedAfterwards() { + AccessRequirement requirement = AccessRequirement.of(AccessAction.SELECT); + + Assertions.assertThrows(UnsupportedOperationException.class, + () -> requirement.getActions().add(AccessAction.ADMIN)); + } + + @Test + public void testMatchIsPartOfIdentity() { + AccessRequirement any = AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.GRANT); + AccessRequirement all = AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT); + + Assertions.assertNotEquals(any, all); + Assertions.assertEquals(any, AccessRequirement.anyOf(AccessAction.GRANT, AccessAction.SELECT)); + Assertions.assertEquals(any.hashCode(), + AccessRequirement.anyOf(AccessAction.GRANT, AccessAction.SELECT).hashCode()); + } + + @Test + public void testNullsAreRefused() { + Assertions.assertThrows(NullPointerException.class, () -> AccessRequirement.of((AccessAction) null)); + Assertions.assertThrows(NullPointerException.class, + () -> AccessRequirement.of(Collections.singleton(AccessAction.SELECT), null)); + Assertions.assertThrows(NullPointerException.class, + () -> AccessRequirement.of(AccessAction.SELECT).isSatisfiedBy(null)); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedResourceTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedResourceTest.java new file mode 100644 index 00000000000000..3e582bc68ff81f --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedResourceTest.java @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; + +public class AuthorizedResourceTest { + + @Test + public void testEachFactoryReportsItsKind() { + Assertions.assertEquals(ResourceKind.GLOBAL, AuthorizedResource.global().getKind()); + Assertions.assertEquals(ResourceKind.CATALOG, AuthorizedResource.catalog("ctl").getKind()); + Assertions.assertEquals(ResourceKind.DATABASE, AuthorizedResource.database("ctl", "db").getKind()); + Assertions.assertEquals(ResourceKind.TABLE, AuthorizedResource.table("ctl", "db", "tbl").getKind()); + Assertions.assertEquals(ResourceKind.COLUMNS, + AuthorizedResource.columns("ctl", "db", "tbl", Set.of("c")).getKind()); + Assertions.assertEquals(ResourceKind.RESOURCE, AuthorizedResource.resource("r").getKind()); + Assertions.assertEquals(ResourceKind.WORKLOAD_GROUP, AuthorizedResource.workloadGroup("wg").getKind()); + Assertions.assertEquals(ResourceKind.STORAGE_VAULT, AuthorizedResource.storageVault("v").getKind()); + Assertions.assertEquals(ResourceKind.CLOUD_STAGE, + AuthorizedResource.cloud(ResourceKind.CLOUD_STAGE, "s").getKind()); + } + + @Test + public void testNameAloneDoesNotIdentifyASystemObject() { + // A workload group and a resource may well share a name; treating them as the same object would + // answer a question about one of them with the policies of the other. + Assertions.assertNotEquals(AuthorizedResource.resource("shared"), + AuthorizedResource.workloadGroup("shared")); + Assertions.assertNotEquals(AuthorizedResource.storageVault("shared"), + AuthorizedResource.cloud(ResourceKind.CLOUD_STORAGE_VAULT, "shared")); + } + + @Test + public void testOnlyCloudKindsAreAcceptedAsCloudObjects() { + for (ResourceKind kind : ResourceKind.values()) { + if (kind.name().startsWith("CLOUD_")) { + Assertions.assertEquals(kind, AuthorizedResource.cloud(kind, "name").getKind()); + } else { + Assertions.assertThrows(IllegalArgumentException.class, + () -> AuthorizedResource.cloud(kind, "name")); + } + } + } + + @Test + public void testColumnsKeepTheOrderTheyWereAskedIn() { + // A refusal names the first column that failed. If the set reordered them, which column the user is + // told about would depend on hashing rather than on the query. + List asked = Arrays.asList("z_col", "a_col", "m_col"); + + AuthorizedResource.Columns columns = + AuthorizedResource.columns("ctl", "db", "tbl", new LinkedHashSet<>(asked)); + + Assertions.assertEquals(asked, new ArrayList<>(columns.getColumns())); + } + + @Test + public void testColumnsCannotBeChangedAfterwards() { + AuthorizedResource.Columns columns = + AuthorizedResource.columns("ctl", "db", "tbl", Set.of("c1")); + + Assertions.assertThrows(UnsupportedOperationException.class, () -> columns.getColumns().add("c2")); + } + + @Test + public void testResourcesAreCompared() { + Assertions.assertEquals(AuthorizedResource.table("c", "d", "t"), AuthorizedResource.table("c", "d", "t")); + Assertions.assertEquals(AuthorizedResource.table("c", "d", "t").hashCode(), + AuthorizedResource.table("c", "d", "t").hashCode()); + Assertions.assertNotEquals(AuthorizedResource.table("c", "d", "t"), + AuthorizedResource.table("c", "d", "other")); + Assertions.assertNotEquals(AuthorizedResource.database("c", "d"), AuthorizedResource.catalog("c")); + Assertions.assertSame(AuthorizedResource.global(), AuthorizedResource.global()); + } + + @Test + public void testMissingNamesAreRefused() { + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedResource.catalog(null)); + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedResource.database("c", null)); + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedResource.table("c", "d", null)); + Assertions.assertThrows(NullPointerException.class, + () -> AuthorizedResource.columns("c", "d", "t", null)); + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedResource.resource(null)); + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index 347b91cf314be5..5fe2450da2a1b9 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -19,11 +19,14 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.catalog.AuthorizationInfo; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.info.TableNameInfo; +import org.apache.doris.common.AuthorizationException; import org.apache.doris.common.Config; import org.apache.doris.common.UserException; import org.apache.doris.common.util.ClassLoaderUtils; @@ -302,13 +305,92 @@ public Auth getAuth() { return this.auth; } + /** + * Answers whether {@code subject} may act on {@code resource} as {@code requirement} demands. + * + *

This is the one place a check is routed. Which controller is asked follows from the resource + * alone - system-wide objects and catalog-level grants go to the controller + * {@code access_controller_type} installs, everything inside a catalog goes to the controller that + * catalog is bound to - and whatever it answers is the answer. Combining two controllers, or granting + * anything before asking, would have to happen here, and deliberately does not. + * + *

Columns are not decided here: see {@link #decideColumns}. + */ + public boolean decide(UserIdentity subject, AuthorizedResource resource, AccessRequirement requirement) { + PrivPredicate wanted = AccessTranslation.privPredicateOf(requirement); + switch (resource.getKind()) { + case GLOBAL: + return systemScopeController().checkGlobalPriv(subject, wanted); + case CATALOG: + // Catalog level grants are only ever stored by the system scope controller, so it answers + // for every catalog, including those bound to a controller of their own. + return systemScopeController().checkCtlPriv(subject, + ((AuthorizedResource.Catalog) resource).getCatalog(), wanted); + case DATABASE: { + AuthorizedResource.Database database = (AuthorizedResource.Database) resource; + return controllerOf(database.getCatalog()) + .checkDbPriv(subject, database.getCatalog(), database.getDatabase(), wanted); + } + case TABLE: { + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + return controllerOf(table.getCatalog()).checkTblPriv(subject, table.getCatalog(), + table.getDatabase(), table.getTable(), wanted); + } + case RESOURCE: + return systemScopeController() + .checkResourcePriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); + case WORKLOAD_GROUP: + return systemScopeController() + .checkWorkloadGroupPriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); + case STORAGE_VAULT: + return systemScopeController() + .checkStorageVaultPriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); + case CLOUD_GENERAL: + case CLOUD_COMPUTE_GROUP: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + return systemScopeController().checkCloudPriv(subject, + ((AuthorizedResource.Named) resource).getName(), wanted, + AccessTranslation.cloudTypeOf(resource.getKind())); + case COLUMNS: + throw new IllegalArgumentException("column access is decided by decideColumns(), which" + + " reports which column was refused instead of a yes or no"); + default: + throw new IllegalStateException("no route for resource kind " + resource.getKind()); + } + } + + /** + * Checks access to named columns, reporting the column that was refused rather than a yes or no. + * + *

Kept apart from {@link #decide} because the answer has a different shape, not because the routing + * differs: it is the same controller the table itself would be asked about. + */ + public void decideColumns(UserIdentity subject, AuthorizedResource.Columns columns, + AccessRequirement requirement) throws AuthorizationException { + controllerOf(columns.getCatalog()).checkColsPriv(subject, columns.getCatalog(), columns.getDatabase(), + columns.getTable(), columns.getColumns(), AccessTranslation.privPredicateOf(requirement)); + } + + /** + * The controller governing everything that is not inside a catalog: global privileges, resources, + * workload groups, cloud objects, storage vaults - and catalog level grants, which only it stores. + */ + private CatalogAccessController systemScopeController() { + return defaultAccessController; + } + + private CatalogAccessController controllerOf(String ctl) { + return getAccessControllerOrDefault(ctl); + } + // ==== Global ==== public boolean checkGlobalPriv(ConnectContext ctx, PrivPredicate wanted) { return checkGlobalPriv(ctx.getCurrentUserIdentity(), wanted); } public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { - return defaultAccessController.checkGlobalPriv(currentUser, wanted); + return decide(currentUser, AuthorizedResource.global(), AccessTranslation.requirementOf(wanted)); } // ==== Catalog ==== @@ -330,22 +412,16 @@ public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate if (catalog == null) { return false; } - if (catalog.isInternalCatalog()) { - return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); + // An external catalog bound to a controller of its own keeps no catalog level grants anywhere, + // so with the check switched off there is nobody left to ask. Every other catalog still goes + // through the normal route below. + String className = catalog.isInternalCatalog() ? "" + : (String) catalog.getProperties().getOrDefault(CatalogMgr.ACCESS_CONTROLLER_CLASS_PROP, ""); + if (!Strings.isNullOrEmpty(className)) { + return true; } - // If catalog not set access controller, use internal access controller - // otherwise, skip catalog priv check - String className = (String) catalog.getProperties().getOrDefault(CatalogMgr.ACCESS_CONTROLLER_CLASS_PROP, - ""); - if (Strings.isNullOrEmpty(className)) { - // not set access controller, use internal access controller - return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); - } - return true; } - // for checking catalog priv, always use InternalAccessController. - // because catalog priv is only saved in InternalAccessController. - return defaultAccessController.checkCtlPriv(currentUser, ctl, wanted); + return decide(currentUser, AuthorizedResource.catalog(ctl), AccessTranslation.requirementOf(wanted)); } // ==== Database ==== @@ -354,7 +430,7 @@ public boolean checkDbPriv(ConnectContext ctx, String ctl, String db, PrivPredic } public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - return getAccessControllerOrDefault(ctl).checkDbPriv(currentUser, ctl, db, wanted); + return decide(currentUser, AuthorizedResource.database(ctl, db), AccessTranslation.requirementOf(wanted)); } // ==== Table ==== @@ -372,7 +448,7 @@ public boolean checkTblPriv(ConnectContext ctx, String qualifiedCtl, } public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - return getAccessControllerOrDefault(ctl).checkTblPriv(currentUser, ctl, db, tbl, wanted); + return decide(currentUser, AuthorizedResource.table(ctl, db, tbl), AccessTranslation.requirementOf(wanted)); } // ==== Column ==== @@ -388,9 +464,9 @@ public void checkColumnsPriv(ConnectContext ctx, String ctl, String qualifiedDb, public void checkColumnsPriv(UserIdentity currentUser, String ctl, String qualifiedDb, String tbl, Set cols, PrivPredicate wanted) throws UserException { - CatalogAccessController accessController = getAccessControllerOrDefault(ctl); long start = System.currentTimeMillis(); - accessController.checkColsPriv(currentUser, ctl, qualifiedDb, tbl, cols, wanted); + decideColumns(currentUser, AuthorizedResource.columns(ctl, qualifiedDb, tbl, cols), + AccessTranslation.requirementOf(wanted)); if (LOG.isDebugEnabled()) { LOG.debug("checkColumnsPriv use {} mills, user: {}, ctl: {}, db: {}, table: {}, cols: {}", System.currentTimeMillis() - start, currentUser, ctl, qualifiedDb, tbl, cols); @@ -403,7 +479,8 @@ public boolean checkResourcePriv(ConnectContext ctx, String resourceName, PrivPr } public boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted) { - return defaultAccessController.checkResourcePriv(currentUser, resourceName, wanted); + return decide(currentUser, AuthorizedResource.resource(resourceName), + AccessTranslation.requirementOf(wanted)); } // ==== Cloud ==== @@ -413,7 +490,8 @@ public boolean checkCloudPriv(ConnectContext ctx, String cloudName, PrivPredicat public boolean checkCloudPriv(UserIdentity currentUser, String cloudName, PrivPredicate wanted, ResourceTypeEnum type) { - return defaultAccessController.checkCloudPriv(currentUser, cloudName, wanted, type); + return decide(currentUser, AuthorizedResource.cloud(AccessTranslation.cloudKindOf(type), cloudName), + AccessTranslation.requirementOf(wanted)); } public boolean checkStorageVaultPriv(ConnectContext ctx, String storageVaultName, PrivPredicate wanted) { @@ -421,7 +499,8 @@ public boolean checkStorageVaultPriv(ConnectContext ctx, String storageVaultName } public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, PrivPredicate wanted) { - return defaultAccessController.checkStorageVaultPriv(currentUser, storageVaultName, wanted); + return decide(currentUser, AuthorizedResource.storageVault(storageVaultName), + AccessTranslation.requirementOf(wanted)); } public boolean checkWorkloadGroupPriv(ConnectContext ctx, String workloadGroupName, PrivPredicate wanted) { @@ -429,7 +508,8 @@ public boolean checkWorkloadGroupPriv(ConnectContext ctx, String workloadGroupNa } public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadGroupName, PrivPredicate wanted) { - return defaultAccessController.checkWorkloadGroupPriv(currentUser, workloadGroupName, wanted); + return decide(currentUser, AuthorizedResource.workloadGroup(workloadGroupName), + AccessTranslation.requirementOf(wanted)); } // ==== Other ==== diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java new file mode 100644 index 00000000000000..214578961e293c --- /dev/null +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java @@ -0,0 +1,210 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.CompoundPredicate.Operator; +import org.apache.doris.analysis.ResourceTypeEnum; +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.ActionMatch; +import org.apache.doris.authorization.ResourceKind; + +import com.google.common.annotations.VisibleForTesting; + +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.util.EnumMap; +import java.util.EnumSet; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; + +/** + * Translates between the vocabulary Doris uses internally and the neutral one authorization sources speak. + * + *

The translation is lossless in both directions and has to stay that way, because the built-in model is + * itself one of those sources: a requirement that reaches it must name exactly the privileges the caller + * asked about. That is why {@link AccessAction} keeps cluster usage and stage usage apart from plain usage + * even though a given plugin may well treat all three alike - the folding is the plugin's to do, not the + * engine's.

+ * + *

Object identity is preserved as well, and that is not a nicety: parts of the engine ask which + * question is being asked by comparing the predicate against a constant with {@code ==} - {@code Role} grants + * "may see this catalog" when any object under it is reachable, the Ranger controllers translate the + * predicate into their own access types the same way. Handing those a freshly built predicate that merely + * equals {@code PrivPredicate.SHOW} loses every one of those branches, silently and only for the users who + * depended on them. So a requirement that came from a constant translates back to that same constant.

+ */ +public final class AccessTranslation { + + private static final Map ACTION_OF_PRIVILEGE = new EnumMap<>(Privilege.class); + private static final Map PRIVILEGE_OF_ACTION = new EnumMap<>(AccessAction.class); + /** + * The predicate constant each requirement came from, discovered reflectively so that a constant added + * later is covered the day it is added rather than the day someone remembers this map. + */ + private static final Map CANONICAL_PREDICATES = new HashMap<>(); + + static { + ACTION_OF_PRIVILEGE.put(Privilege.NODE_PRIV, AccessAction.NODE); + ACTION_OF_PRIVILEGE.put(Privilege.ADMIN_PRIV, AccessAction.ADMIN); + ACTION_OF_PRIVILEGE.put(Privilege.GRANT_PRIV, AccessAction.GRANT); + ACTION_OF_PRIVILEGE.put(Privilege.SELECT_PRIV, AccessAction.SELECT); + ACTION_OF_PRIVILEGE.put(Privilege.LOAD_PRIV, AccessAction.LOAD); + ACTION_OF_PRIVILEGE.put(Privilege.ALTER_PRIV, AccessAction.ALTER); + ACTION_OF_PRIVILEGE.put(Privilege.CREATE_PRIV, AccessAction.CREATE); + ACTION_OF_PRIVILEGE.put(Privilege.DROP_PRIV, AccessAction.DROP); + ACTION_OF_PRIVILEGE.put(Privilege.USAGE_PRIV, AccessAction.USAGE); + ACTION_OF_PRIVILEGE.put(Privilege.CLUSTER_USAGE_PRIV, AccessAction.CLUSTER_USAGE); + ACTION_OF_PRIVILEGE.put(Privilege.STAGE_USAGE_PRIV, AccessAction.STAGE_USAGE); + ACTION_OF_PRIVILEGE.put(Privilege.SHOW_VIEW_PRIV, AccessAction.SHOW_VIEW); + // The retired bit indices carry the same meaning as the ones that replaced them, so they translate + // to the same action - the same normalization Role.upgradeToNewPrivilege() applies to stored grants. + ACTION_OF_PRIVILEGE.put(Privilege.SHOW_VIEW_PRIV_DEPRECATED, AccessAction.SHOW_VIEW); + ACTION_OF_PRIVILEGE.put(Privilege.SHOW_VIEW_PRIV_CLOUD_DEPRECATED, AccessAction.SHOW_VIEW); + ACTION_OF_PRIVILEGE.put(Privilege.CLUSTER_USAGE_PRIV_DEPRECATED, AccessAction.CLUSTER_USAGE); + ACTION_OF_PRIVILEGE.put(Privilege.STAGE_USAGE_PRIV_DEPRECATED, AccessAction.STAGE_USAGE); + + PRIVILEGE_OF_ACTION.put(AccessAction.NODE, Privilege.NODE_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.ADMIN, Privilege.ADMIN_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.GRANT, Privilege.GRANT_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.SELECT, Privilege.SELECT_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.LOAD, Privilege.LOAD_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.ALTER, Privilege.ALTER_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.CREATE, Privilege.CREATE_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.DROP, Privilege.DROP_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.USAGE, Privilege.USAGE_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.CLUSTER_USAGE, Privilege.CLUSTER_USAGE_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.STAGE_USAGE, Privilege.STAGE_USAGE_PRIV); + PRIVILEGE_OF_ACTION.put(AccessAction.SHOW_VIEW, Privilege.SHOW_VIEW_PRIV); + + // Sorted by name so that two constants naming the same privileges with the same match - today + // SHOW_RESOURCES and SHOW_WORKLOAD_GROUP - always resolve to the same one of the pair. They ask the + // same question, so either answers it, but which one it is must not depend on reflection order. + for (Map.Entry constant : new TreeMap<>(declaredPredicates()).entrySet()) { + CANONICAL_PREDICATES.putIfAbsent(requirementOf(constant.getValue()), constant.getValue()); + } + } + + private AccessTranslation() { + } + + private static Map declaredPredicates() { + Map constants = new HashMap<>(); + for (Field field : PrivPredicate.class.getDeclaredFields()) { + if (Modifier.isStatic(field.getModifiers()) && field.getType() == PrivPredicate.class) { + try { + constants.put(field.getName(), (PrivPredicate) field.get(null)); + } catch (IllegalAccessException e) { + throw new IllegalStateException("cannot read PrivPredicate." + field.getName(), e); + } + } + } + return constants; + } + + /** The action {@code privilege} stands for. */ + @VisibleForTesting + static AccessAction actionOf(Privilege privilege) { + AccessAction action = ACTION_OF_PRIVILEGE.get(privilege); + if (action == null) { + throw new IllegalStateException("privilege " + privilege + " has no access action; a new" + + " privilege must be given one before it can be checked"); + } + return action; + } + + /** The privilege {@code action} stands for. */ + @VisibleForTesting + static Privilege privilegeOf(AccessAction action) { + Privilege privilege = PRIVILEGE_OF_ACTION.get(action); + if (privilege == null) { + throw new IllegalStateException("access action " + action + " has no privilege"); + } + return privilege; + } + + /** + * The neutral form of what {@code wanted} asks for. + * + * @throws IllegalArgumentException if the predicate names no privilege at all. Such a predicate is + * satisfied by everyone when combined with AND and by no one when combined with OR, so passing + * one on would decide access by accident; no caller builds one today. + */ + public static AccessRequirement requirementOf(PrivPredicate wanted) { + List privileges = wanted.getPrivs().toPrivilegeList(); + EnumSet actions = EnumSet.noneOf(AccessAction.class); + for (Privilege privilege : privileges) { + actions.add(actionOf(privilege)); + } + if (actions.isEmpty()) { + throw new IllegalArgumentException("privilege predicate names no privilege: " + wanted); + } + return AccessRequirement.of(actions, wanted.getOp() == Operator.AND ? ActionMatch.ALL : ActionMatch.ANY); + } + + /** + * The privilege predicate {@code requirement} stands for; the constant it came from when there is one, + * so that the {@code ==} comparisons the engine still makes against those constants keep working. + */ + public static PrivPredicate privPredicateOf(AccessRequirement requirement) { + PrivPredicate canonical = CANONICAL_PREDICATES.get(requirement); + if (canonical != null) { + return canonical; + } + PrivBitSet privileges = PrivBitSet.of(); + for (AccessAction action : requirement.getActions()) { + privileges.set(privilegeOf(action).getIdx()); + } + return PrivPredicate.of(privileges, + requirement.getMatch() == ActionMatch.ALL ? Operator.AND : Operator.OR); + } + + /** The resource kind standing for a cloud object of {@code type}. */ + public static ResourceKind cloudKindOf(ResourceTypeEnum type) { + switch (type) { + case GENERAL: + return ResourceKind.CLOUD_GENERAL; + case CLUSTER: + return ResourceKind.CLOUD_COMPUTE_GROUP; + case STAGE: + return ResourceKind.CLOUD_STAGE; + case STORAGE_VAULT: + return ResourceKind.CLOUD_STORAGE_VAULT; + default: + throw new IllegalStateException("no resource kind for cloud resource type " + type); + } + } + + /** The cloud resource type {@code kind} stands for; only the cloud kinds have one. */ + public static ResourceTypeEnum cloudTypeOf(ResourceKind kind) { + switch (kind) { + case CLOUD_GENERAL: + return ResourceTypeEnum.GENERAL; + case CLOUD_COMPUTE_GROUP: + return ResourceTypeEnum.CLUSTER; + case CLOUD_STAGE: + return ResourceTypeEnum.STAGE; + case CLOUD_STORAGE_VAULT: + return ResourceTypeEnum.STORAGE_VAULT; + default: + throw new IllegalArgumentException(kind + " is not a cloud resource kind"); + } + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java new file mode 100644 index 00000000000000..ffb950521bac2e --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java @@ -0,0 +1,227 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.CompoundPredicate.Operator; +import org.apache.doris.analysis.ResourceTypeEnum; +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.ActionMatch; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.common.jmockit.Deencapsulation; + +import org.junit.Assert; +import org.junit.Test; + +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.EnumSet; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; + +/** + * The neutral vocabulary must say exactly what Doris's own vocabulary says - no more, no less. + * + *

This matters because the built-in privilege model is one of the sources being asked. A translation that + * merges two privileges into one action, or that turns "all of these" into "any of these", would hand it a + * question different from the one the caller asked and it would answer that different question truthfully. + * The round trip is therefore tested as an identity on the bits, not as "every constant has an entry": most + * predicates are not constants at all - {@code GRANT SELECT ON t TO u} builds one on the spot out of the + * privileges named in the statement, and it is the only place the "all of these" form occurs. + */ +public class AccessTranslationTest { + + /** The privileges a caller can actually ask for; the rest are retired bit indices kept for old metadata. */ + private static final List LIVE_PRIVILEGES = Arrays.asList( + Privilege.NODE_PRIV, Privilege.ADMIN_PRIV, Privilege.GRANT_PRIV, Privilege.SELECT_PRIV, + Privilege.LOAD_PRIV, Privilege.ALTER_PRIV, Privilege.CREATE_PRIV, Privilege.DROP_PRIV, + Privilege.USAGE_PRIV, Privilege.CLUSTER_USAGE_PRIV, Privilege.STAGE_USAGE_PRIV, + Privilege.SHOW_VIEW_PRIV); + + @Test + public void testEveryPrivilegeHasAnAction() { + for (Privilege privilege : Privilege.values()) { + // Retired privileges are included on purpose: a stored grant read back in the mode that owns + // its bit index comes out as one of them, and a missing entry would only fail at run time. + Assert.assertNotNull("privilege " + privilege + " has no access action", + AccessTranslation.actionOf(privilege)); + } + } + + @Test + public void testEveryActionHasAPrivilege() { + for (AccessAction action : AccessAction.values()) { + Assert.assertNotNull("action " + action + " has no privilege", + AccessTranslation.privilegeOf(action)); + } + } + + @Test + public void testEveryPredicateConstantSurvivesTheRoundTrip() { + for (Map.Entry constant : allPrivPredicates().entrySet()) { + assertRoundTrips("PrivPredicate." + constant.getKey(), constant.getValue()); + } + } + + @Test + public void testEveryPredicateConstantTranslatesBackToItself() { + // Value equality is not enough. Role grants "may see this catalog" when any object under it is + // reachable, and the Ranger controllers pick their access type, by comparing the predicate against + // a constant with ==. A translation that returns an equal but different object drops those branches + // for every user who depended on them, which is what the behaviour baseline caught the first time + // this was written. + Map constants = allPrivPredicates(); + Set askTheSameQuestion = constantsSharingAQuestion(constants); + + for (Map.Entry constant : constants.entrySet()) { + PrivPredicate translated = + AccessTranslation.privPredicateOf(AccessTranslation.requirementOf(constant.getValue())); + if (askTheSameQuestion.contains(constant.getKey())) { + // Two constants naming the same privileges with the same match are interchangeable. + Assert.assertEquals(bitsOf(constant.getValue()), bitsOf(translated)); + } else { + Assert.assertSame("PrivPredicate." + constant.getKey() + " must translate back to itself", + constant.getValue(), translated); + } + } + } + + @Test + public void testEveryCombinationOfPrivilegesSurvivesTheRoundTrip() { + int combinations = 1 << LIVE_PRIVILEGES.size(); + for (int mask = 1; mask < combinations; mask++) { + List privileges = new ArrayList<>(); + for (int i = 0; i < LIVE_PRIVILEGES.size(); i++) { + if ((mask & (1 << i)) != 0) { + privileges.add(LIVE_PRIVILEGES.get(i)); + } + } + assertRoundTrips("any of " + privileges, PrivPredicate.of(PrivBitSet.of(privileges), Operator.OR)); + assertRoundTrips("all of " + privileges, PrivPredicate.of(PrivBitSet.of(privileges), Operator.AND)); + } + } + + @Test + public void testGrantStatementPredicateKeepsItsAllSemantics() { + // What GrantTablePrivilegeCommand builds: the privilege being granted plus the right to grant it, + // and the caller must hold both. Read as "any of", holding either one would be enough to hand out + // the other, which is the difference between an authorization model and none. + PrivPredicate wanted = PrivPredicate.of( + PrivBitSet.of(Privilege.SELECT_PRIV, Privilege.GRANT_PRIV), Operator.AND); + + AccessRequirement requirement = AccessTranslation.requirementOf(wanted); + + Assert.assertEquals(ActionMatch.ALL, requirement.getMatch()); + Assert.assertEquals(EnumSet.of(AccessAction.SELECT, AccessAction.GRANT), requirement.getActions()); + Assert.assertEquals(Operator.AND, AccessTranslation.privPredicateOf(requirement).getOp()); + } + + @Test + public void testVisibilityBecomesAnyOfThePrivilegesThatImplyIt() { + // "May this subject see the object" is not a privilege anyone grants; it is a question about + // holding any of several. If it ever translated to a single action, every source would have to + // invent a privilege named SHOW that nobody can be granted. + AccessRequirement requirement = AccessTranslation.requirementOf(PrivPredicate.SHOW); + + Assert.assertEquals(ActionMatch.ANY, requirement.getMatch()); + Assert.assertEquals(EnumSet.of(AccessAction.ADMIN, AccessAction.SELECT, AccessAction.LOAD, + AccessAction.ALTER, AccessAction.CREATE, AccessAction.DROP, AccessAction.SHOW_VIEW), + requirement.getActions()); + } + + @Test + public void testUsageFlavoursStayApart() { + // Ranger folds all three into one access type, and that is Ranger's business. Folding them here + // would leave the built-in model unable to tell which of its three privilege bits was asked about. + AccessRequirement requirement = AccessTranslation.requirementOf(PrivPredicate.USAGE); + + Assert.assertTrue(requirement.getActions().containsAll(EnumSet.of( + AccessAction.USAGE, AccessAction.CLUSTER_USAGE, AccessAction.STAGE_USAGE))); + assertRoundTrips("PrivPredicate.USAGE", PrivPredicate.USAGE); + } + + @Test + public void testPredicateNamingNoPrivilegeIsRefused() { + // Such a predicate is satisfied by everyone under "all of" - nothing to hold - and by no one under + // "any of". Passing it on would decide access by accident, so it fails where it is built instead. + PrivPredicate empty = PrivPredicate.of(PrivBitSet.of(), Operator.AND); + + Assert.assertThrows(IllegalArgumentException.class, () -> AccessTranslation.requirementOf(empty)); + } + + @Test + public void testCloudResourceKindsRoundTrip() { + for (ResourceTypeEnum type : ResourceTypeEnum.values()) { + ResourceKind kind = AccessTranslation.cloudKindOf(type); + Assert.assertEquals("cloud resource type " + type, type, AccessTranslation.cloudTypeOf(kind)); + } + } + + @Test + public void testOnlyCloudKindsHaveACloudResourceType() { + Assert.assertThrows(IllegalArgumentException.class, + () -> AccessTranslation.cloudTypeOf(ResourceKind.TABLE)); + } + + private void assertRoundTrips(String what, PrivPredicate wanted) { + PrivPredicate translated = AccessTranslation.privPredicateOf(AccessTranslation.requirementOf(wanted)); + Assert.assertEquals(what + ": privileges changed", bitsOf(wanted), bitsOf(translated)); + Assert.assertEquals(what + ": match changed", wanted.getOp(), translated.getOp()); + } + + private long bitsOf(PrivPredicate predicate) { + return Deencapsulation.getField(predicate.getPrivs(), "set"); + } + + /** The constants some other constant is indistinguishable from: same privileges, same match. */ + private static Set constantsSharingAQuestion(Map constants) { + Map> byQuestion = new HashMap<>(); + for (Map.Entry constant : constants.entrySet()) { + byQuestion.computeIfAbsent(AccessTranslation.requirementOf(constant.getValue()), + key -> new ArrayList<>()).add(constant.getKey()); + } + Set shared = new HashSet<>(); + for (List names : byQuestion.values()) { + if (names.size() > 1) { + shared.addAll(names); + } + } + return shared; + } + + private static Map allPrivPredicates() { + Map sorted = new TreeMap<>(); + for (Field field : PrivPredicate.class.getDeclaredFields()) { + if (Modifier.isStatic(field.getModifiers()) && field.getType() == PrivPredicate.class) { + try { + sorted.put(field.getName(), (PrivPredicate) field.get(null)); + } catch (IllegalAccessException e) { + throw new IllegalStateException("cannot read PrivPredicate." + field.getName(), e); + } + } + } + Assert.assertFalse("no PrivPredicate constants found", sorted.isEmpty()); + return sorted; + } +} From 8f27f53f85b69c16198faada616c96e5164ec161 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 18:34:02 +0800 Subject: [PATCH 07/22] [feat](authorization) make deciding access a contract a plugin can implement Until now "who decides access" and "how the decision is made" were the same code. The manager routed to a controller, and that controller was an fe-core interface with a method per kind of object, each answering with a boolean. Nothing outside this repository could be that thing without also being part of it. There is a contract now, in fe-authorization-spi, and the built-in privilege model is the first source implementing it. The manager keeps only the routing - a table from resource kind to the source that governs it, and one conversion of a refusal back into the boolean its callers still expect. The switch over nine controller methods now lives inside each source, which is where the knowledge of how that source answers belongs. The contract carries only what the engine asks today: check a requirement, check one action, ask for row filters, ask for column masks, and a lifecycle. Filtering a list of objects, "has any privilege below here", enumerating grants and declaring capabilities are designed and deliberately absent - each has its consumer in a later phase, and the filtering signature in particular should follow from the 45 call sites that will use it rather than from guessing at them. An interface freezes when a release ships it, not when it is written, so growing it before then costs nothing. A refusal is thrown rather than returned. There is no third answer for a source to express and no boolean for a caller to forget, and the reason - which every caller currently invents for itself - has somewhere to live for the first time. The exception records no stack trace and composes its message only if something reads it: listing what a user may see refuses most of the objects that exist, so refusing is the common path here, not the exceptional one. Three things were settled by what the code does rather than by design: - the neutral subject carries the account and nothing else, and that is lossless. The built-in model has to turn it back into a UserIdentity to look up grants and row policies, and UserIdentity's own equality is exactly user, host, and whether the host is a domain; the certificate fields take part in authentication, never in a decision about what an account may do. It reads the account with getUser() rather than getQualifiedUser() - the two return the same field, the second additionally insists the identity went through analysis, and callers exist that check access with one that has not. - roles are not on the subject. Ranger needs them, the built-in model never does, and filtering what a user may see asks thousands of questions per statement, so they are a lazy lookup on AuthorizationContext instead. - controllers written against the older interface keep working, wrapped. That interface is what a catalog's access_controller.class names, so implementations of it exist outside this repository. Wrapping breaks one thing that had to be repaired. The Ranger controllers ask "am I myself the one governing global scope?" by object identity, so as not to ask themselves a question they are about to answer. Reached through a wrapper, that comparison silently stops matching - the behaviour is identical, the same Ranger policies are simply evaluated a second time on every database, table and column check. The manager answers that question now, and sees through the wrapper to do it. Two defects this introduced were caught by the tests written for it, and neither of them is visible to the behaviour baseline: - a refused column check answers by naming the column that failed, and that message leaves the privilege model as an AuthorizationException and reaches the caller as one again. Carrying the rendered form across the middle instead of the bare wording puts that class's error-code prefix in twice, changing what every denied column query prints. - checking access with an identity that never went through analysis used to work. Asking it for its qualified name on every check turned those callers into IllegalStateException. Verified: behaviour baseline unchanged; 428 tests across the 79 classes that touch the decision path, no failures, the four skips pre-existing; 34 tests in the two new modules; checkstyle clean. Mutations: reading "all of these actions" as "any of them" turns three contract tests red; dropping "the host is a domain" from the subject turns the round trip red; routing a storage vault through the resource check turns the adapter's routing test red and moves one baseline line; and making the global-scope identity comparison stop seeing through the wrapper leaves the baseline green while turning red only the test that counts policy evaluations - which is precisely why that test counts them. Co-Authored-By: Claude Opus 5 (1M context) --- .../doris/authorization/AccessContext.java | 57 +++++ .../authorization/AccessDeniedException.java | 108 ++++++++++ .../authorization/AuthorizedSubject.java | 95 +++++++++ .../AccessDeniedExceptionTest.java | 80 +++++++ .../authorization/AuthorizedSubjectTest.java | 58 +++++ .../fe-authorization-spi/pom.xml | 67 ++++++ .../spi/AuthorizationContext.java | 73 +++++++ .../spi/AuthorizationPlugin.java | 151 +++++++++++++ .../spi/AuthorizationPluginFactory.java | 48 +++++ .../spi/AuthorizationPluginContractTest.java | 162 ++++++++++++++ fe/fe-authorization/pom.xml | 6 +- fe/fe-core/pom.xml | 5 + .../ranger/RangerAccessController.java | 7 +- .../hive/RangerHiveAccessController.java | 6 +- .../privilege/AccessControllerManager.java | 200 ++++++++++-------- .../mysql/privilege/AccessTranslation.java | 31 +++ .../privilege/CatalogAccessController.java | 11 +- .../privilege/ConnectionAccessContext.java | 63 ++++++ .../privilege/InternalAccessController.java | 140 ------------ .../InternalAuthorizationPlugin.java | 194 +++++++++++++++++ .../LegacyAccessControllerPlugin.java | 174 +++++++++++++++ .../RangerGlobalScopeDeferenceTest.java | 12 +- .../AccessControlBehaviorBaselineTest.java | 11 +- .../AccessControllerManagerTest.java | 12 +- .../privilege/AccessTranslationTest.java | 47 ++++ .../InternalAccessControllerTest.java | 139 ------------ .../InternalAuthorizationPluginTest.java | 196 +++++++++++++++++ .../LegacyAccessControllerPluginTest.java | 189 +++++++++++++++++ .../privileges/TestCheckPrivileges.java | 3 +- 29 files changed, 1959 insertions(+), 386 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessContext.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessDeniedException.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedSubject.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessDeniedExceptionTest.java create mode 100644 fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedSubjectTest.java create mode 100644 fe/fe-authorization/fe-authorization-spi/pom.xml create mode 100644 fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationContext.java create mode 100644 fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPlugin.java create mode 100644 fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java create mode 100644 fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginContractTest.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/ConnectionAccessContext.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAuthorizationPlugin.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java delete mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAuthorizationPluginTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPluginTest.java diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessContext.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessContext.java new file mode 100644 index 00000000000000..5218977d015988 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessContext.java @@ -0,0 +1,57 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Optional; + +/** + * What surrounds one access decision: the circumstances of the statement asking for it, as opposed to who is + * asking ({@link AuthorizedSubject}) or about what ({@link AuthorizedResource}). + * + *

This is where a decision that depends on more than the subject and the resource - the address the client + * connected from, the time of day, which statement is running - gets its input. A plugin that decides purely + * from grants ignores it; the built-in model does.

+ * + *

Every value is optional because the checks Doris makes do not all originate from a client statement: + * background jobs and internal maintenance ask the same questions with no connection behind them. Absent + * means "not applicable here", never "denied" - a plugin that requires a signal it did not get must say so + * rather than treat the gap as a decision.

+ * + *

An implementation is expected to be lazy: the engine hands one to every check, so producing a query id + * that nobody reads would be paid for on each of them.

+ */ +public interface AccessContext { + + /** No circumstances known - the check does not come from a client statement. */ + AccessContext NONE = new AccessContext() { + @Override + public String toString() { + return "no access context"; + } + }; + + /** The address the client connected from. */ + default Optional getClientIp() { + return Optional.empty(); + } + + /** The id of the query being planned, in the form the engine prints it in logs and profiles. */ + default Optional getQueryId() { + return Optional.empty(); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessDeniedException.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessDeniedException.java new file mode 100644 index 00000000000000..cb8922c23f779e --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessDeniedException.java @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Objects; +import java.util.Optional; + +/** + * Refusal of an access request, carrying why it was refused and who refused it. + * + *

A refusal is an answer, not a failure: deciding that a user may not see a table is the normal outcome of + * asking about a table the user has no grant on, and listing what a user may see asks that question about + * every object there is. So this exception is built for the common case rather than the exceptional one - it + * records no stack trace, and its message is composed only if somebody reads it. What makes it an + * exception at all is that a refusal must not be silently discardable: a plugin that returns without throwing + * has allowed the access, and there is no third answer to forget to handle.

+ * + *

Who refused, and why, is the part that has nowhere else to live. The engine's own callers each phrase + * their own error message today and so lose it; carrying it here is what later lets an error message name the + * plugin that governs the object and lets an audit record say the same.

+ */ +public class AccessDeniedException extends Exception { + + private final AuthorizedSubject subject; + private final AuthorizedResource resource; + private final AccessRequirement requirement; + private final String deniedBy; + private final String explicitMessage; + + private AccessDeniedException(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, String deniedBy, String explicitMessage) { + // No cause, no suppression, and above all no stack trace: filling one in costs more than the whole + // decision that produced it, and this is thrown once per object a user may not see. + super(null, null, false, false); + this.subject = subject; + this.resource = resource; + this.requirement = requirement; + this.deniedBy = deniedBy; + this.explicitMessage = explicitMessage; + } + + /** + * The refusal of {@code requirement} on {@code resource}. + * + * @param deniedBy name of the authorization source that refused, or null when it does not identify itself + */ + public static AccessDeniedException of(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, String deniedBy) { + Objects.requireNonNull(subject, "subject is required"); + Objects.requireNonNull(resource, "resource is required"); + Objects.requireNonNull(requirement, "requirement is required"); + return new AccessDeniedException(subject, resource, requirement, deniedBy, null); + } + + /** + * A refusal that already has its wording, used where the message is itself the answer - a column check + * refuses by naming the column that failed, and rephrasing it here would lose which one it was. + */ + public static AccessDeniedException withMessage(String message, AuthorizedResource resource, + String deniedBy) { + return new AccessDeniedException(null, resource, null, + deniedBy, Objects.requireNonNull(message, "message is required")); + } + + /** What was being accessed. */ + public AuthorizedResource getResource() { + return resource; + } + + /** What was required on it, absent when the refusal came with its own wording. */ + public Optional getRequirement() { + return Optional.ofNullable(requirement); + } + + /** The authorization source that refused, when it identifies itself. */ + public Optional getDeniedBy() { + return Optional.ofNullable(deniedBy); + } + + @Override + public String getMessage() { + if (explicitMessage != null) { + return explicitMessage; + } + StringBuilder message = new StringBuilder("Permission denied: user [").append(subject) + .append("] does not have privilege for [").append(requirement) + .append("] on [").append(resource).append("]"); + if (deniedBy != null) { + message.append(", denied by [").append(deniedBy).append("]"); + } + return message.toString(); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedSubject.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedSubject.java new file mode 100644 index 00000000000000..4f7eb33aac81fd --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AuthorizedSubject.java @@ -0,0 +1,95 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import java.util.Objects; + +/** + * Whose access is being decided. + * + *

A subject is a MySQL style account: a user name together with the host it connects from, which may be + * written either as an address pattern or as a domain name. All three parts identify the account - two + * accounts differing only in host are different accounts with different grants - so all three are carried + * here, and nothing else is: everything the engine knows about a connection beyond its account (the + * certificate it presented, for one) takes no part in deciding access.

+ * + *

Roles are deliberately absent. They belong to {@code AuthorizationContext}, where a plugin asks + * for them and pays for them only if it needs them. Resolving them up front would cost every check the price + * of the one plugin that wants them - the built-in model never does, and filtering what a user may see walks + * thousands of objects per statement.

+ */ +public final class AuthorizedSubject { + + private final String user; + private final String host; + private final boolean domain; + + private AuthorizedSubject(String user, String host, boolean domain) { + this.user = Objects.requireNonNull(user, "user is required"); + this.host = Objects.requireNonNull(host, "host is required"); + this.domain = domain; + } + + /** An account whose host is an address or address pattern. */ + public static AuthorizedSubject of(String user, String host) { + return new AuthorizedSubject(user, host, false); + } + + /** + * @param domain whether {@code host} names a domain to be resolved rather than an address pattern + */ + public static AuthorizedSubject of(String user, String host, boolean domain) { + return new AuthorizedSubject(user, host, domain); + } + + /** The account name, qualified as the engine qualifies it. */ + public String getUser() { + return user; + } + + /** The host part of the account: an address pattern, or a domain name when {@link #isDomain()}. */ + public String getHost() { + return host; + } + + public boolean isDomain() { + return domain; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof AuthorizedSubject)) { + return false; + } + AuthorizedSubject that = (AuthorizedSubject) o; + return domain == that.domain && user.equals(that.user) && host.equals(that.host); + } + + @Override + public int hashCode() { + return Objects.hash(user, host, domain); + } + + @Override + public String toString() { + return "'" + user + "'@'" + host + "'"; + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessDeniedExceptionTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessDeniedExceptionTest.java new file mode 100644 index 00000000000000..465cde13bfdafa --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AccessDeniedExceptionTest.java @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +public class AccessDeniedExceptionTest { + + private static final AuthorizedSubject SUBJECT = AuthorizedSubject.of("someone", "%"); + private static final AuthorizedResource TABLE = AuthorizedResource.table("ctl", "db", "tbl"); + + /** + * Refusal is the ordinary answer, not an error: listing what a user may see asks about every object there + * is and is refused for most of them. Recording a stack trace for each one would cost more than the + * decisions themselves, so this exception must not have one - and nothing in the class can quietly + * reintroduce it. + */ + @Test + public void testARefusalCarriesNoStackTrace() { + AccessDeniedException denied = AccessDeniedException.of(SUBJECT, TABLE, + AccessRequirement.of(AccessAction.SELECT), "somewhere"); + + Assertions.assertEquals(0, denied.getStackTrace().length); + } + + @Test + public void testARefusalSaysWhoWasRefusedWhatAndByWhom() { + AccessDeniedException denied = AccessDeniedException.of(SUBJECT, TABLE, + AccessRequirement.of(AccessAction.SELECT), "ranger-doris"); + + Assertions.assertSame(TABLE, denied.getResource()); + Assertions.assertEquals(AccessRequirement.of(AccessAction.SELECT), denied.getRequirement().orElse(null)); + Assertions.assertEquals("ranger-doris", denied.getDeniedBy().orElse(null)); + String message = denied.getMessage(); + Assertions.assertTrue(message.contains("'someone'@'%'"), message); + Assertions.assertTrue(message.contains("SELECT"), message); + Assertions.assertTrue(message.contains("ctl.db.tbl"), message); + Assertions.assertTrue(message.contains("ranger-doris"), message); + } + + /** + * Some refusals are already worded, and the wording is the answer - a column check refuses by naming the + * column that failed. Recomposing the message from the requirement would reduce "this column" to "these + * columns". + */ + @Test + public void testAWordedRefusalIsReportedAsWritten() { + AccessDeniedException denied = AccessDeniedException.withMessage( + "Permission denied on column [salary]", TABLE, "default"); + + Assertions.assertEquals("Permission denied on column [salary]", denied.getMessage()); + Assertions.assertFalse(denied.getRequirement().isPresent()); + } + + /** A source that does not name itself still produces a usable message. */ + @Test + public void testAnAnonymousRefusalStillReads() { + AccessDeniedException denied = AccessDeniedException.of(SUBJECT, AuthorizedResource.global(), + AccessRequirement.of(AccessAction.ADMIN), null); + + Assertions.assertFalse(denied.getDeniedBy().isPresent()); + Assertions.assertTrue(denied.getMessage().contains("global"), denied.getMessage()); + } +} diff --git a/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedSubjectTest.java b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedSubjectTest.java new file mode 100644 index 00000000000000..12a0b5b4315c92 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/test/java/org/apache/doris/authorization/AuthorizedSubjectTest.java @@ -0,0 +1,58 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +public class AuthorizedSubjectTest { + + /** + * All three parts identify the account. Two accounts differing only in where they connect from are + * different accounts holding different grants, and a subject that compared equal across that difference + * would let a source answer for the wrong one. + */ + @Test + public void testAnAccountIsItsNameItsHostAndHowTheHostIsRead() { + AuthorizedSubject anywhere = AuthorizedSubject.of("user", "%"); + + Assertions.assertEquals(AuthorizedSubject.of("user", "%"), anywhere); + Assertions.assertEquals(AuthorizedSubject.of("user", "%").hashCode(), anywhere.hashCode()); + Assertions.assertNotEquals(AuthorizedSubject.of("other", "%"), anywhere); + Assertions.assertNotEquals(AuthorizedSubject.of("user", "192.168.%"), anywhere); + Assertions.assertNotEquals(AuthorizedSubject.of("user", "%", true), anywhere); + } + + @Test + public void testAHostReadAsADomainSaysSo() { + Assertions.assertFalse(AuthorizedSubject.of("user", "doris.example.com").isDomain()); + Assertions.assertTrue(AuthorizedSubject.of("user", "doris.example.com", true).isDomain()); + } + + /** Printed the way an account is written in Doris, because it ends up in refusal messages. */ + @Test + public void testAnAccountPrintsAsAnAccount() { + Assertions.assertEquals("'user'@'192.168.%'", AuthorizedSubject.of("user", "192.168.%").toString()); + } + + @Test + public void testAnIncompleteAccountIsRefused() { + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedSubject.of(null, "%")); + Assertions.assertThrows(NullPointerException.class, () -> AuthorizedSubject.of("user", null)); + } +} diff --git a/fe/fe-authorization/fe-authorization-spi/pom.xml b/fe/fe-authorization/fe-authorization-spi/pom.xml new file mode 100644 index 00000000000000..ea51ffe4149c15 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/pom.xml @@ -0,0 +1,67 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization + ../pom.xml + + fe-authorization-spi + jar + Doris FE Authorization SPI + + + + org.apache.doris + fe-authorization-api + ${project.version} + + + org.apache.doris + fe-extension-spi + ${project.version} + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + + **/*Test.java + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 17 + 17 + + + + + diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationContext.java b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationContext.java new file mode 100644 index 00000000000000..fd0fde17bfa36d --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationContext.java @@ -0,0 +1,73 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization.spi; + +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; + +import java.util.Optional; +import java.util.Set; + +/** + * What the engine will tell a plugin that asks, so that a plugin can decide things it has no data of its own + * about. + * + *

The engine implements this and each plugin gets its own instance; everything on it is answered on + * demand, so a plugin that decides purely from its own policies never pays for any of it. That is the point + * of the shape: the alternative - handing every check a subject with its roles already resolved - would make + * every check pay for the one plugin that wanted them.

+ */ +public interface AuthorizationContext { + + /** + * The roles {@code subject} holds in Doris. + * + *

For an external source that keeps its own roles this is the bridge between the two: policies written + * against Doris roles can be evaluated because the engine, not the source, knows who holds them.

+ */ + Set rolesOf(AuthorizedSubject subject); + + /** + * Whether {@code subject} already holds {@code requirement} at instance scope - as decided by whoever + * governs instance scope, which may or may not be the plugin asking. + * + *

This is how a plugin honours "an administrator of this instance may reach what I govern" without the + * engine imposing it. Asking is a choice: a plugin that refuses to recognise any authority but its own + * simply never calls this, and one that does call it gets an answer from the same source that would + * answer a global check, so the exemption it grants matches what the instance actually considers + * administrative rather than assuming that is the built-in model.

+ * + *

Answers false when the plugin asking is itself the instance-scope authority: it would otherwise be + * asking itself a question it is about to answer anyway, at the price of evaluating the same policies + * twice.

+ */ + boolean grantedByGlobalScopeAuthority(AuthorizedSubject subject, AccessRequirement requirement); + + /** + * Who owns {@code resource}, when the engine records an owner for it. + * + *

Ownership is metadata, not a privilege: the engine stores it, and what it entitles the owner to is + * each plugin's own rule - all rights over the object, only the right to grant it away, or nothing. + * Empty today for every resource, the channel being here so that adding an owner to a kind of object is a + * change in the engine rather than in this contract.

+ */ + default Optional ownerOf(AuthorizedResource resource) { + return Optional.empty(); + } +} diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPlugin.java b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPlugin.java new file mode 100644 index 00000000000000..444f5f7dc04f03 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPlugin.java @@ -0,0 +1,151 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization.spi; + +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.ActionMatch; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.extension.spi.Plugin; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * An authorization source: whatever decides, for the resources it governs, what a user may do with them. + * + *

Doris ships one of these for its own {@code GRANT} model, and one per external system it integrates + * with. Which plugin is asked follows from the resource alone - the plugin a catalog is bound to answers for + * everything inside that catalog, the plugin installed for the instance answers for everything else - and its + * answer is the whole answer: nothing grants access before it is asked and no second plugin is consulted + * after it. That is what makes the policies in force on an object readable from the configuration, and it is + * why exemptions that used to be the engine's ("an administrator may go anywhere") are each plugin's own to + * grant or refuse, with {@link AuthorizationContext} there to ask the questions such a decision needs.

+ * + *

Refusing

+ * + *

A check that returns has allowed the access; a check that refuses throws {@link + * AccessDeniedException}. There is deliberately no third outcome for a plugin to express and no boolean for a + * caller to ignore. Every check method here defaults to refusing, so a plugin gets access control for the + * questions it did not think about rather than a hole; the two data-policy methods are the exception, and + * their empty default means "this source defines no policy", which is not the same as allowing anything.

+ * + *

What is asked, and how often

+ * + *

The engine asks about a whole requirement - a set of actions plus whether one of them or all of them are + * needed - rather than one action at a time, because sources answer that way: the built-in model tests a bit + * set in one pass, and a policy engine walking a resource hierarchy remembers which actions an outer level + * already granted. A plugin with no such structure implements {@link #checkAction} alone and lets the default + * below take the requirement apart.

+ * + *

These methods are on the path of every statement, several times over: planning one query checks each + * table it reads, and listing what a user may see checks every object that exists. Whatever caching an + * external source needs belongs inside the plugin, where it can be invalidated on the source's own terms; the + * engine adds none of its own and cannot, since it does not know when a policy changed.

+ */ +public interface AuthorizationPlugin extends Plugin { + + /** The name this source is configured and reported under, e.g. {@code "ranger-doris"}. */ + String name(); + + /** + * Refuses unless {@code subject} may act on {@code resource} as {@code requirement} demands. + * + *

The default takes the requirement apart and asks {@link #checkAction} about one action at a time, + * which is correct for any source but costs one evaluation per action. Override when the source can + * answer about a set of actions in one go.

+ * + * @param resource what is being accessed; a plugin should treat a kind it does not recognise as a + * refusal rather than guess, the resource kinds being closed and known at compile time + * @param context circumstances of the statement asking, for a decision that depends on more than the + * subject and the resource; ignorable + * @throws AccessDeniedException if the access is not allowed + */ + default void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + if (requirement.getMatch() == ActionMatch.ALL) { + for (AccessAction action : requirement.getActions()) { + checkAction(subject, resource, action, context); + } + return; + } + AccessDeniedException refused = null; + for (AccessAction action : requirement.getActions()) { + try { + checkAction(subject, resource, action, context); + return; + } catch (AccessDeniedException e) { + refused = e; + } + } + // A requirement always names at least one action, so one refusal was recorded; report that one + // rather than a summary, because it says which action was actually tested last. + throw refused != null ? refused + : AccessDeniedException.of(subject, resource, requirement, name()); + } + + /** + * Refuses unless {@code subject} may perform {@code action} on {@code resource}. + * + *

This is the hook the default {@link #checkPrivilege} takes a requirement apart into, and the engine + * reaches it only that way - it always asks about a whole requirement. So a plugin implements this one + * or {@link #checkPrivilege}, whichever matches how its source answers, and leaving the other at + * its default is not a hole: nothing calls a {@code checkAction} that a plugin answering whole + * requirements never meant to provide.

+ * + * @throws AccessDeniedException if the access is not allowed + */ + default void checkAction(AuthorizedSubject subject, AuthorizedResource resource, AccessAction action, + AccessContext context) throws AccessDeniedException { + throw AccessDeniedException.of(subject, resource, AccessRequirement.of(action), name()); + } + + /** + * The row-level filters this source imposes on {@code table} for {@code subject}, empty when it imposes + * none. + * + *

Each filter is a SQL predicate in Doris dialect; the engine parses it, type-checks it and plans it + * as a filter over the table, combining several of them as each one's merge type says. Returning no + * filter is not a decision about access - whether the table may be read at all was settled by {@link + * #checkPrivilege}.

+ */ + default List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext context) { + return Collections.emptyList(); + } + + /** + * How the named columns of {@code table} must be rewritten before {@code subject} may read them: an entry + * per column that is masked, keyed by the column name as it was asked about, and no entry for a column + * that is not. + * + *

Asked about all the columns at once because a source that answers over the network would otherwise + * be asked once per column of every table in the statement.

+ */ + default Map getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table, + Set columns, AccessContext context) { + return Collections.emptyMap(); + } +} diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java new file mode 100644 index 00000000000000..a14482b4404edd --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java @@ -0,0 +1,48 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization.spi; + +import java.util.Map; + +/** + * Creates an {@link AuthorizationPlugin}. This is what a plugin jar publishes for the engine to discover. + * + *

A plugin is created once and kept: unlike an authentication attempt, an authorization decision happens + * many times within a single statement, and a source that caches policies has to be the same instance + * throughout. The engine builds a new one only when what configures it changes.

+ */ +public interface AuthorizationPluginFactory { + + /** + * The name this source is selected by in configuration, and the name its plugin reports. + * + * @return plugin name, e.g. {@code "ranger-doris"} + */ + String name(); + + /** One line about what this source is, for logs and diagnostics. */ + default String description() { + return ""; + } + + /** + * @param properties configuration for this instance of the source, as configured + * @param context what the engine will answer if this plugin asks; see {@link AuthorizationContext} + */ + AuthorizationPlugin create(Map properties, AuthorizationContext context); +} diff --git a/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginContractTest.java b/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginContractTest.java new file mode 100644 index 00000000000000..588c6e9f713340 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginContractTest.java @@ -0,0 +1,162 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization.spi; + +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.EnumSet; +import java.util.List; +import java.util.Set; + +/** + * What a plugin gets for free, and what it must not get for free. + * + *

Two properties are being held still here. A plugin that says nothing refuses everything, so a question + * an author did not think about - a resource kind added after the plugin was written, say - closes rather + * than opens. And a plugin that answers one action at a time has its requirements taken apart the way the + * requirement itself says: "any of these" is satisfied by one, "all of these" by nothing less than all of + * them. Getting that backwards would not fail loudly anywhere - it would quietly let a {@code GRANT} + * statement through for someone holding only half of what it needs. + */ +public class AuthorizationPluginContractTest { + + private static final AuthorizedSubject SUBJECT = AuthorizedSubject.of("someone", "%"); + private static final AuthorizedResource TABLE = AuthorizedResource.table("ctl", "db", "tbl"); + + /** A source that has been asked nothing and answers nothing. */ + private static class SilentPlugin implements AuthorizationPlugin { + @Override + public String name() { + return "silent"; + } + } + + /** A source that only knows how to answer about one action at a time. */ + private static class PerActionPlugin implements AuthorizationPlugin { + private final Set granted = EnumSet.noneOf(AccessAction.class); + private final List asked = new ArrayList<>(); + + PerActionPlugin(AccessAction... granted) { + Collections.addAll(this.granted, granted); + } + + @Override + public String name() { + return "per-action"; + } + + @Override + public void checkAction(AuthorizedSubject subject, AuthorizedResource resource, AccessAction action, + AccessContext context) throws AccessDeniedException { + asked.add(action); + if (!granted.contains(action)) { + throw AccessDeniedException.of(subject, resource, AccessRequirement.of(action), name()); + } + } + } + + private static boolean allows(AuthorizationPlugin plugin, AccessRequirement requirement) { + try { + plugin.checkPrivilege(SUBJECT, TABLE, requirement, AccessContext.NONE); + return true; + } catch (AccessDeniedException e) { + return false; + } + } + + @Test + public void testAPluginThatSaysNothingRefusesEverything() { + SilentPlugin plugin = new SilentPlugin(); + + Assertions.assertFalse(allows(plugin, AccessRequirement.of(AccessAction.SELECT))); + Assertions.assertFalse(allows(plugin, AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.LOAD))); + Assertions.assertFalse(allows(plugin, AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT))); + Assertions.assertThrows(AccessDeniedException.class, () -> plugin.checkAction(SUBJECT, TABLE, + AccessAction.SELECT, AccessContext.NONE)); + } + + /** + * Refusing is about access, not about policy: a source with no row filter to impose has not thereby + * refused the table, and returning "denied" here would hide every row of every table from everyone. + */ + @Test + public void testSayingNothingMeansNoDataPolicyRatherThanRefusal() { + SilentPlugin plugin = new SilentPlugin(); + + Assertions.assertTrue(plugin.getRowFilters(SUBJECT, AuthorizedResource.table("ctl", "db", "tbl"), + AccessContext.NONE).isEmpty()); + Assertions.assertTrue(plugin.getDataMasks(SUBJECT, AuthorizedResource.table("ctl", "db", "tbl"), + Collections.singleton("col"), AccessContext.NONE).isEmpty()); + } + + @Test + public void testAnyIsSatisfiedByOneActionOfTheSet() { + Assertions.assertTrue(allows(new PerActionPlugin(AccessAction.LOAD), + AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.LOAD))); + Assertions.assertFalse(allows(new PerActionPlugin(AccessAction.DROP), + AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.LOAD))); + } + + /** + * The shape of the check a {@code GRANT} statement makes - the grantor must hold what is being granted + * and the right to grant it - so reading this as "any" would hand out privileges to someone + * holding only one of the two. + */ + @Test + public void testAllNeedsEveryActionOfTheSet() { + Assertions.assertTrue(allows(new PerActionPlugin(AccessAction.SELECT, AccessAction.GRANT), + AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT))); + Assertions.assertFalse(allows(new PerActionPlugin(AccessAction.SELECT), + AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT))); + Assertions.assertFalse(allows(new PerActionPlugin(AccessAction.GRANT), + AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT))); + } + + /** + * "Any of these" stops at the first action that is granted. A source reached over the network pays per + * question asked, and the ones a satisfied requirement no longer needs are the easiest to stop asking. + */ + @Test + public void testAnyStopsAtTheFirstActionThatIsGranted() { + PerActionPlugin plugin = new PerActionPlugin(AccessAction.SELECT); + + Assertions.assertTrue(allows(plugin, AccessRequirement.anyOf(AccessAction.SELECT, AccessAction.LOAD))); + Assertions.assertEquals(Collections.singletonList(AccessAction.SELECT), plugin.asked); + } + + /** A refusal names the source that produced it, which is the whole point of carrying it as an object. */ + @Test + public void testARefusalSaysWhoRefused() { + AccessDeniedException denied = Assertions.assertThrows(AccessDeniedException.class, + () -> new SilentPlugin().checkPrivilege(SUBJECT, TABLE, + AccessRequirement.of(AccessAction.SELECT), AccessContext.NONE)); + + Assertions.assertEquals("silent", denied.getDeniedBy().orElse(null)); + Assertions.assertSame(TABLE, denied.getResource()); + } +} diff --git a/fe/fe-authorization/pom.xml b/fe/fe-authorization/pom.xml index 138ee4ba32aed5..c627ee7aa88a26 100644 --- a/fe/fe-authorization/pom.xml +++ b/fe/fe-authorization/pom.xml @@ -32,10 +32,12 @@ under the License. fe-authorization-api + fe-authorization-spi diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index e7825f36633a50..0585cbf0d18e45 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -395,6 +395,11 @@ under the License. fe-authorization-api ${project.version} + + ${project.groupId} + fe-authorization-spi + ${project.version} + org.springframework.boot spring-boot-devtools diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java index 10bb4a1137c657..9ba6b19c44ac07 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java @@ -23,7 +23,7 @@ import org.apache.doris.catalog.Env; import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; import org.apache.doris.common.AuthorizationException; -import org.apache.doris.datasource.InternalCatalog; +import org.apache.doris.mysql.privilege.AccessControllerManager; import org.apache.doris.mysql.privilege.CatalogAccessController; import org.apache.doris.mysql.privilege.PrivPredicate; @@ -60,9 +60,8 @@ public abstract class RangerAccessController implements CatalogAccessController * own global check answers the same question one line later. */ protected boolean grantedByGlobalScopeAuthority(UserIdentity currentUser, PrivPredicate wanted) { - CatalogAccessController authority = Env.getCurrentEnv().getAccessManager() - .getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME); - return authority != this && authority.checkGlobalPriv(currentUser, wanted); + AccessControllerManager manager = Env.getCurrentEnv().getAccessManager(); + return !manager.isGlobalScopeAuthority(this) && manager.checkGlobalPriv(currentUser, wanted); } protected static boolean checkRequestResult(RangerAccessRequestImpl request, diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java index 3f2973ba678ad4..e42464d3f56e5c 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java @@ -25,7 +25,6 @@ import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; import org.apache.doris.common.AuthorizationException; import org.apache.doris.common.ThreadPoolManager; -import org.apache.doris.datasource.InternalCatalog; import org.apache.doris.mysql.privilege.PrivPredicate; import com.google.common.collect.Maps; @@ -198,9 +197,8 @@ private HiveAccessType convertToAccessType(PrivPredicate predicate) { @Override public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { // hive ranger plugin does not support global privilege - // use internal access controller to check - return Env.getCurrentEnv().getAccessManager().getAccessControllerOrDefault( - InternalCatalog.INTERNAL_CATALOG_NAME).checkGlobalPriv(currentUser, wanted); + // use whichever authorization source governs global scope + return Env.getCurrentEnv().getAccessManager().checkGlobalPriv(currentUser, wanted); } @Override diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index 5fe2450da2a1b9..b521f4180b87af 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -19,10 +19,13 @@ import org.apache.doris.analysis.ResourceTypeEnum; import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessDeniedException; import org.apache.doris.authorization.AccessRequirement; import org.apache.doris.authorization.AuthorizedResource; import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.ResourceKind; import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.AuthorizationInfo; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.info.TableNameInfo; @@ -45,6 +48,7 @@ import org.apache.logging.log4j.Logger; import java.io.IOException; +import java.util.Collections; import java.util.List; import java.util.Map; import java.util.Objects; @@ -55,24 +59,22 @@ /** * AccessControllerManager is the entry point of privilege authentication. - * There are 2 kinds of access controller: - * SystemAccessController: for global level priv, resource priv and other Doris internal priv checking - * CatalogAccessController: for specified catalog's priv checking, can be customized. - * And using InternalCatalogAccessController as default. * - *

It routes and nothing more: each check goes to the single controller that governs the resource, and that - * controller's answer is the answer. The manager establishes no privilege of its own beforehand and never - * combines two controllers' verdicts, so which policies apply to a resource is readable from which controller - * the catalog is bound to. + *

Access is decided by authorization sources - the built-in privilege model, or a plugin standing for an + * external system - and this class decides only which one to ask: system-wide objects and catalog level + * grants go to the source {@code access_controller_type} installs, everything inside a catalog goes to the + * source that catalog is bound to. Whatever that source answers is the answer. The manager establishes no + * privilege of its own beforehand and never combines two sources' verdicts, so which policies apply to a + * resource is readable from which source the catalog is bound to. */ public class AccessControllerManager { private static final Logger LOG = LogManager.getLogger(AccessControllerManager.class); private Auth auth; - // Default access controller instance used for handling cases where no specific controller is specified - private CatalogAccessController defaultAccessController; - // A catalog name can be reused after DROP. Keep the catalog id next to the controller so cleanup from - // an old catalog generation can never remove or close the replacement generation's controller. + // Governs everything no catalog-bound source governs; the built-in model unless configured otherwise + private AuthorizationPlugin defaultAccessController; + // A catalog name can be reused after DROP. Keep the catalog id next to the source so cleanup from + // an old catalog generation can never remove or close the replacement generation's source. private Map ctlToCtlAccessController = Maps.newConcurrentMap(); // Cache of loaded access controller factories for quick creation of new access controllers private ConcurrentHashMap accessControllerFactoriesCache @@ -92,22 +94,22 @@ public AccessControllerManager(Auth auth) { private static final class CatalogAccessControllerEntry { private final long catalogId; - private final CatalogAccessController accessController; - // The default controller is shared with the internal catalog. Catalog aliases must detach it but never + private final AuthorizationPlugin accessController; + // The default source is shared with the internal catalog. Catalog aliases must detach it but never // close it when an external catalog is reset or dropped. private final boolean owned; private CatalogAccessControllerEntry( - long catalogId, CatalogAccessController accessController, boolean owned) { + long catalogId, AuthorizationPlugin accessController, boolean owned) { this.catalogId = catalogId; this.accessController = accessController; this.owned = owned; } } - private CatalogAccessController loadAccessControllerOrThrow(String accessControllerName) { - if (accessControllerName.equalsIgnoreCase("default")) { - return new InternalAccessController(auth); + private AuthorizationPlugin loadAccessControllerOrThrow(String accessControllerName) { + if (accessControllerName.equalsIgnoreCase(InternalAuthorizationPlugin.NAME)) { + return new InternalAuthorizationPlugin(auth); } if (accessControllerFactoriesCache.containsKey(accessControllerName)) { Map prop; @@ -117,12 +119,18 @@ private CatalogAccessController loadAccessControllerOrThrow(String accessControl throw new RuntimeException("Failed to load authorization properties." + "Please check the configuration file, authorization name is " + accessControllerName, e); } - return accessControllerFactoriesCache.get(accessControllerName).createAccessController(prop); + return adapt(accessControllerName, + accessControllerFactoriesCache.get(accessControllerName).createAccessController(prop)); } throw new RuntimeException("No authorization plugin factory found for " + accessControllerName + ". Please confirm that your plugin is placed in the correct location."); } + /** Presents a controller written against the older per-scope interface as an authorization source. */ + private AuthorizationPlugin adapt(String name, CatalogAccessController controller) { + return new LegacyAccessControllerPlugin(name, controller); + } + private void loadAccessControllerPlugins() { ServiceLoader loaderFromClasspath = ServiceLoader.load(AccessControllerFactory.class); for (AccessControllerFactory factory : loaderFromClasspath) { @@ -143,7 +151,8 @@ private void loadAccessControllerPlugins() { } } - public CatalogAccessController getAccessControllerOrDefault(String ctl) { + /** The authorization source governing the objects inside {@code ctl}. */ + public AuthorizationPlugin getAccessControllerOrDefault(String ctl) { if (InternalCatalog.INTERNAL_CATALOG_NAME.equals(ctl)) { return defaultAccessController; } @@ -214,8 +223,8 @@ public boolean checkIfAccessControllerExist(String ctl) { public void createAccessController(ExternalCatalog catalog, String acFactoryClassName, Map prop, boolean isDryRun) { String pluginIdentifier = getPluginIdentifierForAccessController(acFactoryClassName); - CatalogAccessController accessController = accessControllerFactoriesCache.get(pluginIdentifier) - .createAccessController(prop); + AuthorizationPlugin accessController = adapt(pluginIdentifier, + accessControllerFactoriesCache.get(pluginIdentifier).createAccessController(prop)); if (isDryRun) { closeAccessController(catalog.getName(), accessController); return; @@ -291,7 +300,7 @@ private void closeEntry(String ctl, CatalogAccessControllerEntry entry) { closeAccessController(ctl, entry.accessController); } - private void closeAccessController(String ctl, CatalogAccessController accessController) { + private void closeAccessController(String ctl, AuthorizationPlugin accessController) { try { accessController.close(); } catch (Throwable e) { @@ -308,80 +317,105 @@ public Auth getAuth() { /** * Answers whether {@code subject} may act on {@code resource} as {@code requirement} demands. * - *

This is the one place a check is routed. Which controller is asked follows from the resource - * alone - system-wide objects and catalog-level grants go to the controller - * {@code access_controller_type} installs, everything inside a catalog goes to the controller that - * catalog is bound to - and whatever it answers is the answer. Combining two controllers, or granting - * anything before asking, would have to happen here, and deliberately does not. + *

This is the one place a check is routed. Which source is asked follows from the resource alone - + * system-wide objects and catalog-level grants go to the source {@code access_controller_type} + * installs, everything inside a catalog goes to the source that catalog is bound to - and whatever it + * answers is the answer. Combining two sources, or granting anything before asking, would have to + * happen here, and deliberately does not. * *

Columns are not decided here: see {@link #decideColumns}. */ public boolean decide(UserIdentity subject, AuthorizedResource resource, AccessRequirement requirement) { - PrivPredicate wanted = AccessTranslation.privPredicateOf(requirement); + if (resource.getKind() == ResourceKind.COLUMNS) { + throw new IllegalArgumentException("column access is decided by decideColumns(), which" + + " reports which column was refused instead of a yes or no"); + } + try { + ask(subject, resource, requirement); + return true; + } catch (AccessDeniedException e) { + // The reason travels no further for now: every caller of the boolean facades phrases its own + // error message. It is carried this far so that the day one of them stops doing so, there is + // something to phrase it from. + return false; + } + } + + /** + * Checks access to named columns, reporting the column that was refused rather than a yes or no. + * + *

Kept apart from {@link #decide} because the answer has a different shape, not because the routing + * differs: it is the same source the table itself would be asked about. + */ + public void decideColumns(UserIdentity subject, AuthorizedResource.Columns columns, + AccessRequirement requirement) throws AuthorizationException { + try { + ask(subject, columns, requirement); + } catch (AccessDeniedException e) { + throw new AuthorizationException(e.getMessage()); + } + } + + private void ask(UserIdentity subject, AuthorizedResource resource, AccessRequirement requirement) + throws AccessDeniedException { + controllerOf(resource).checkPrivilege(AccessTranslation.subjectOf(subject), resource, requirement, + ConnectionAccessContext.current()); + } + + /** + * The authorization source that answers for {@code resource}. This is the whole of the routing: system + * wide objects and catalog level grants belong to the source installed for the instance, everything + * inside a catalog to the source that catalog is bound to. + */ + private AuthorizationPlugin controllerOf(AuthorizedResource resource) { switch (resource.getKind()) { case GLOBAL: - return systemScopeController().checkGlobalPriv(subject, wanted); - case CATALOG: - // Catalog level grants are only ever stored by the system scope controller, so it answers - // for every catalog, including those bound to a controller of their own. - return systemScopeController().checkCtlPriv(subject, - ((AuthorizedResource.Catalog) resource).getCatalog(), wanted); - case DATABASE: { - AuthorizedResource.Database database = (AuthorizedResource.Database) resource; - return controllerOf(database.getCatalog()) - .checkDbPriv(subject, database.getCatalog(), database.getDatabase(), wanted); - } - case TABLE: { - AuthorizedResource.Table table = (AuthorizedResource.Table) resource; - return controllerOf(table.getCatalog()).checkTblPriv(subject, table.getCatalog(), - table.getDatabase(), table.getTable(), wanted); - } case RESOURCE: - return systemScopeController() - .checkResourcePriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); case WORKLOAD_GROUP: - return systemScopeController() - .checkWorkloadGroupPriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); case STORAGE_VAULT: - return systemScopeController() - .checkStorageVaultPriv(subject, ((AuthorizedResource.Named) resource).getName(), wanted); case CLOUD_GENERAL: case CLOUD_COMPUTE_GROUP: case CLOUD_STAGE: case CLOUD_STORAGE_VAULT: - return systemScopeController().checkCloudPriv(subject, - ((AuthorizedResource.Named) resource).getName(), wanted, - AccessTranslation.cloudTypeOf(resource.getKind())); + return systemScopeController(); + case CATALOG: + // Catalog level grants are only ever stored by the system scope source, so it answers for + // every catalog, including those bound to a source of their own. + return systemScopeController(); + case DATABASE: + return getAccessControllerOrDefault(((AuthorizedResource.Database) resource).getCatalog()); + case TABLE: + return getAccessControllerOrDefault(((AuthorizedResource.Table) resource).getCatalog()); case COLUMNS: - throw new IllegalArgumentException("column access is decided by decideColumns(), which" - + " reports which column was refused instead of a yes or no"); + return getAccessControllerOrDefault(((AuthorizedResource.Columns) resource).getCatalog()); default: throw new IllegalStateException("no route for resource kind " + resource.getKind()); } } /** - * Checks access to named columns, reporting the column that was refused rather than a yes or no. - * - *

Kept apart from {@link #decide} because the answer has a different shape, not because the routing - * differs: it is the same controller the table itself would be asked about. - */ - public void decideColumns(UserIdentity subject, AuthorizedResource.Columns columns, - AccessRequirement requirement) throws AuthorizationException { - controllerOf(columns.getCatalog()).checkColsPriv(subject, columns.getCatalog(), columns.getDatabase(), - columns.getTable(), columns.getColumns(), AccessTranslation.privPredicateOf(requirement)); - } - - /** - * The controller governing everything that is not inside a catalog: global privileges, resources, + * The source governing everything that is not inside a catalog: global privileges, resources, * workload groups, cloud objects, storage vaults - and catalog level grants, which only it stores. */ - private CatalogAccessController systemScopeController() { + private AuthorizationPlugin systemScopeController() { return defaultAccessController; } - private CatalogAccessController controllerOf(String ctl) { - return getAccessControllerOrDefault(ctl); + /** + * Whether {@code candidate} is itself the source governing instance scope. + * + *

Asked by a source that would otherwise defer to that authority, so that it does not ask itself a + * question it is about to answer - the two would agree, at the price of evaluating the same policies + * twice. Identity is the question, so a controller reached through an adapter is compared against the + * controller, not against its wrapper. + */ + public boolean isGlobalScopeAuthority(Object candidate) { + AuthorizationPlugin authority = systemScopeController(); + if (authority == candidate) { + return true; + } + return authority instanceof LegacyAccessControllerPlugin + && ((LegacyAccessControllerPlugin) authority).getController() == candidate; } // ==== Global ==== @@ -532,15 +566,6 @@ public boolean checkPrivByAuthInfo(ConnectContext ctx, AuthorizationInfo authInf return true; } - public Map> evalDataMaskPolicies(UserIdentity currentUser, String - ctl, String db, String tbl, Set cols) { - Map> res = Maps.newHashMap(); - for (String col : cols) { - res.put(col, evalDataMaskPolicy(currentUser, ctl, db, tbl, col)); - } - return res; - } - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, String col) { Objects.requireNonNull(currentUser, "require currentUser object"); @@ -548,7 +573,14 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Strin Objects.requireNonNull(db, "require db object"); Objects.requireNonNull(tbl, "require tbl object"); Objects.requireNonNull(col, "require col object"); - return getAccessControllerOrDefault(ctl).evalDataMaskPolicy(currentUser, ctl, db, tbl, col.toLowerCase()); + // Sources are asked about columns in lower case, which is how the ones that store policies per + // column have them written. + String column = col.toLowerCase(); + AuthorizedResource.Table table = AuthorizedResource.table(ctl, db, tbl); + return Optional.ofNullable(controllerOf(table) + .getDataMasks(AccessTranslation.subjectOf(currentUser), table, + Collections.singleton(column), ConnectionAccessContext.current()) + .get(column)); } public List evalRowFilterPolicies(UserIdentity currentUser, String @@ -557,6 +589,8 @@ public List evalRowFilterPolicies(UserIdentity currentUser, Strin Objects.requireNonNull(ctl, "require ctl object"); Objects.requireNonNull(db, "require db object"); Objects.requireNonNull(tbl, "require tbl object"); - return getAccessControllerOrDefault(ctl).evalRowFilterPolicies(currentUser, ctl, db, tbl); + AuthorizedResource.Table table = AuthorizedResource.table(ctl, db, tbl); + return controllerOf(table).getRowFilters(AccessTranslation.subjectOf(currentUser), table, + ConnectionAccessContext.current()); } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java index 214578961e293c..0dd37b59dcafb7 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessTranslation.java @@ -19,9 +19,11 @@ import org.apache.doris.analysis.CompoundPredicate.Operator; import org.apache.doris.analysis.ResourceTypeEnum; +import org.apache.doris.analysis.UserIdentity; import org.apache.doris.authorization.AccessAction; import org.apache.doris.authorization.AccessRequirement; import org.apache.doris.authorization.ActionMatch; +import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.ResourceKind; import com.google.common.annotations.VisibleForTesting; @@ -176,6 +178,35 @@ public static PrivPredicate privPredicateOf(AccessRequirement requirement) { requirement.getMatch() == ActionMatch.ALL ? Operator.AND : Operator.OR); } + /** + * The neutral form of the account {@code user} names. + * + *

Only the three parts that identify an account are carried over, which is exactly what makes the + * translation reversible: {@link UserIdentity#equals} compares those three and nothing else, and every + * lookup an authorization decision performs - the privilege tables, the row policies - matches on them. + * The certificate fields a connection may also carry take part in authentication, never in a decision + * about what an account may do, so leaving them behind loses nothing. + * + *

Read with {@code getUser()} rather than {@code getQualifiedUser()}. The two return the same field; + * the second additionally insists the identity has been through analysis, and callers exist that check + * access with one that has not - {@code checkCloudPriv} is reached that way today. Translating is not the + * place to start enforcing that, since it happens on every check and would turn callers that work into + * callers that throw. + */ + public static AuthorizedSubject subjectOf(UserIdentity user) { + return AuthorizedSubject.of(user.getUser(), user.getHost(), user.isDomain()); + } + + /** + * The account {@code subject} names, in the form the built-in privilege model looks accounts up by. + * Equal to the identity it was translated from; see {@link #subjectOf}. + */ + public static UserIdentity userIdentityOf(AuthorizedSubject subject) { + return subject.isDomain() + ? UserIdentity.createAnalyzedUserIdentWithDomain(subject.getUser(), subject.getHost()) + : UserIdentity.createAnalyzedUserIdentWithIp(subject.getUser(), subject.getHost()); + } + /** The resource kind standing for a cloud object of {@code type}. */ public static ResourceKind cloudKindOf(ResourceTypeEnum type) { switch (type) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java index 00ce013746e0ca..13a17be97a15fd 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java @@ -28,14 +28,19 @@ import java.util.Set; /** - * Decides access to the resources of one catalog. + * Decides access to the resources of one catalog, one kind of object at a time. * *

A controller is asked only about the resources it governs, and its answer is final: nothing outside it * grants first. In particular the engine no longer establishes a global privilege before routing, so an * implementation that wants "holding the privilege globally is enough" has to say so itself - see - * {@link InternalAccessController}, which checks global privileges ahead of the fine grained ones, and * {@link org.apache.doris.catalog.authorizer.ranger.RangerAccessController}, which defers to whichever - * controller owns global scope. + * source owns global scope. + * + *

This is the older shape of that contract, kept because a catalog's {@code access_controller.class} + * names an implementation of it and such implementations exist outside this repository. The engine reaches + * one through {@link LegacyAccessControllerPlugin}; a source written today implements + * {@link org.apache.doris.authorization.spi.AuthorizationPlugin} instead, which asks a single question about + * a typed resource and answers by refusing rather than by returning false. */ public interface CatalogAccessController { default void close() { diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/ConnectionAccessContext.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/ConnectionAccessContext.java new file mode 100644 index 00000000000000..9b8317c5f1c7e3 --- /dev/null +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/ConnectionAccessContext.java @@ -0,0 +1,63 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.common.util.DebugUtil; +import org.apache.doris.qe.ConnectContext; +import org.apache.doris.thrift.TUniqueId; + +import com.google.common.base.Strings; + +import java.util.Optional; + +/** + * The circumstances of a check, read from the connection the statement is running on. + * + *

Nothing is read until asked for. A plugin that decides from grants alone asks for none of it, and there + * are enough checks per statement - one per object a statement touches, and every object there is when a + * statement lists what a user may see - that formatting a query id nobody reads would be a real cost. + */ +class ConnectionAccessContext implements AccessContext { + + private final ConnectContext connection; + + private ConnectionAccessContext(ConnectContext connection) { + this.connection = connection; + } + + /** + * The circumstances of the statement running on this thread, or {@link AccessContext#NONE} when the check + * comes from somewhere other than a client statement - a background job, or replaying an edit log. + */ + static AccessContext current() { + ConnectContext connection = ConnectContext.get(); + return connection == null ? AccessContext.NONE : new ConnectionAccessContext(connection); + } + + @Override + public Optional getClientIp() { + return Optional.ofNullable(Strings.emptyToNull(connection.getRemoteIP())); + } + + @Override + public Optional getQueryId() { + TUniqueId queryId = connection.queryId(); + return queryId == null ? Optional.empty() : Optional.of(DebugUtil.printId(queryId)); + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java deleted file mode 100644 index 333cb8e6983fb3..00000000000000 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAccessController.java +++ /dev/null @@ -1,140 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.analysis.ResourceTypeEnum; -import org.apache.doris.analysis.UserIdentity; -import org.apache.doris.authorization.DataMaskSpec; -import org.apache.doris.authorization.RowFilterMergeType; -import org.apache.doris.authorization.RowFilterSpec; -import org.apache.doris.catalog.Env; -import org.apache.doris.common.AnalysisException; -import org.apache.doris.common.AuthorizationException; -import org.apache.doris.policy.FilterType; -import org.apache.doris.policy.RowPolicy; - -import com.google.common.collect.ImmutableList; - -import java.util.List; -import java.util.Optional; -import java.util.Set; - -/** - * The privilege model Doris ships with: users, roles and {@code GRANT} statements. - * - *

Every scoped check answers "globally, then at this scope". That order is this implementation's own - * decision, not something the engine arranges for it, and it is not redundant with the checks {@link Auth} - * runs per role: {@link Auth} refuses NODE privileges below global level, so a caller holding only global - * NODE_PRIV is granted by the global check here and by nothing else. - */ -public class InternalAccessController implements CatalogAccessController { - private Auth auth; - - public InternalAccessController(Auth auth) { - this.auth = auth; - } - - @Override - public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { - return auth.checkGlobalPriv(currentUser, wanted); - } - - @Override - public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - return checkGlobalPriv(currentUser, wanted) || auth.checkCtlPriv(currentUser, ctl, wanted); - } - - @Override - public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - return checkGlobalPriv(currentUser, wanted) || auth.checkDbPriv(currentUser, ctl, db, wanted); - } - - @Override - public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - return checkGlobalPriv(currentUser, wanted) || auth.checkTblPriv(currentUser, ctl, db, tbl, wanted); - } - - @Override - public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, - PrivPredicate wanted) throws AuthorizationException { - if (checkGlobalPriv(currentUser, wanted)) { - return; - } - auth.checkColsPriv(currentUser, ctl, db, tbl, cols, wanted); - } - - @Override - public boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted) { - return auth.checkResourcePriv(currentUser, resourceName, wanted); - } - - @Override - public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadGroupName, PrivPredicate wanted) { - return auth.checkWorkloadGroupPriv(currentUser, workloadGroupName, wanted); - } - - @Override - public boolean checkCloudPriv(UserIdentity currentUser, String cloudName, - PrivPredicate wanted, ResourceTypeEnum type) { - return auth.checkCloudPriv(currentUser, cloudName, wanted, type); - } - - @Override - public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, PrivPredicate wanted) { - return auth.checkStorageVaultPriv(currentUser, storageVaultName, wanted); - } - - @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, - String col) { - // The built-in privilege model has no column masking: there is no DDL to define one. - return Optional.empty(); - } - - @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, - String tbl) { - List policies = Env.getCurrentEnv().getPolicyMgr().getUserPolicies(ctl, db, tbl, currentUser); - ImmutableList.Builder specs = ImmutableList.builderWithExpectedSize(policies.size()); - for (RowPolicy policy : policies) { - try { - specs.add(new RowFilterSpec(policy.getPolicyIdent(), policy.getFilterSql(), - mergeTypeOf(policy.getFilterType()))); - } catch (AnalysisException e) { - // A policy whose statement no longer parses cannot be turned into a filter, and dropping it - // would silently widen access to the whole table. Fail the query with the same message the - // planner used to raise when it asked the policy for its expression. - throw new org.apache.doris.nereids.exceptions.AnalysisException(e.getMessage(), e); - } - } - return specs.build(); - } - - private static RowFilterMergeType mergeTypeOf(FilterType filterType) { - switch (filterType) { - case PERMISSIVE: - return RowFilterMergeType.PERMISSIVE; - case RESTRICTIVE: - return RowFilterMergeType.RESTRICTIVE; - default: - // Same shape as the planner's merge switch used to have: an unmapped filter type is a bug, - // and guessing either way would change which rows the user sees. - throw new IllegalStateException("Invalid operator"); - } - } -} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAuthorizationPlugin.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAuthorizationPlugin.java new file mode 100644 index 00000000000000..a1ecc315828ee0 --- /dev/null +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/InternalAuthorizationPlugin.java @@ -0,0 +1,194 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterMergeType; +import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.catalog.Env; +import org.apache.doris.common.AnalysisException; +import org.apache.doris.common.AuthorizationException; +import org.apache.doris.policy.FilterType; +import org.apache.doris.policy.RowPolicy; + +import com.google.common.collect.ImmutableList; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * The privilege model Doris ships with: users, roles and {@code GRANT} statements. + * + *

It is an authorization source like any other and is reached the same way, which is what keeps the + * built-in behaviour describable in the same terms as an external system's: it governs whatever no plugin has + * been bound to, and what it decides there it decides alone. + * + *

Every scoped check answers "globally, then at this scope". That order is this implementation's own + * decision, not something the engine arranges for it, and it is not redundant with the checks {@link Auth} + * runs per role: {@link Auth} refuses NODE privileges below global level, so a caller holding only global + * NODE_PRIV is granted by the global check here and by nothing else. The system-wide objects below - a + * resource, a workload group, a vault, a cloud object - are deliberately not preceded by one: {@link Auth} + * already folds the global grants into those answers itself. + */ +public class InternalAuthorizationPlugin implements AuthorizationPlugin { + + /** The name the built-in model is selected by, and the value {@code access_controller_type} defaults to. */ + public static final String NAME = "default"; + + private final Auth auth; + + public InternalAuthorizationPlugin(Auth auth) { + this.auth = auth; + } + + @Override + public String name() { + return NAME; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + UserIdentity currentUser = AccessTranslation.userIdentityOf(subject); + PrivPredicate wanted = AccessTranslation.privPredicateOf(requirement); + switch (resource.getKind()) { + case GLOBAL: + refuseUnless(auth.checkGlobalPriv(currentUser, wanted), subject, resource, requirement); + return; + case CATALOG: { + AuthorizedResource.Catalog catalog = (AuthorizedResource.Catalog) resource; + refuseUnless(auth.checkGlobalPriv(currentUser, wanted) + || auth.checkCtlPriv(currentUser, catalog.getCatalog(), wanted), + subject, resource, requirement); + return; + } + case DATABASE: { + AuthorizedResource.Database database = (AuthorizedResource.Database) resource; + refuseUnless(auth.checkGlobalPriv(currentUser, wanted) + || auth.checkDbPriv(currentUser, database.getCatalog(), + database.getDatabase(), wanted), + subject, resource, requirement); + return; + } + case TABLE: { + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + refuseUnless(auth.checkGlobalPriv(currentUser, wanted) + || auth.checkTblPriv(currentUser, table.getCatalog(), table.getDatabase(), + table.getTable(), wanted), + subject, resource, requirement); + return; + } + case COLUMNS: { + AuthorizedResource.Columns columns = (AuthorizedResource.Columns) resource; + if (auth.checkGlobalPriv(currentUser, wanted)) { + return; + } + try { + auth.checkColsPriv(currentUser, columns.getCatalog(), columns.getDatabase(), + columns.getTable(), columns.getColumns(), wanted); + } catch (AuthorizationException e) { + // The message names the column that failed, which is the answer here; rephrasing it + // would reduce "this column" to "these columns". Carried as the bare wording rather + // than as rendered: the engine puts this back into an AuthorizationException on the way + // out, and that class prefixes its own error code when it renders. + throw AccessDeniedException.withMessage(e.getDetailMessage(), resource, NAME); + } + return; + } + case RESOURCE: + refuseUnless(auth.checkResourcePriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), subject, resource, requirement); + return; + case WORKLOAD_GROUP: + refuseUnless(auth.checkWorkloadGroupPriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), subject, resource, requirement); + return; + case STORAGE_VAULT: + refuseUnless(auth.checkStorageVaultPriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), subject, resource, requirement); + return; + case CLOUD_GENERAL: + case CLOUD_COMPUTE_GROUP: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + refuseUnless(auth.checkCloudPriv(currentUser, ((AuthorizedResource.Named) resource).getName(), + wanted, AccessTranslation.cloudTypeOf(resource.getKind())), + subject, resource, requirement); + return; + default: + throw new IllegalStateException("the built-in privilege model has no answer for resource kind " + + resource.getKind()); + } + } + + private void refuseUnless(boolean allowed, AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement) throws AccessDeniedException { + if (!allowed) { + throw AccessDeniedException.of(subject, resource, requirement, NAME); + } + } + + @Override + public Map getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table, + Set columns, AccessContext context) { + // The built-in privilege model has no column masking: there is no DDL to define one. + return Collections.emptyMap(); + } + + @Override + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext context) { + List policies = Env.getCurrentEnv().getPolicyMgr().getUserPolicies(table.getCatalog(), + table.getDatabase(), table.getTable(), AccessTranslation.userIdentityOf(subject)); + ImmutableList.Builder specs = ImmutableList.builderWithExpectedSize(policies.size()); + for (RowPolicy policy : policies) { + try { + specs.add(new RowFilterSpec(policy.getPolicyIdent(), policy.getFilterSql(), + mergeTypeOf(policy.getFilterType()))); + } catch (AnalysisException e) { + // A policy whose statement no longer parses cannot be turned into a filter, and dropping it + // would silently widen access to the whole table. Fail the query with the same message the + // planner used to raise when it asked the policy for its expression. + throw new org.apache.doris.nereids.exceptions.AnalysisException(e.getMessage(), e); + } + } + return specs.build(); + } + + private static RowFilterMergeType mergeTypeOf(FilterType filterType) { + switch (filterType) { + case PERMISSIVE: + return RowFilterMergeType.PERMISSIVE; + case RESTRICTIVE: + return RowFilterMergeType.RESTRICTIVE; + default: + // Same shape as the planner's merge switch used to have: an unmapped filter type is a bug, + // and guessing either way would change which rows the user sees. + throw new IllegalStateException("Invalid operator"); + } + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java new file mode 100644 index 00000000000000..4557cd87ee93ad --- /dev/null +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java @@ -0,0 +1,174 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.common.AuthorizationException; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; +import java.util.Set; + +/** + * Presents an access controller written against the older, per-scope interface as an authorization source. + * + *

That interface asks a separate question per kind of object and answers each with a boolean; this one + * asks a single question about a typed resource and answers by refusing or not. The translation is the whole + * of this class, and it is not a temporary shim: {@code CatalogAccessController} is what a catalog's + * {@code access_controller.class} names, so implementations of it exist outside this repository and keep + * working unchanged. + */ +public class LegacyAccessControllerPlugin implements AuthorizationPlugin { + + private final String name; + private final CatalogAccessController controller; + + public LegacyAccessControllerPlugin(String name, CatalogAccessController controller) { + this.name = Objects.requireNonNull(name, "name is required"); + this.controller = Objects.requireNonNull(controller, "controller is required"); + } + + /** + * The controller this presents. Needed where an identity, not a behaviour, is the question - a controller + * asking whether it is itself the one governing instance scope has to compare against the object it is, + * not against the wrapper it is reached through. + */ + public CatalogAccessController getController() { + return controller; + } + + @Override + public String name() { + return name; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + UserIdentity currentUser = AccessTranslation.userIdentityOf(subject); + PrivPredicate wanted = AccessTranslation.privPredicateOf(requirement); + switch (resource.getKind()) { + case GLOBAL: + refuseUnless(controller.checkGlobalPriv(currentUser, wanted), subject, resource, requirement); + return; + case CATALOG: + refuseUnless(controller.checkCtlPriv(currentUser, + ((AuthorizedResource.Catalog) resource).getCatalog(), wanted), + subject, resource, requirement); + return; + case DATABASE: { + AuthorizedResource.Database database = (AuthorizedResource.Database) resource; + refuseUnless(controller.checkDbPriv(currentUser, database.getCatalog(), + database.getDatabase(), wanted), subject, resource, requirement); + return; + } + case TABLE: { + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + refuseUnless(controller.checkTblPriv(currentUser, table.getCatalog(), table.getDatabase(), + table.getTable(), wanted), subject, resource, requirement); + return; + } + case COLUMNS: { + AuthorizedResource.Columns columns = (AuthorizedResource.Columns) resource; + try { + controller.checkColsPriv(currentUser, columns.getCatalog(), columns.getDatabase(), + columns.getTable(), columns.getColumns(), wanted); + } catch (AuthorizationException e) { + // The message names the column that failed; that is the answer, so it is carried over + // as written rather than restated in terms of the whole column set. As the bare wording, + // not as rendered - the engine wraps it in an AuthorizationException again on the way + // out, and that class prefixes its own error code when it renders. + throw AccessDeniedException.withMessage(e.getDetailMessage(), resource, name); + } + return; + } + case RESOURCE: + refuseUnless(controller.checkResourcePriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), + subject, resource, requirement); + return; + case WORKLOAD_GROUP: + refuseUnless(controller.checkWorkloadGroupPriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), + subject, resource, requirement); + return; + case STORAGE_VAULT: + refuseUnless(controller.checkStorageVaultPriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted), + subject, resource, requirement); + return; + case CLOUD_GENERAL: + case CLOUD_COMPUTE_GROUP: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + refuseUnless(controller.checkCloudPriv(currentUser, + ((AuthorizedResource.Named) resource).getName(), wanted, + AccessTranslation.cloudTypeOf(resource.getKind())), subject, resource, requirement); + return; + default: + throw new IllegalStateException("access controller " + name + " has no method answering for" + + " resource kind " + resource.getKind()); + } + } + + private void refuseUnless(boolean allowed, AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement) throws AccessDeniedException { + if (!allowed) { + throw AccessDeniedException.of(subject, resource, requirement, name); + } + } + + @Override + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext context) { + return controller.evalRowFilterPolicies(AccessTranslation.userIdentityOf(subject), table.getCatalog(), + table.getDatabase(), table.getTable()); + } + + @Override + public Map getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table, + Set columns, AccessContext context) { + UserIdentity currentUser = AccessTranslation.userIdentityOf(subject); + Map masks = new HashMap<>(); + for (String column : columns) { + // One question per column, which is what the older interface offers. A source reached over the + // network pays for that per column of every table in the statement; implementing the batch + // method directly is how a plugin stops paying it. + Optional mask = controller.evalDataMaskPolicy(currentUser, table.getCatalog(), + table.getDatabase(), table.getTable(), column); + mask.ifPresent(spec -> masks.put(column, spec)); + } + return masks; + } + + @Override + public void close() { + controller.close(); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java index a47e38e8d40372..a302d56667873e 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java @@ -24,6 +24,7 @@ import org.apache.doris.mysql.privilege.AccessControllerManager; import org.apache.doris.mysql.privilege.Auth; import org.apache.doris.mysql.privilege.CatalogAccessController; +import org.apache.doris.mysql.privilege.LegacyAccessControllerPlugin; import org.apache.doris.mysql.privilege.PrivPredicate; import org.apache.doris.mysql.privilege.StubRangerPolicyEngine; @@ -112,10 +113,17 @@ private int requestsWhile(CatalogAccessController authority, BooleanSupplier che return engine.requests.get(); } - /** Runs {@code check} against an FE whose {@code access_controller_type} resolves to {@code authority}. */ + /** + * Runs {@code check} against an FE whose {@code access_controller_type} resolves to {@code authority}. + * + *

Installed the way the engine installs one written against the older interface - behind the adapter - + * because that is what makes "is this authority me?" a question about the controller rather than about + * the object the manager happens to hold. + */ private boolean withGlobalScopeAuthority(CatalogAccessController authority, BooleanSupplier check) { AccessControllerManager manager = new AccessControllerManager(new Auth()); - Deencapsulation.setField(manager, "defaultAccessController", authority); + Deencapsulation.setField(manager, "defaultAccessController", + new LegacyAccessControllerPlugin("authority", authority)); try (MockedStatic mockedEnv = Mockito.mockStatic(Env.class)) { Env env = Mockito.mock(Env.class); mockedEnv.when(Env::getCurrentEnv).thenReturn(env); diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java index 618cd120fbfbda..0c75a4877e850f 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java @@ -20,6 +20,7 @@ import org.apache.doris.analysis.UserIdentity; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.Column; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.PrimitiveType; @@ -213,11 +214,11 @@ private List renderSnapshot() { lines.add("USERS: " + String.join(",", USERS)); lines.add(""); AccessControllerManager manager = Env.getCurrentEnv().getAccessManager(); - CatalogAccessController builtin = Deencapsulation.getField(manager, "defaultAccessController"); + AuthorizationPlugin builtin = Deencapsulation.getField(manager, "defaultAccessController"); renderWithDefaultController(lines, manager, "builtin", builtin); - renderWithDefaultController(lines, manager, "ranger", - new RangerDorisAccessController(new StubRangerPolicyEngine())); + renderWithDefaultController(lines, manager, "ranger", new LegacyAccessControllerPlugin( + "stub-ranger-doris", new RangerDorisAccessController(new StubRangerPolicyEngine()))); return lines; } @@ -226,8 +227,8 @@ private List renderSnapshot() { * {@code fe.conf: access_controller_type}. */ private void renderWithDefaultController(List lines, AccessControllerManager manager, - String label, CatalogAccessController defaultController) { - CatalogAccessController original = Deencapsulation.getField(manager, "defaultAccessController"); + String label, AuthorizationPlugin defaultController) { + AuthorizationPlugin original = Deencapsulation.getField(manager, "defaultAccessController"); Map routes = Deencapsulation.getField(manager, "ctlToCtlAccessController"); Map savedRoutes = new HashMap<>(routes); try { diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java index 06d8ca6d56653d..8913e56edee120 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControllerManagerTest.java @@ -278,7 +278,7 @@ public void testOldGenerationCannotRemoveReplacementController() { manager.createAccessController(newCatalog, "test-controller", ImmutableMap.of(), false); manager.removeAccessController("same_name", oldCatalog.getId()); - Assert.assertSame(newController, manager.getAccessControllerOrDefault("same_name")); + Assert.assertSame(newController, controllerOf(manager, "same_name")); } Mockito.verify(oldController).close(); @@ -324,7 +324,7 @@ public void testRemovingFallbackAliasDoesNotCloseSharedDefaultController() { Mockito.when(env.getCatalogMgr()).thenReturn(catalogMgr); Mockito.when(catalogMgr.getCatalog("fallback")).thenReturn(catalog); - Assert.assertSame(defaultAccessController, manager.getAccessControllerOrDefault("fallback")); + Assert.assertSame(defaultAccessController, controllerOf(manager, "fallback")); manager.removeAccessController("fallback", catalog.getId()); } @@ -351,7 +351,13 @@ private void withCurrentCatalog(ExternalCatalog catalog, Runnable action) { private AccessControllerManager createAccessControllerManager(CatalogAccessController defaultAccessController) { AccessControllerManager accessControllerManager = new AccessControllerManager(new Auth()); - Deencapsulation.setField(accessControllerManager, "defaultAccessController", defaultAccessController); + Deencapsulation.setField(accessControllerManager, "defaultAccessController", + new LegacyAccessControllerPlugin("mock", defaultAccessController)); return accessControllerManager; } + + /** The controller behind an installed source; what the manager holds is the source, not the controller. */ + private CatalogAccessController controllerOf(AccessControllerManager manager, String ctl) { + return ((LegacyAccessControllerPlugin) manager.getAccessControllerOrDefault(ctl)).getController(); + } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java index ffb950521bac2e..1bd134047549ba 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessTranslationTest.java @@ -19,9 +19,11 @@ import org.apache.doris.analysis.CompoundPredicate.Operator; import org.apache.doris.analysis.ResourceTypeEnum; +import org.apache.doris.analysis.UserIdentity; import org.apache.doris.authorization.AccessAction; import org.apache.doris.authorization.AccessRequirement; import org.apache.doris.authorization.ActionMatch; +import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.ResourceKind; import org.apache.doris.common.jmockit.Deencapsulation; @@ -184,6 +186,51 @@ public void testOnlyCloudKindsHaveACloudResourceType() { () -> AccessTranslation.cloudTypeOf(ResourceKind.TABLE)); } + /** + * An account survives the trip out to a source and back into the privilege tables. + * + *

The built-in model is itself one of the sources, and it looks accounts up in tables keyed by + * account. So the neutral form has to carry everything an account is identified by - and only a test can + * say that it does, because dropping a part of an identity does not fail: it silently matches a + * different account's grants, or none. + */ + @Test + public void testAnAccountSurvivesTheRoundTrip() { + for (UserIdentity original : Arrays.asList( + UserIdentity.createAnalyzedUserIdentWithIp("user", "192.168.%"), + UserIdentity.createAnalyzedUserIdentWithIp("user", "%"), + UserIdentity.createAnalyzedUserIdentWithDomain("user", "doris.example.com"), + UserIdentity.ROOT, + UserIdentity.ADMIN, + // Callers that check access with an identity they never put through analysis exist - the + // cloud checks are reached that way. Translating one must answer, not object. + new UserIdentity("user", "%"))) { + UserIdentity translated = AccessTranslation.userIdentityOf(AccessTranslation.subjectOf(original)); + + Assert.assertEquals(original.toString(), original, translated); + Assert.assertEquals(original.toString(), original.getUser(), translated.getUser()); + Assert.assertEquals(original.toString(), original.getHost(), translated.getHost()); + Assert.assertEquals(original.toString(), original.isDomain(), translated.isDomain()); + } + } + + /** + * Two accounts that differ only in where they connect from are different accounts with different grants, + * so they must not collapse into one subject on the way out. + */ + @Test + public void testAccountsDifferingOnlyInHostStayApart() { + AuthorizedSubject anywhere = AccessTranslation.subjectOf( + UserIdentity.createAnalyzedUserIdentWithIp("user", "%")); + AuthorizedSubject fromOffice = AccessTranslation.subjectOf( + UserIdentity.createAnalyzedUserIdentWithIp("user", "192.168.%")); + AuthorizedSubject inDomain = AccessTranslation.subjectOf( + UserIdentity.createAnalyzedUserIdentWithDomain("user", "192.168.%")); + + Assert.assertNotEquals(anywhere, fromOffice); + Assert.assertNotEquals(fromOffice, inDomain); + } + private void assertRoundTrips(String what, PrivPredicate wanted) { PrivPredicate translated = AccessTranslation.privPredicateOf(AccessTranslation.requirementOf(wanted)); Assert.assertEquals(what + ": privileges changed", bitsOf(wanted), bitsOf(translated)); diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java deleted file mode 100644 index 49d0d19f7368d5..00000000000000 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAccessControllerTest.java +++ /dev/null @@ -1,139 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -package org.apache.doris.mysql.privilege; - -import org.apache.doris.analysis.UserIdentity; -import org.apache.doris.common.AuthorizationException; - -import com.google.common.collect.ImmutableSet; -import org.junit.Assert; -import org.junit.Test; -import org.mockito.Mockito; - -/** - * The built-in controller answers "globally, then at this scope". - * - *

Both halves of that sentence are load bearing. Answering globally first is what lets an administrator - * reach a resource no grant names, and it must happen before the scoped lookup rather than instead of - * it - the scoped lookups are the expensive ones, and one of them refuses to answer at all for privileges that - * only exist globally. The engine used to arrange this order on every controller's behalf; now each controller - * owns it, so these tests watch the built-in one keep it. - */ -public class InternalAccessControllerTest { - private static final UserIdentity USER = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); - - private final Auth auth = Mockito.mock(Auth.class); - private final InternalAccessController controller = new InternalAccessController(auth); - - private void holdsGlobally(boolean granted) { - Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.SELECT)).thenReturn(granted); - } - - @Test - public void testCatalogCheckIsSkippedWhenPrivilegeIsHeldGlobally() { - holdsGlobally(true); - - Assert.assertTrue(controller.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)); - Mockito.verify(auth, Mockito.never()).checkCtlPriv(Mockito.any(), Mockito.anyString(), Mockito.any()); - } - - @Test - public void testCatalogCheckDecidesWhenPrivilegeIsNotHeldGlobally() { - holdsGlobally(false); - Mockito.when(auth.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)).thenReturn(true); - - Assert.assertTrue(controller.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)); - Assert.assertFalse(controller.checkCtlPriv(USER, "other_ctl", PrivPredicate.SELECT)); - } - - @Test - public void testDatabaseCheckIsSkippedWhenPrivilegeIsHeldGlobally() { - holdsGlobally(true); - - Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)); - Mockito.verify(auth, Mockito.never()) - .checkDbPriv(Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); - } - - @Test - public void testDatabaseCheckDecidesWhenPrivilegeIsNotHeldGlobally() { - holdsGlobally(false); - Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)).thenReturn(true); - - Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)); - Assert.assertFalse(controller.checkDbPriv(USER, "ctl", "other_db", PrivPredicate.SELECT)); - } - - @Test - public void testTableCheckIsSkippedWhenPrivilegeIsHeldGlobally() { - holdsGlobally(true); - - Assert.assertTrue(controller.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)); - Mockito.verify(auth, Mockito.never()).checkTblPriv( - Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); - } - - @Test - public void testTableCheckDecidesWhenPrivilegeIsNotHeldGlobally() { - holdsGlobally(false); - Mockito.when(auth.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)).thenReturn(true); - - Assert.assertTrue(controller.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)); - Assert.assertFalse(controller.checkTblPriv(USER, "ctl", "db", "other_tbl", PrivPredicate.SELECT)); - } - - @Test - public void testColumnCheckIsSkippedWhenPrivilegeIsHeldGlobally() throws Exception { - holdsGlobally(true); - - controller.checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - Mockito.verify(auth, Mockito.never()).checkColsPriv(Mockito.any(), Mockito.anyString(), - Mockito.anyString(), Mockito.anyString(), Mockito.any(), Mockito.any()); - } - - @Test - public void testColumnCheckDecidesWhenPrivilegeIsNotHeldGlobally() throws Exception { - holdsGlobally(false); - - controller.checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - Mockito.verify(auth).checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - } - - @Test - public void testColumnDenialIsReportedWhenPrivilegeIsNotHeldGlobally() throws Exception { - holdsGlobally(false); - Mockito.doThrow(new AuthorizationException("denied")).when(auth).checkColsPriv( - USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); - - Assert.assertThrows(AuthorizationException.class, () -> controller.checkColsPriv( - USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT)); - } - - /** - * A caller holding only global NODE_PRIV is why the global check has to run first instead of being folded - * into the scoped one: {@link Auth} refuses NODE privileges below global level, so the scoped lookup would - * turn the administrator away. - */ - @Test - public void testGloballyHeldNodePrivilegeIsNotRefusedByTheScopedCheck() { - Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.OPERATOR)).thenReturn(true); - Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.OPERATOR)).thenReturn(false); - - Assert.assertTrue(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.OPERATOR)); - } -} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAuthorizationPluginTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAuthorizationPluginTest.java new file mode 100644 index 00000000000000..ab6ddd4a32b672 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/InternalAuthorizationPluginTest.java @@ -0,0 +1,196 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.common.AuthorizationException; + +import com.google.common.collect.ImmutableSet; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; + +/** + * The built-in source answers "globally, then at this scope". + * + *

Both halves of that sentence are load bearing. Answering globally first is what lets an administrator + * reach a resource no grant names, and it must happen before the scoped lookup rather than instead of + * it - the scoped lookups are the expensive ones, and one of them refuses to answer at all for privileges that + * only exist globally. The engine used to arrange this order on every source's behalf; now each source owns + * it, so these tests watch the built-in one keep it. + * + *

They also pin the translation on the way in, without asserting anything about it directly: the mocked + * {@link Auth} is told to expect the identity and the predicate the caller started from, so a translation + * that produced anything else would leave every expectation unmatched. + */ +public class InternalAuthorizationPluginTest { + private static final UserIdentity USER = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); + private static final AuthorizedSubject SUBJECT = AccessTranslation.subjectOf(USER); + private static final AccessRequirement SELECT = AccessTranslation.requirementOf(PrivPredicate.SELECT); + + private final Auth auth = Mockito.mock(Auth.class); + private final InternalAuthorizationPlugin plugin = new InternalAuthorizationPlugin(auth); + + private void holdsGlobally(boolean granted) { + Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.SELECT)).thenReturn(granted); + } + + private boolean allows(AuthorizedResource resource) { + return allows(resource, SELECT); + } + + private boolean allows(AuthorizedResource resource, AccessRequirement requirement) { + try { + plugin.checkPrivilege(SUBJECT, resource, requirement, AccessContext.NONE); + return true; + } catch (AccessDeniedException e) { + return false; + } + } + + @Test + public void testCatalogCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(allows(AuthorizedResource.catalog("ctl"))); + Mockito.verify(auth, Mockito.never()).checkCtlPriv(Mockito.any(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testCatalogCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.catalog("ctl"))); + Assert.assertFalse(allows(AuthorizedResource.catalog("other_ctl"))); + } + + @Test + public void testDatabaseCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(allows(AuthorizedResource.database("ctl", "db"))); + Mockito.verify(auth, Mockito.never()) + .checkDbPriv(Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testDatabaseCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.database("ctl", "db"))); + Assert.assertFalse(allows(AuthorizedResource.database("ctl", "other_db"))); + } + + @Test + public void testTableCheckIsSkippedWhenPrivilegeIsHeldGlobally() { + holdsGlobally(true); + + Assert.assertTrue(allows(AuthorizedResource.table("ctl", "db", "tbl"))); + Mockito.verify(auth, Mockito.never()).checkTblPriv( + Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.anyString(), Mockito.any()); + } + + @Test + public void testTableCheckDecidesWhenPrivilegeIsNotHeldGlobally() { + holdsGlobally(false); + Mockito.when(auth.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.table("ctl", "db", "tbl"))); + Assert.assertFalse(allows(AuthorizedResource.table("ctl", "db", "other_tbl"))); + } + + @Test + public void testColumnCheckIsSkippedWhenPrivilegeIsHeldGlobally() throws Exception { + holdsGlobally(true); + + plugin.checkPrivilege(SUBJECT, AuthorizedResource.columns("ctl", "db", "tbl", ImmutableSet.of("col1")), + SELECT, AccessContext.NONE); + Mockito.verify(auth, Mockito.never()).checkColsPriv(Mockito.any(), Mockito.anyString(), + Mockito.anyString(), Mockito.anyString(), Mockito.any(), Mockito.any()); + } + + @Test + public void testColumnCheckDecidesWhenPrivilegeIsNotHeldGlobally() throws Exception { + holdsGlobally(false); + + plugin.checkPrivilege(SUBJECT, AuthorizedResource.columns("ctl", "db", "tbl", ImmutableSet.of("col1")), + SELECT, AccessContext.NONE); + Mockito.verify(auth).checkColsPriv(USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + } + + /** + * A refused column check answers with the column that failed, so the wording the privilege model produced + * has to survive the trip out - restating it in terms of the whole column set would lose which one it was. + * + *

What the second assertion holds still is the message the user actually reads. The refusal leaves the + * privilege model as an {@link AuthorizationException} and reaches the caller as one again, and that class + * renders itself with its error code in front. Carrying the rendered form across the middle instead of the + * bare wording puts that prefix in twice - which does not fail anywhere, it just changes what every denied + * column query prints. + */ + @Test + public void testColumnDenialKeepsTheMessageNamingTheColumn() throws Exception { + holdsGlobally(false); + AuthorizationException fromThePrivilegeModel = new AuthorizationException("denied on col1"); + Mockito.doThrow(fromThePrivilegeModel).when(auth).checkColsPriv( + USER, "ctl", "db", "tbl", ImmutableSet.of("col1"), PrivPredicate.SELECT); + + AccessDeniedException denied = Assert.assertThrows(AccessDeniedException.class, + () -> plugin.checkPrivilege(SUBJECT, + AuthorizedResource.columns("ctl", "db", "tbl", ImmutableSet.of("col1")), + SELECT, AccessContext.NONE)); + + Assert.assertEquals("denied on col1", denied.getMessage()); + Assert.assertEquals(fromThePrivilegeModel.getMessage(), + new AuthorizationException(denied.getMessage()).getMessage()); + } + + /** + * A caller holding only global NODE_PRIV is why the global check has to run first instead of being folded + * into the scoped one: {@link Auth} refuses NODE privileges below global level, so the scoped lookup would + * turn the administrator away. + */ + @Test + public void testGloballyHeldNodePrivilegeIsNotRefusedByTheScopedCheck() { + Mockito.when(auth.checkGlobalPriv(USER, PrivPredicate.OPERATOR)).thenReturn(true); + Mockito.when(auth.checkDbPriv(USER, "ctl", "db", PrivPredicate.OPERATOR)).thenReturn(false); + + Assert.assertTrue(allows(AuthorizedResource.database("ctl", "db"), + AccessTranslation.requirementOf(PrivPredicate.OPERATOR))); + } + + /** + * The system-wide objects are deliberately not preceded by a global check here: {@link Auth} + * folds the global grants into those answers itself, and asking twice would evaluate the same grants + * again for the same answer. + */ + @Test + public void testSystemWideObjectsAreLeftEntirelyToTheGrantLookup() { + Mockito.when(auth.checkResourcePriv(USER, "res", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.resource("res"))); + Mockito.verify(auth, Mockito.never()).checkGlobalPriv(Mockito.any(), Mockito.any()); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPluginTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPluginTest.java new file mode 100644 index 00000000000000..785c0c845eb271 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPluginTest.java @@ -0,0 +1,189 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.ResourceTypeEnum; +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.common.AuthorizationException; + +import com.google.common.collect.ImmutableSet; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.Mockito; + +import java.util.LinkedHashSet; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + +/** + * A controller written against the older per-scope interface, asked the way an authorization source is asked. + * + *

The translation this covers is a lookup table with no logic in it, which is exactly why it is worth + * testing: every resource kind has to reach the one method that used to answer for it, and a wire crossed + * between two of them - a storage vault asked about as if it were a resource - produces a plausible answer + * from the wrong policy set, silently. So each kind is checked against the method it must land on, and each + * is checked to be refused when that method says no. + */ +public class LegacyAccessControllerPluginTest { + + private static final UserIdentity USER = UserIdentity.createAnalyzedUserIdentWithIp("test_user", "%"); + private static final AuthorizedSubject SUBJECT = AccessTranslation.subjectOf(USER); + private static final AccessRequirement SELECT = AccessTranslation.requirementOf(PrivPredicate.SELECT); + + private final CatalogAccessController controller = Mockito.mock(CatalogAccessController.class); + private final LegacyAccessControllerPlugin plugin = new LegacyAccessControllerPlugin("legacy", controller); + + private boolean allows(AuthorizedResource resource) { + try { + plugin.checkPrivilege(SUBJECT, resource, SELECT, AccessContext.NONE); + return true; + } catch (AccessDeniedException e) { + return false; + } + } + + @Test + public void testGlobalGoesToTheGlobalCheck() { + Mockito.when(controller.checkGlobalPriv(USER, PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.global())); + } + + @Test + public void testCatalogGoesToTheCatalogCheck() { + Mockito.when(controller.checkCtlPriv(USER, "ctl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.catalog("ctl"))); + Assert.assertFalse(allows(AuthorizedResource.catalog("other"))); + } + + @Test + public void testDatabaseGoesToTheDatabaseCheck() { + Mockito.when(controller.checkDbPriv(USER, "ctl", "db", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.database("ctl", "db"))); + Assert.assertFalse(allows(AuthorizedResource.database("ctl", "other"))); + } + + @Test + public void testTableGoesToTheTableCheck() { + Mockito.when(controller.checkTblPriv(USER, "ctl", "db", "tbl", PrivPredicate.SELECT)).thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.table("ctl", "db", "tbl"))); + Assert.assertFalse(allows(AuthorizedResource.table("ctl", "db", "other"))); + } + + /** + * The three system-wide names are the ones a crossed wire would hide best: they are all "a name and a + * privilege", so asking about a vault through the resource check would look like it worked and would + * consult grants nobody made about the vault. + */ + @Test + public void testEachSystemWideNameGoesToItsOwnCheck() { + Mockito.when(controller.checkResourcePriv(USER, "name", PrivPredicate.SELECT)).thenReturn(true); + Mockito.when(controller.checkWorkloadGroupPriv(USER, "name", PrivPredicate.SELECT)).thenReturn(false); + Mockito.when(controller.checkStorageVaultPriv(USER, "name", PrivPredicate.SELECT)).thenReturn(false); + + Assert.assertTrue(allows(AuthorizedResource.resource("name"))); + Assert.assertFalse(allows(AuthorizedResource.workloadGroup("name"))); + Assert.assertFalse(allows(AuthorizedResource.storageVault("name"))); + + Mockito.verify(controller).checkResourcePriv(USER, "name", PrivPredicate.SELECT); + Mockito.verify(controller).checkWorkloadGroupPriv(USER, "name", PrivPredicate.SELECT); + Mockito.verify(controller).checkStorageVaultPriv(USER, "name", PrivPredicate.SELECT); + } + + /** Which cloud object it is travels along, because the privileges live in different tables per type. */ + @Test + public void testCloudObjectsCarryTheirTypeThrough() { + Mockito.when(controller.checkCloudPriv(USER, "cg", PrivPredicate.SELECT, ResourceTypeEnum.CLUSTER)) + .thenReturn(true); + + Assert.assertTrue(allows(AuthorizedResource.cloud(ResourceKind.CLOUD_COMPUTE_GROUP, "cg"))); + Assert.assertFalse(allows(AuthorizedResource.cloud(ResourceKind.CLOUD_STAGE, "cg"))); + } + + @Test + public void testColumnsGoToTheColumnCheck() throws Exception { + Set columns = ImmutableSet.of("col1", "col2"); + + plugin.checkPrivilege(SUBJECT, AuthorizedResource.columns("ctl", "db", "tbl", columns), SELECT, + AccessContext.NONE); + + Mockito.verify(controller).checkColsPriv(USER, "ctl", "db", "tbl", columns, PrivPredicate.SELECT); + } + + /** + * The refused column is named in the message; that name is the answer and has to arrive intact - and in + * the bare form the controller wrote it, because the engine puts it back into an + * {@link AuthorizationException} on the way out and that class renders its own error code in front. + * Carrying the rendered form across would print that prefix twice on every denied column query. + */ + @Test + public void testAColumnRefusalKeepsTheMessageThatNamesTheColumn() throws Exception { + AuthorizationException fromTheController = new AuthorizationException("no privilege on [col2]"); + Mockito.doThrow(fromTheController).when(controller).checkColsPriv( + Mockito.any(), Mockito.anyString(), Mockito.anyString(), Mockito.anyString(), + Mockito.any(), Mockito.any()); + + AccessDeniedException denied = Assert.assertThrows(AccessDeniedException.class, + () -> plugin.checkPrivilege(SUBJECT, + AuthorizedResource.columns("ctl", "db", "tbl", ImmutableSet.of("col1", "col2")), + SELECT, AccessContext.NONE)); + + Assert.assertEquals("no privilege on [col2]", denied.getMessage()); + Assert.assertEquals("legacy", denied.getDeniedBy().orElse(null)); + Assert.assertEquals(fromTheController.getMessage(), + new AuthorizationException(denied.getMessage()).getMessage()); + } + + /** + * The batch mask question is asked of a controller that only answers per column, so it asks per column - + * and reports only the columns that came back masked, an absent entry being how "not masked" is said. + */ + @Test + public void testDataMasksAreCollectedOneColumnAtATime() { + DataMaskSpec spec = new DataMaskSpec("policy", "CONCAT(LEFT(col1,1),'***')"); + Mockito.when(controller.evalDataMaskPolicy(USER, "ctl", "db", "tbl", "col1")) + .thenReturn(Optional.of(spec)); + Mockito.when(controller.evalDataMaskPolicy(USER, "ctl", "db", "tbl", "col2")) + .thenReturn(Optional.empty()); + + Map masks = plugin.getDataMasks(SUBJECT, + AuthorizedResource.table("ctl", "db", "tbl"), + new LinkedHashSet<>(ImmutableSet.of("col1", "col2")), AccessContext.NONE); + + Assert.assertEquals(ImmutableSet.of("col1"), masks.keySet()); + Assert.assertSame(spec, masks.get("col1")); + } + + @Test + public void testClosingTheSourceClosesTheController() { + plugin.close(); + + Mockito.verify(controller).close(); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java b/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java index 76c7d3d91eb11f..56344470523a22 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java +++ b/fe/fe-core/src/test/java/org/apache/doris/nereids/privileges/TestCheckPrivileges.java @@ -21,6 +21,7 @@ import org.apache.doris.analysis.UserIdentity; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.Column; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.PrimitiveType; @@ -130,7 +131,7 @@ public void testPrivilegesAndPolicies() throws Exception { ); AccessControllerManager accessManager = Env.getCurrentEnv().getAccessManager(); - CatalogAccessController catalogAccessController = accessManager.getAccessControllerOrDefault(catalog); + AuthorizationPlugin catalogAccessController = accessManager.getAccessControllerOrDefault(catalog); AccessControllerManager spyAccessManager = Mockito.spy(accessManager); Mockito.doReturn(catalogAccessController).when(spyAccessManager) .getAccessControllerOrDefault("internal"); From 12fa0a250fd365f93aeedde575e42643cc1f7f2d Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 20:21:18 +0800 Subject: [PATCH 08/22] [improvement](authorization) let the Ranger sources answer the plugin contract themselves The two Ranger sources shipped in the tree were reached through the adapter for the older per-scope controller interface, so the contract had exactly one native implementation - the built-in privilege model, which answers out of its own tables. A contract that has only been implemented by the thing it was extracted from has not been tried. They now implement it directly: one question about a typed resource, answered by refusing or not, dispatched inside the source. What each of them decides is unchanged, down to walking the resource hierarchy one privilege per request while remembering which ones an outer level already granted - that walk is why the engine asks about a whole requirement rather than one action at a time. Two things they used to reach into the engine for now arrive through the context the engine hands them: the roles a Doris account holds, which a Ranger policy may be written against and which only the engine can resolve, and whether whoever governs instance scope already grants the privilege. The latter is also configurable now, so a deployment can decide that inside what Ranger governs only Ranger's policies grant anything - not even to an administrator of the instance. Recognising which question is being asked used to be object identity against the engine's own predicate constants. Those constants cannot follow the sources out of the engine, so the questions the engine asks by name are now values in the neutral module, compared by equality and pinned on both sides: a source that stopped recognising "may this subject see the object" would silently answer a different question rather than fail. Behaviour matrix unchanged. The refusal message for a column now names the requirement in neutral terms. --- .../authorization/AccessRequirements.java | 82 ++++ .../ranger/RangerAccessController.java | 124 ++++-- .../ranger/doris/DorisAccessType.java | 43 +- .../doris/RangerDorisAccessController.java | 410 +++++++++--------- .../hive/RangerHiveAccessController.java | 270 ++++++------ .../RangerHiveAccessControllerFactory.java | 20 +- .../privilege/AccessControllerManager.java | 97 +++-- .../privilege/EngineAuthorizationContext.java | 83 ++++ .../LegacyAccessControllerPlugin.java | 6 +- .../RangerDorisAccessControllerFactory.java | 41 +- ...horization.spi.AuthorizationPluginFactory} | 2 +- .../RangerGlobalScopeDeferenceTest.java | 167 ++++--- .../hive/RangerHiveAccessControllerTest.java | 96 ++++ .../AccessControlBehaviorBaselineTest.java | 12 +- .../AccessRequirementVocabularyTest.java | 69 +++ .../EngineAuthorizationContextTest.java | 146 +++++++ ...angerDorisAccessControllerFactoryTest.java | 27 +- .../doris/mysql/privilege/RangerTest.java | 237 ++++++---- .../StubRangerAccessControllerFactory.java | 13 +- ...thorization.spi.AuthorizationPluginFactory | 18 + ...is.mysql.privilege.AccessControllerFactory | 1 - 21 files changed, 1381 insertions(+), 583 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirements.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/EngineAuthorizationContext.java rename fe/fe-core/src/main/resources/META-INF/services/{org.apache.doris.mysql.privilege.AccessControllerFactory => org.apache.doris.authorization.spi.AuthorizationPluginFactory} (98%) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessRequirementVocabularyTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/EngineAuthorizationContextTest.java create mode 100644 fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory diff --git a/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirements.java b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirements.java new file mode 100644 index 00000000000000..805bfdbef39d2c --- /dev/null +++ b/fe/fe-authorization/fe-authorization-api/src/main/java/org/apache/doris/authorization/AccessRequirements.java @@ -0,0 +1,82 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization; + +/** + * The requirements the engine asks about by name, so that a plugin can recognise which question it is being + * asked. + * + *

Doris does not ask "may this subject select"; it asks for a set of actions and how many of them are + * needed. A plugin that answers out of a model of its own has to map that back onto its own vocabulary, and + * for that it must be able to tell "may this subject read the table" apart from "may this subject see that the + * table exists" - two questions that differ only in which actions they name. Comparing against these + * constants is how that is done; they are values, so equality is the comparison. + * + *

These are the questions the engine has always asked, named here rather than left implicit in the + * built-in privilege model, because a plugin that lives outside this repository cannot read that model and + * would otherwise have to restate the action sets and drift when one of them changes. + * + *

A requirement not listed here is not a defect and must not be treated as one: privilege checks are also + * built at run time - granting a privilege requires holding both it and the right to grant it - and a plugin + * that recognises none of these is free to answer from the action set alone. + */ +public final class AccessRequirements { + + /** + * May the subject see that the object exists at all. Not a privilege anyone grants: holding any privilege + * that implies knowledge of the object answers it. + */ + public static final AccessRequirement VISIBILITY = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.SELECT, AccessAction.LOAD, AccessAction.ALTER, AccessAction.CREATE, + AccessAction.DROP, AccessAction.SHOW_VIEW); + + /** May the subject read the object's data. */ + public static final AccessRequirement SELECT = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.SELECT); + + /** May the subject write data into the object. */ + public static final AccessRequirement LOAD = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.LOAD); + + /** May the subject change the object's definition. */ + public static final AccessRequirement ALTER = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.ALTER); + + /** May the subject create the object. */ + public static final AccessRequirement CREATE = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.CREATE); + + /** May the subject drop the object. */ + public static final AccessRequirement DROP = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.DROP); + + /** Is the subject an administrator of this instance. */ + public static final AccessRequirement ADMINISTRATION = AccessRequirement.of(AccessAction.ADMIN); + + /** + * Does the subject hold any privilege at all on the object. Asked where a statement needs the object to + * be usable without knowing yet what will be done with it. + */ + public static final AccessRequirement ANY_PRIVILEGE = AccessRequirement.anyOf(AccessAction.NODE, + AccessAction.ADMIN, AccessAction.SELECT, AccessAction.LOAD, AccessAction.ALTER, + AccessAction.CREATE, AccessAction.DROP, AccessAction.USAGE, AccessAction.CLUSTER_USAGE, + AccessAction.STAGE_USAGE); + + private AccessRequirements() { + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java index 9ba6b19c44ac07..2a008e284098c3 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java @@ -17,15 +17,16 @@ package org.apache.doris.catalog.authorizer.ranger; -import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; -import org.apache.doris.catalog.Env; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; -import org.apache.doris.common.AuthorizationException; -import org.apache.doris.mysql.privilege.AccessControllerManager; -import org.apache.doris.mysql.privilege.CatalogAccessController; -import org.apache.doris.mysql.privilege.PrivPredicate; import com.google.common.collect.Lists; import org.apache.commons.lang3.StringUtils; @@ -38,30 +39,73 @@ import org.apache.ranger.plugin.service.RangerBasePlugin; import java.util.Collection; +import java.util.HashMap; import java.util.List; +import java.util.Map; +import java.util.Objects; import java.util.Optional; +import java.util.Set; -public abstract class RangerAccessController implements CatalogAccessController { +/** + * What the Ranger-backed authorization sources have in common: they answer out of a Ranger service's + * policies, and they honour whoever governs instance scope. + */ +public abstract class RangerAccessController implements AuthorizationPlugin { private static final Logger LOG = LogManager.getLogger(RangerAccessController.class); protected static final String CLIENT_TYPE_DORIS = "doris"; /** - * Whether the privilege is already held at global scope, which the Ranger plugins honour as a grant on + * Property switching off deference to whoever governs instance scope, so that a deployment can decide + * that inside what Ranger governs, only Ranger's policies grant anything - not even to an administrator + * of the instance. Defaults to deferring, which is what Doris did before this was a source's own choice. + */ + public static final String DEFER_TO_GLOBAL_SCOPE_AUTHORITY = "ranger.defer_to_global_scope_authority"; + + private final AuthorizationContext context; + private final boolean deferToGlobalScopeAuthority; + + protected RangerAccessController(Map properties, AuthorizationContext context) { + this.context = Objects.requireNonNull(context, "authorization context is required"); + this.deferToGlobalScopeAuthority = deferenceFrom(properties); + } + + /** What this source may ask the engine. */ + protected AuthorizationContext getContext() { + return context; + } + + /** + * Whether the requirement is already held at instance scope, which these sources honour as a grant on * everything they govern. * - *

Global scope is not a Ranger catalog: it belongs to whichever controller {@code access_controller_type} - * installs, so that is who gets asked. With the built-in controller there this reproduces "an administrator - * of the cluster can reach a Ranger-governed catalog"; with Ranger installed globally, Ranger decides its - * own exemptions and the built-in grants stay out of it. Deciding this here rather than in the engine is - * what lets a third-party controller refuse the exemption outright. - * - *

Returns false without asking when this controller is itself the global-scope authority: the caller's - * own global check answers the same question one line later. + *

Instance scope is not a Ranger service: it belongs to whichever source {@code access_controller_type} + * installs, so that is who the engine asks on our behalf. With the built-in model there this reproduces + * "an administrator of the cluster can reach a Ranger-governed catalog"; with Ranger installed for the + * instance, Ranger decides its own exemptions and the built-in grants stay out of it. Deciding it here + * rather than in the engine is what lets a source refuse the exemption outright - as this one does when + * configured to. */ - protected boolean grantedByGlobalScopeAuthority(UserIdentity currentUser, PrivPredicate wanted) { - AccessControllerManager manager = Env.getCurrentEnv().getAccessManager(); - return !manager.isGlobalScopeAuthority(this) && manager.checkGlobalPriv(currentUser, wanted); + protected boolean grantedByGlobalScopeAuthority(AuthorizedSubject subject, AccessRequirement requirement) { + return deferToGlobalScopeAuthority && context.grantedByGlobalScopeAuthority(subject, requirement); + } + + private static boolean deferenceFrom(Map properties) { + String configured = properties == null ? null : properties.get(DEFER_TO_GLOBAL_SCOPE_AUTHORITY); + if (configured == null) { + return true; + } + String value = configured.trim(); + if ("true".equalsIgnoreCase(value)) { + return true; + } + if ("false".equalsIgnoreCase(value)) { + return false; + } + // Parsing this leniently would read a typo as "false" and silently take away an administrator's + // access to every Ranger-governed object. + throw new IllegalArgumentException(DEFER_TO_GLOBAL_SCOPE_AUTHORITY + " must be true or false, but is \"" + + configured + "\""); } protected static boolean checkRequestResult(RangerAccessRequestImpl request, @@ -88,8 +132,12 @@ protected static boolean checkRequestResult(RangerAccessRequestImpl request, } } - public static void checkRequestResults(Collection results, String name) - throws AuthorizationException { + /** + * Refuses on the first request Ranger denied, naming the resource that request was about - which is the + * answer when a batch of requests stands for the columns of one table. + */ + protected void checkRequestResults(Collection results, String name, + AuthorizedResource resource) throws AccessDeniedException { for (RangerAccessResult result : results) { if (LOG.isDebugEnabled()) { LOG.debug("request {} match policy {}", result.getAccessRequest(), result.getPolicyId()); @@ -98,11 +146,12 @@ public static void checkRequestResults(Collection results, S if (LOG.isDebugEnabled()) { LOG.debug(result.getReason()); } - throw new AuthorizationException(String.format( + throw AccessDeniedException.withMessage(String.format( "Permission denied: user [%s] does not have privilege for [%s] command on [%s]", result.getAccessRequest().getUser(), name, Optional.ofNullable(result.getAccessRequest().getResource().getAsString()) - .orElse("unknown resource").replaceAll("/", "."))); + .orElse("unknown resource").replaceAll("/", ".")), + resource, name()); } } } @@ -117,10 +166,11 @@ private static String policyIdent(RangerAccessResult policy) { } @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, - String tbl) { - RangerAccessResourceImpl resource = createResource(ctl, db, tbl); - RangerAccessRequestImpl request = createRequest(currentUser); + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext context) { + RangerAccessResourceImpl resource = createResource(table.getCatalog(), table.getDatabase(), + table.getTable()); + RangerAccessRequestImpl request = createRequest(subject); // If the access type is not set here, it defaults to ANY1 ACCESS. // The internal logic of the ranger is to traverse all permission items. // Since the ranger UI will set the access type to 'SELECT', @@ -149,10 +199,22 @@ public List evalRowFilterPolicies(UserIdentity currentUser, Strin } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, + public Map getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table, + Set columns, AccessContext context) { + Map masks = new HashMap<>(); + for (String column : columns) { + // One request per column: a masking policy in Ranger is written against a column, and the plugin + // evaluates them one resource at a time. + evalDataMaskPolicy(subject, table, column).ifPresent(mask -> masks.put(column, mask)); + } + return masks; + } + + private Optional evalDataMaskPolicy(AuthorizedSubject subject, AuthorizedResource.Table table, String col) { - RangerAccessResourceImpl resource = createResource(ctl, db, tbl, col); - RangerAccessRequestImpl request = createRequest(currentUser); + RangerAccessResourceImpl resource = createResource(table.getCatalog(), table.getDatabase(), + table.getTable(), col); + RangerAccessRequestImpl request = createRequest(subject); request.setAccessType(DorisAccessType.SELECT.name()); request.setResource(resource); @@ -190,7 +252,7 @@ public Optional evalDataMaskPolicy(UserIdentity currentUser, Strin } } - protected abstract RangerAccessRequestImpl createRequest(UserIdentity currentUser); + protected abstract RangerAccessRequestImpl createRequest(AuthorizedSubject subject); protected abstract RangerAccessResourceImpl createResource(String ctl, String db, String tbl); diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java index 68a926f39cbe45..711375303bc2e3 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java @@ -17,9 +17,9 @@ package org.apache.doris.catalog.authorizer.ranger.doris; -import org.apache.doris.mysql.privilege.Privilege; +import org.apache.doris.authorization.AccessAction; -// Same as defined in PrivPredicate.java +// The access types a Doris Ranger service defines, one per privilege Doris grants. public enum DorisAccessType { NODE, ADMIN, @@ -32,32 +32,41 @@ public enum DorisAccessType { USAGE, SHOW_VIEW, NONE; - public static DorisAccessType toAccessType(Privilege privilege) { - switch (privilege) { - case ADMIN_PRIV: + + /** + * The access type standing for {@code action}. The three ways of using something are one access type + * here, which is a folding the Ranger service can afford and the engine cannot: its policies are written + * against the resource, so a policy on a compute group and one on a stage are told apart by what they are + * on rather than by the name of the privilege. + */ + public static DorisAccessType of(AccessAction action) { + switch (action) { + case ADMIN: return ADMIN; - case NODE_PRIV: + case NODE: return NODE; - case GRANT_PRIV: + case GRANT: return GRANT; - case SELECT_PRIV: + case SELECT: return SELECT; - case LOAD_PRIV: + case LOAD: return LOAD; - case ALTER_PRIV: + case ALTER: return ALTER; - case CREATE_PRIV: + case CREATE: return CREATE; - case DROP_PRIV: + case DROP: return DROP; - case USAGE_PRIV: - case STAGE_USAGE_PRIV: - case CLUSTER_USAGE_PRIV: + case USAGE: + case STAGE_USAGE: + case CLUSTER_USAGE: return USAGE; - case SHOW_VIEW_PRIV: + case SHOW_VIEW: return SHOW_VIEW; default: - return NONE; + // Guessing would ask Ranger about an access type its service definition does not have, and + // every such request is denied - an action added to Doris would silently stop being grantable. + throw new IllegalStateException("no Ranger access type for action " + action); } } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java index d02c9c7f2eb1a9..d32f5a7639b2e5 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java @@ -17,13 +17,15 @@ package org.apache.doris.catalog.authorizer.ranger.doris; -import org.apache.doris.analysis.ResourceTypeEnum; -import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; -import org.apache.doris.common.AuthorizationException; -import org.apache.doris.mysql.privilege.PrivBitSet; -import org.apache.doris.mysql.privilege.PrivPredicate; -import org.apache.doris.mysql.privilege.Privilege; import org.apache.doris.resource.workloadgroup.WorkloadGroupMgr; import com.google.common.annotations.VisibleForTesting; @@ -36,11 +38,21 @@ import org.apache.ranger.plugin.service.RangerAuthContextListener; import org.apache.ranger.plugin.service.RangerBasePlugin; +import java.util.Collections; import java.util.Date; -import java.util.Set; +import java.util.EnumSet; +import java.util.Map; +/** + * A Doris Ranger service: it answers about every kind of object Doris has, which is why it is also the one + * source that can be installed for the whole instance. + */ public class RangerDorisAccessController extends RangerAccessController { private static final Logger LOG = LogManager.getLogger(RangerDorisAccessController.class); + + /** The name this source is selected by, in {@code access_controller_type} and in catalog properties. */ + public static final String NAME = "ranger-doris"; + // ranger must set name, we agreed that this name must be used private static final String GLOBAL_PRIV_FIXED_NAME = "*"; @@ -49,11 +61,14 @@ public class RangerDorisAccessController extends RangerAccessController { // "ranger-doris-audit-log-flusher-timer", true); // private RangerHiveAuditHandler auditHandler; - public RangerDorisAccessController(String serviceName) { - this(serviceName, null); + public RangerDorisAccessController(String serviceName, Map properties, + AuthorizationContext context) { + this(serviceName, null, properties, context); } - public RangerDorisAccessController(String serviceName, RangerAuthContextListener rangerAuthContextListener) { + public RangerDorisAccessController(String serviceName, RangerAuthContextListener rangerAuthContextListener, + Map properties, AuthorizationContext context) { + super(properties, context); dorisPlugin = new RangerDorisPlugin(serviceName, rangerAuthContextListener); // auditHandler = new RangerHiveAuditHandler(dorisPlugin.getConfig()); // start a timed log flusher @@ -61,22 +76,120 @@ public RangerDorisAccessController(String serviceName, RangerAuthContextListener } @VisibleForTesting - public RangerDorisAccessController(RangerBasePlugin plugin) { + public RangerDorisAccessController(RangerBasePlugin plugin, AuthorizationContext context) { + this(plugin, Collections.emptyMap(), context); + } + + @VisibleForTesting + public RangerDorisAccessController(RangerBasePlugin plugin, Map properties, + AuthorizationContext context) { + super(properties, context); dorisPlugin = plugin; } - private RangerAccessRequestImpl createRequest(UserIdentity currentUser, DorisAccessType accessType) { - RangerAccessRequestImpl request = createRequest(currentUser); + @Override + public String name() { + return NAME; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + switch (resource.getKind()) { + case GLOBAL: + refuseUnless(checkGlobal(subject, requirement), subject, resource, requirement); + return; + case CATALOG: + refuseUnless(checkCatalog(subject, ((AuthorizedResource.Catalog) resource).getCatalog(), + requirement), subject, resource, requirement); + return; + case DATABASE: { + AuthorizedResource.Database database = (AuthorizedResource.Database) resource; + refuseUnless(checkDatabase(subject, database.getCatalog(), database.getDatabase(), requirement), + subject, resource, requirement); + return; + } + case TABLE: { + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + refuseUnless(checkTable(subject, table.getCatalog(), table.getDatabase(), table.getTable(), + requirement), subject, resource, requirement); + return; + } + case COLUMNS: + checkColumns(subject, (AuthorizedResource.Columns) resource, requirement); + return; + case RESOURCE: { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + refuseUnless(checkGlobalInternal(subject, requirement, granted) + || ask(subject, requirement, granted, new RangerDorisResource( + DorisObjectType.RESOURCE, named(resource))), + subject, resource, requirement); + return; + } + case WORKLOAD_GROUP: { + // For compatibility with older versions, it is not needed to check the privileges of the + // default group. + if (WorkloadGroupMgr.DEFAULT_GROUP_NAME.equals(named(resource))) { + return; + } + EnumSet granted = EnumSet.noneOf(AccessAction.class); + refuseUnless(checkGlobalInternal(subject, requirement, granted) + || ask(subject, requirement, granted, new RangerDorisResource( + DorisObjectType.WORKLOAD_GROUP, named(resource))), + subject, resource, requirement); + return; + } + case STORAGE_VAULT: { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + refuseUnless(checkGlobalInternal(subject, requirement, granted) + || ask(subject, requirement, granted, new RangerDorisResource( + DorisObjectType.STORAGE_VAULT, named(resource))), + subject, resource, requirement); + return; + } + case CLOUD_COMPUTE_GROUP: { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + refuseUnless(checkGlobalInternal(subject, requirement, granted) + || ask(subject, requirement, granted, new RangerDorisResource( + DorisObjectType.COMPUTE_GROUP, named(resource))), + subject, resource, requirement); + return; + } + case CLOUD_GENERAL: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + // A general cloud resource is asked about as a resource and a vault as a vault; a stage is + // reached through `copy into`, which is on its way out, so Ranger never governed it. + throw AccessDeniedException.of(subject, resource, requirement, NAME); + default: + throw new IllegalStateException("the Ranger Doris service has no answer for resource kind " + + resource.getKind()); + } + } + + private static String named(AuthorizedResource resource) { + return ((AuthorizedResource.Named) resource).getName(); + } + + private void refuseUnless(boolean allowed, AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement) throws AccessDeniedException { + if (!allowed) { + throw AccessDeniedException.of(subject, resource, requirement, NAME); + } + } + + private RangerAccessRequestImpl createRequest(AuthorizedSubject subject, DorisAccessType accessType) { + RangerAccessRequestImpl request = createRequest(subject); request.setAction(accessType.name()); request.setAccessType(accessType.name()); return request; } @Override - protected RangerAccessRequestImpl createRequest(UserIdentity currentUser) { + protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject) { RangerAccessRequestImpl request = new RangerAccessRequestImpl(); - request.setUser(currentUser.getQualifiedUser()); - request.setClientIPAddress(currentUser.getHost()); + request.setUser(subject.getUser()); + request.setClientIPAddress(subject.getHost()); request.setClusterType(CLIENT_TYPE_DORIS); request.setClientType(CLIENT_TYPE_DORIS); request.setAccessTime(new Date()); @@ -84,9 +197,9 @@ protected RangerAccessRequestImpl createRequest(UserIdentity currentUser) { return request; } - private boolean checkPrivilegeByPlugin(UserIdentity currentUser, DorisAccessType accessType, + private boolean checkPrivilegeByPlugin(AuthorizedSubject subject, DorisAccessType accessType, RangerDorisResource resource) { - RangerAccessRequestImpl request = createRequest(currentUser, accessType); + RangerAccessRequestImpl request = createRequest(subject, accessType); request.setResource(resource); if (LOG.isDebugEnabled()) { LOG.debug("ranger request: {}", request); @@ -95,8 +208,8 @@ private boolean checkPrivilegeByPlugin(UserIdentity currentUser, DorisAccessType return checkRequestResult(request, result, accessType.name()); } - private boolean checkShowPrivilegeByPlugin(UserIdentity currentUser, RangerDorisResource resource) { - RangerAccessRequestImpl request = createRequest(currentUser); + private boolean checkShowPrivilegeByPlugin(AuthorizedSubject subject, RangerDorisResource resource) { + RangerAccessRequestImpl request = createRequest(subject); request.setResource(resource); request.setResourceMatchingScope(ResourceMatchingScope.SELF_OR_DESCENDANTS); if (LOG.isDebugEnabled()) { @@ -106,215 +219,133 @@ private boolean checkShowPrivilegeByPlugin(UserIdentity currentUser, RangerDoris return checkRequestResult(request, result, DorisAccessType.NONE.name()); } - private boolean checkPrivilege(UserIdentity currentUser, PrivPredicate wanted, - RangerDorisResource resource, PrivBitSet checkedPrivs) { - PrivBitSet copy = wanted.getPrivs().copy(); - // avoid duplicate check auth at different levels - copy.remove(checkedPrivs); - for (Privilege privilege : copy.toPrivilegeList()) { - boolean res = checkPrivilegeByPlugin(currentUser, DorisAccessType.toAccessType(privilege), resource); - if (res) { - checkedPrivs.set(privilege.getIdx()); + /** + * Asks Ranger about the actions still outstanding, one request each, until the requirement is met. + * + *

{@code granted} carries what the levels already walked have granted, so a privilege answered on the + * catalog is not asked about again on the database and on the table: the same requirement checked down a + * three-level path costs one evaluation per action, not three. + */ + private boolean ask(AuthorizedSubject subject, AccessRequirement requirement, + EnumSet granted, RangerDorisResource resource) { + EnumSet outstanding = EnumSet.copyOf(requirement.getActions()); + outstanding.removeAll(granted); + for (AccessAction action : outstanding) { + if (checkPrivilegeByPlugin(subject, DorisAccessType.of(action), resource)) { + granted.add(action); } - if (Privilege.satisfy(checkedPrivs, wanted)) { + if (requirement.isSatisfiedBy(granted)) { return true; } } return false; } - @Override - public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { - PrivBitSet checkedPrivs = PrivBitSet.of(); - return checkGlobalPrivInternal(currentUser, wanted, checkedPrivs); + private boolean checkGlobal(AuthorizedSubject subject, AccessRequirement requirement) { + return checkGlobalInternal(subject, requirement, EnumSet.noneOf(AccessAction.class)); } - private boolean checkGlobalPrivInternal(UserIdentity currentUser, PrivPredicate wanted, PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.GLOBAL, GLOBAL_PRIV_FIXED_NAME); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); + private boolean checkGlobalInternal(AuthorizedSubject subject, AccessRequirement requirement, + EnumSet granted) { + return ask(subject, requirement, granted, + new RangerDorisResource(DorisObjectType.GLOBAL, GLOBAL_PRIV_FIXED_NAME)); } - @Override - public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - PrivBitSet checkedPrivs = PrivBitSet.of(); - if (checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs)) { - return true; - } - if (wanted == PrivPredicate.SHOW && checkAnyPrivWithinCtl(currentUser, ctl)) { + private boolean checkCatalog(AuthorizedSubject subject, String ctl, AccessRequirement requirement) { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + if (checkGlobalInternal(subject, requirement, granted) + || checkCatalogInternal(subject, ctl, requirement, granted)) { return true; } - return false; - } - - private boolean checkCtlPrivInternal(UserIdentity currentUser, String ctl, PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.CATALOG, ctl); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); + return isVisibility(requirement) && anyPrivilegeWithin(subject, + new RangerDorisResource(DorisObjectType.CATALOG, ctl)); } - private boolean checkAnyPrivWithinCtl(UserIdentity currentUser, String ctl) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.CATALOG, ctl); - return checkShowPrivilegeByPlugin(currentUser, resource); + private boolean checkCatalogInternal(AuthorizedSubject subject, String ctl, AccessRequirement requirement, + EnumSet granted) { + return ask(subject, requirement, granted, new RangerDorisResource(DorisObjectType.CATALOG, ctl)); } - @Override - public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + private boolean checkDatabase(AuthorizedSubject subject, String ctl, String db, + AccessRequirement requirement) { + if (grantedByGlobalScopeAuthority(subject, requirement)) { return true; } - PrivBitSet checkedPrivs = PrivBitSet.of(); - if (checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) - || checkDbPrivInternal(currentUser, ctl, db, wanted, checkedPrivs)) { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + if (checkGlobalInternal(subject, requirement, granted) + || checkCatalogInternal(subject, ctl, requirement, granted) + || checkDatabaseInternal(subject, ctl, db, requirement, granted)) { return true; } - if (wanted == PrivPredicate.SHOW && checkAnyPrivWithinDb(currentUser, ctl, db)) { - return true; - } - return false; + return isVisibility(requirement) && anyPrivilegeWithin(subject, + new RangerDorisResource(DorisObjectType.DATABASE, ctl, db)); } - private boolean checkDbPrivInternal(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.DATABASE, ctl, - db); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); + private boolean checkDatabaseInternal(AuthorizedSubject subject, String ctl, String db, + AccessRequirement requirement, EnumSet granted) { + return ask(subject, requirement, granted, new RangerDorisResource(DorisObjectType.DATABASE, ctl, db)); } - private boolean checkAnyPrivWithinDb(UserIdentity currentUser, String ctl, String db) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.DATABASE, ctl, - db); - return checkShowPrivilegeByPlugin(currentUser, resource); - } - - @Override - public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + private boolean checkTable(AuthorizedSubject subject, String ctl, String db, String tbl, + AccessRequirement requirement) { + if (grantedByGlobalScopeAuthority(subject, requirement)) { return true; } - PrivBitSet checkedPrivs = PrivBitSet.of(); - if (checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) - || checkDbPrivInternal(currentUser, ctl, db, wanted, checkedPrivs) - || checkTblPrivInternal(currentUser, ctl, db, tbl, wanted, checkedPrivs)) { + EnumSet granted = EnumSet.noneOf(AccessAction.class); + if (checkGlobalInternal(subject, requirement, granted) + || checkCatalogInternal(subject, ctl, requirement, granted) + || checkDatabaseInternal(subject, ctl, db, requirement, granted) + || checkTableInternal(subject, ctl, db, tbl, requirement, granted)) { return true; } - if (wanted == PrivPredicate.SHOW && checkAnyPrivWithinTbl(currentUser, ctl, db, tbl)) { - return true; - } - return false; + return isVisibility(requirement) && anyPrivilegeWithin(subject, + new RangerDorisResource(DorisObjectType.TABLE, ctl, db, tbl)); } - private boolean checkTblPrivInternal(UserIdentity currentUser, String ctl, String db, String tbl, - PrivPredicate wanted, PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.TABLE, - ctl, db, tbl); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); - } - - private boolean checkAnyPrivWithinTbl(UserIdentity currentUser, String ctl, String db, String tbl) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.TABLE, - ctl, db, tbl); - return checkShowPrivilegeByPlugin(currentUser, resource); + private boolean checkTableInternal(AuthorizedSubject subject, String ctl, String db, String tbl, + AccessRequirement requirement, EnumSet granted) { + return ask(subject, requirement, granted, new RangerDorisResource(DorisObjectType.TABLE, ctl, db, tbl)); } - @Override - public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, - PrivPredicate wanted) throws AuthorizationException { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { + private void checkColumns(AuthorizedSubject subject, AuthorizedResource.Columns columns, + AccessRequirement requirement) throws AccessDeniedException { + if (grantedByGlobalScopeAuthority(subject, requirement)) { return; } - PrivBitSet checkedPrivs = PrivBitSet.of(); - boolean hasTablePriv = checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkCtlPrivInternal(currentUser, ctl, wanted, checkedPrivs) - || checkDbPrivInternal(currentUser, ctl, db, wanted, checkedPrivs) - || checkTblPrivInternal(currentUser, ctl, db, tbl, wanted, checkedPrivs); + String ctl = columns.getCatalog(); + String db = columns.getDatabase(); + String tbl = columns.getTable(); + EnumSet granted = EnumSet.noneOf(AccessAction.class); + boolean hasTablePriv = checkGlobalInternal(subject, requirement, granted) + || checkCatalogInternal(subject, ctl, requirement, granted) + || checkDatabaseInternal(subject, ctl, db, requirement, granted) + || checkTableInternal(subject, ctl, db, tbl, requirement, granted); if (hasTablePriv) { return; } - for (String col : cols) { - if (!checkColPrivInternal(currentUser, ctl, db, tbl, col, wanted, checkedPrivs.copy())) { - throw new AuthorizationException(String.format( + for (String col : columns.getColumns()) { + // Each column starts from what the levels above granted, and none of them may add to it: a + // privilege held on one column says nothing about the next one. + if (!ask(subject, requirement, EnumSet.copyOf(granted), + new RangerDorisResource(DorisObjectType.COLUMN, ctl, db, tbl, col))) { + throw AccessDeniedException.withMessage(String.format( "Permission denied: user [%s] does not have privilege for [%s] command on [%s].[%s].[%s].[%s]", - currentUser, wanted, ctl, db, tbl, col)); + subject, requirement, ctl, db, tbl, col), columns, NAME); } } } - private boolean checkColPrivInternal(UserIdentity currentUser, String ctl, String db, String tbl, String col, - PrivPredicate wanted, PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.COLUMN, - ctl, db, tbl, col); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); - } - - @Override - public boolean checkCloudPriv(UserIdentity currentUser, String cloudName, - PrivPredicate wanted, ResourceTypeEnum type) { - // only support CLUSTER, - // STORAGE_VAULT should call `checkStorageVaultPriv` - // GENERAL should call `checkResourcePriv` - // STAGE is used to support `copy into`, but this feature will soon expire, - // so it is no longer supported through Ranger - if (!ResourceTypeEnum.CLUSTER.equals(type)) { - return false; - } - PrivBitSet checkedPrivs = PrivBitSet.of(); - return checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkComputeGroupPrivInternal(currentUser, cloudName, wanted, checkedPrivs); - } - - private boolean checkComputeGroupPrivInternal(UserIdentity currentUser, String computeGroupName, - PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.COMPUTE_GROUP, computeGroupName); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); + /** + * Whether this is the question "may the subject see the object at all", which Ranger answers by looking + * for any policy anywhere under it rather than for a privilege on it. + */ + private static boolean isVisibility(AccessRequirement requirement) { + return AccessRequirements.VISIBILITY.equals(requirement); } - @Override - public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, PrivPredicate wanted) { - PrivBitSet checkedPrivs = PrivBitSet.of(); - return checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkStorageVaultPrivInternal(currentUser, storageVaultName, wanted, checkedPrivs); - } - - private boolean checkStorageVaultPrivInternal(UserIdentity currentUser, String storageVaultName, - PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.STORAGE_VAULT, storageVaultName); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); - } - - @Override - public boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted) { - PrivBitSet checkedPrivs = PrivBitSet.of(); - return checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkResourcePrivInternal(currentUser, resourceName, wanted, checkedPrivs); - } - - private boolean checkResourcePrivInternal(UserIdentity currentUser, String resourceName, PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.RESOURCE, resourceName); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); - } - - @Override - public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadGroupName, PrivPredicate wanted) { - // For compatibility with older versions, it is not needed to check the privileges of the default group. - if (WorkloadGroupMgr.DEFAULT_GROUP_NAME.equals(workloadGroupName)) { - return true; - } - PrivBitSet checkedPrivs = PrivBitSet.of(); - return checkGlobalPrivInternal(currentUser, wanted, checkedPrivs) - || checkWorkloadGroupInternal(currentUser, workloadGroupName, wanted, checkedPrivs); - } - - private boolean checkWorkloadGroupInternal(UserIdentity currentUser, String workloadGroupName, PrivPredicate wanted, - PrivBitSet checkedPrivs) { - RangerDorisResource resource = new RangerDorisResource(DorisObjectType.WORKLOAD_GROUP, workloadGroupName); - return checkPrivilege(currentUser, wanted, resource, checkedPrivs); + private boolean anyPrivilegeWithin(AuthorizedSubject subject, RangerDorisResource resource) { + return checkShowPrivilegeByPlugin(subject, resource); } @Override @@ -338,17 +369,4 @@ protected RangerBasePlugin getPlugin() { protected RangerAccessResultProcessor getAccessResultProcessor() { return null; } - - // For test only - public static void main(String[] args) { - RangerDorisAccessController ac = new RangerDorisAccessController("doris"); - UserIdentity user = new UserIdentity("user1", "127.0.0.1"); - user.setIsAnalyzed(); - boolean res = ac.checkDbPriv(user, "internal", "db1", PrivPredicate.SHOW); - System.out.println("res: " + res); - user = new UserIdentity("user2", "127.0.0.1"); - user.setIsAnalyzed(); - res = ac.checkTblPriv(user, "internal", "db1", "tbl1", PrivPredicate.SELECT); - System.out.println("res: " + res); - } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java index e42464d3f56e5c..d6f1bc9e7a30e8 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java @@ -17,17 +17,19 @@ package org.apache.doris.catalog.authorizer.ranger.hive; -import org.apache.doris.analysis.ResourceTypeEnum; -import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.RowFilterSpec; -import org.apache.doris.catalog.Env; +import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; -import org.apache.doris.common.AuthorizationException; import org.apache.doris.common.ThreadPoolManager; -import org.apache.doris.mysql.privilege.PrivPredicate; -import com.google.common.collect.Maps; +import com.google.common.annotations.VisibleForTesting; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.ranger.plugin.policyengine.RangerAccessRequest; @@ -40,21 +42,30 @@ import java.util.ArrayList; import java.util.Collection; +import java.util.Collections; import java.util.Date; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.Set; import java.util.concurrent.ScheduledFuture; import java.util.concurrent.ScheduledThreadPoolExecutor; import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.ReentrantReadWriteLock; -import java.util.stream.Collectors; +/** + * A Hive Ranger service governing one catalog: it knows databases, tables and columns, and nothing else Doris + * has. What it is not asked about it refuses, except for the two kinds it has to let through for a catalog + * bound to it to be usable at all - the catalog itself, and workload groups, neither of which a Hive service + * has policies for. + */ public class RangerHiveAccessController extends RangerAccessController { private static final Logger LOG = LogManager.getLogger(RangerHiveAccessController.class); private static final ScheduledThreadPoolExecutor LOG_FLUSH_TIMER = ThreadPoolManager.newDaemonScheduledThreadPool(1, "ranger-hive-audit-log-flusher-timer", true); + + /** The name this source is selected by in catalog properties. */ + public static final String NAME = "ranger-hive"; + private RangerHivePlugin hivePlugin; private RangerHiveAuditHandler auditHandler; private ScheduledFuture logFlushFuture; @@ -63,12 +74,13 @@ public class RangerHiveAccessController extends RangerAccessController { private final ReentrantReadWriteLock lifecycleLock = new ReentrantReadWriteLock(); private boolean closed; - public RangerHiveAccessController(Map properties) { - this(properties, null); + public RangerHiveAccessController(Map properties, AuthorizationContext context) { + this(properties, null, context); } public RangerHiveAccessController(Map properties, - RangerAuthContextListener rangerAuthContextListener) { + RangerAuthContextListener rangerAuthContextListener, AuthorizationContext context) { + super(properties, context); String serviceName = properties.get("ranger.service.name"); hivePlugin = new RangerHivePlugin(serviceName, rangerAuthContextListener); auditHandler = new RangerHiveAuditHandler(hivePlugin.getConfig()); @@ -77,6 +89,11 @@ public RangerHiveAccessController(Map properties, new RangerHiveAuditLogFlusher(auditHandler), 10, 20L, TimeUnit.SECONDS); } + @Override + public String name() { + return NAME; + } + @Override public void close() { lifecycleLock.writeLock().lock(); @@ -110,8 +127,85 @@ public void close() { } } - private RangerAccessRequestImpl createRequest(UserIdentity currentUser, HiveAccessType accessType) { - RangerAccessRequestImpl request = createRequest(currentUser); + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + switch (resource.getKind()) { + case GLOBAL: + // A Hive service has no notion of an instance-wide privilege, so this is entirely whoever + // governs instance scope. Installed as that authority itself, it grants nothing: it would + // otherwise be asking itself, and this configuration is not one a Hive service can serve. + refuseUnless(grantedByGlobalScopeAuthority(subject, requirement), subject, resource, requirement); + return; + case CATALOG: + // The catalog is the thing bound to this service; the policies are about what is inside it. + return; + case DATABASE: { + AuthorizedResource.Database database = (AuthorizedResource.Database) resource; + refuseUnless(checkResource(subject, requirement, + new RangerHiveResource(HiveObjectType.DATABASE, database.getDatabase())), + subject, resource, requirement); + return; + } + case TABLE: { + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + refuseUnless(checkResource(subject, requirement, + new RangerHiveResource(HiveObjectType.TABLE, table.getDatabase(), table.getTable())), + subject, resource, requirement); + return; + } + case COLUMNS: + checkColumns(subject, (AuthorizedResource.Columns) resource, requirement); + return; + case WORKLOAD_GROUP: + // Not support workload group privilege in ranger hive plugin. + // So always allow to pass the check + return; + case RESOURCE: + case STORAGE_VAULT: + case CLOUD_GENERAL: + case CLOUD_COMPUTE_GROUP: + case CLOUD_STAGE: + case CLOUD_STORAGE_VAULT: + throw AccessDeniedException.of(subject, resource, requirement, NAME); + default: + throw new IllegalStateException("the Ranger Hive service has no answer for resource kind " + + resource.getKind()); + } + } + + private void refuseUnless(boolean allowed, AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement) throws AccessDeniedException { + if (!allowed) { + throw AccessDeniedException.of(subject, resource, requirement, NAME); + } + } + + private boolean checkResource(AuthorizedSubject subject, AccessRequirement requirement, + RangerHiveResource resource) { + if (grantedByGlobalScopeAuthority(subject, requirement)) { + return true; + } + return checkPrivilege(subject, accessTypeOf(requirement), resource); + } + + private void checkColumns(AuthorizedSubject subject, AuthorizedResource.Columns columns, + AccessRequirement requirement) throws AccessDeniedException { + if (grantedByGlobalScopeAuthority(subject, requirement)) { + return; + } + List resources = new ArrayList<>(); + for (String col : columns.getColumns()) { + RangerHiveResource resource = new RangerHiveResource(HiveObjectType.COLUMN, + columns.getDatabase(), columns.getTable(), col); + resources.add(resource); + } + + checkPrivileges(subject, accessTypeOf(requirement), resources, columns); + } + + private RangerAccessRequestImpl createRequest(AuthorizedSubject subject, HiveAccessType accessType) { + RangerAccessRequestImpl request = createRequest(subject); if (accessType == HiveAccessType.USE) { request.setAccessType(RangerPolicyEngine.ANY_ACCESS); } else { @@ -121,14 +215,13 @@ private RangerAccessRequestImpl createRequest(UserIdentity currentUser, HiveAcce } @Override - protected RangerAccessRequestImpl createRequest(UserIdentity currentUser) { + protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject) { RangerAccessRequestImpl request = new RangerAccessRequestImpl(); - String user = currentUser.getQualifiedUser(); - request.setUser(user); - Set roles = Env.getCurrentEnv().getAuth().getRolesByUser(currentUser, false); - request.setUserRoles(roles.stream().collect( - Collectors.toSet())); - request.setClientIPAddress(currentUser.getHost()); + request.setUser(subject.getUser()); + // Policies in Ranger may be written against a role rather than a user, and the roles a Doris account + // holds are the engine's to know, not this service's. + request.setUserRoles(getContext().rolesOf(subject)); + request.setClientIPAddress(subject.getHost()); request.setClusterType(CLIENT_TYPE_DORIS); request.setClientType(CLIENT_TYPE_DORIS); request.setAccessTime(new Date()); @@ -136,35 +229,36 @@ protected RangerAccessRequestImpl createRequest(UserIdentity currentUser) { return request; } - private void checkPrivileges(UserIdentity currentUser, HiveAccessType accessType, - List hiveResources) throws AuthorizationException { + private void checkPrivileges(AuthorizedSubject subject, HiveAccessType accessType, + List hiveResources, AuthorizedResource asked) throws AccessDeniedException { lifecycleLock.readLock().lock(); try { if (closed) { - throw new AuthorizationException("Ranger Hive access controller has been closed"); + throw AccessDeniedException.withMessage("Ranger Hive access controller has been closed", + asked, NAME); } List requests = new ArrayList<>(); for (RangerHiveResource resource : hiveResources) { - RangerAccessRequestImpl request = createRequest(currentUser, accessType); + RangerAccessRequestImpl request = createRequest(subject, accessType); request.setResource(resource); requests.add(request); } Collection results = hivePlugin.isAccessAllowed(requests, auditHandler); - checkRequestResults(results, accessType.name()); + checkRequestResults(results, accessType.name(), asked); } finally { lifecycleLock.readLock().unlock(); } } - private boolean checkPrivilege(UserIdentity currentUser, HiveAccessType accessType, + private boolean checkPrivilege(AuthorizedSubject subject, HiveAccessType accessType, RangerHiveResource resource) { lifecycleLock.readLock().lock(); try { if (closed) { return false; } - RangerAccessRequestImpl request = createRequest(currentUser, accessType); + RangerAccessRequestImpl request = createRequest(subject, accessType); request.setResource(resource); RangerAccessResult result = hivePlugin.isAccessAllowed(request, auditHandler); @@ -174,20 +268,29 @@ private boolean checkPrivilege(UserIdentity currentUser, HiveAccessType accessTy } } - private HiveAccessType convertToAccessType(PrivPredicate predicate) { - if (predicate == PrivPredicate.SHOW) { + /** + * The Hive access type standing for the question being asked. + * + *

Only the questions the engine asks by name map onto one; anything else - a requirement assembled for + * one statement, say - is deliberately {@link HiveAccessType#NONE}, which no Hive policy grants. Guessing + * an access type from an unrecognised set of actions would grant on a policy written for something else. + */ + @VisibleForTesting + static HiveAccessType accessTypeOf(AccessRequirement requirement) { + if (AccessRequirements.VISIBILITY.equals(requirement)) { return HiveAccessType.USE; - } else if (predicate == PrivPredicate.SELECT) { + } else if (AccessRequirements.SELECT.equals(requirement)) { return HiveAccessType.SELECT; - } else if (predicate == PrivPredicate.ADMIN || predicate == PrivPredicate.ALL) { + } else if (AccessRequirements.ADMINISTRATION.equals(requirement) + || AccessRequirements.ANY_PRIVILEGE.equals(requirement)) { return HiveAccessType.ALL; - } else if (predicate == PrivPredicate.LOAD) { + } else if (AccessRequirements.LOAD.equals(requirement)) { return HiveAccessType.UPDATE; - } else if (predicate == PrivPredicate.ALTER) { + } else if (AccessRequirements.ALTER.equals(requirement)) { return HiveAccessType.ALTER; - } else if (predicate == PrivPredicate.CREATE) { + } else if (AccessRequirements.CREATE.equals(requirement)) { return HiveAccessType.CREATE; - } else if (predicate == PrivPredicate.DROP) { + } else if (AccessRequirements.DROP.equals(requirement)) { return HiveAccessType.DROP; } else { return HiveAccessType.NONE; @@ -195,93 +298,23 @@ private HiveAccessType convertToAccessType(PrivPredicate predicate) { } @Override - public boolean checkGlobalPriv(UserIdentity currentUser, PrivPredicate wanted) { - // hive ranger plugin does not support global privilege - // use whichever authorization source governs global scope - return Env.getCurrentEnv().getAccessManager().checkGlobalPriv(currentUser, wanted); - } - - @Override - public boolean checkCtlPriv(UserIdentity currentUser, String ctl, PrivPredicate wanted) { - return true; - } - - @Override - public boolean checkDbPriv(UserIdentity currentUser, String ctl, String db, PrivPredicate wanted) { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { - return true; - } - RangerHiveResource resource = new RangerHiveResource(HiveObjectType.DATABASE, - db); - return checkPrivilege(currentUser, convertToAccessType(wanted), resource); - } - - @Override - public boolean checkTblPriv(UserIdentity currentUser, String ctl, String db, String tbl, PrivPredicate wanted) { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { - return true; - } - RangerHiveResource resource = new RangerHiveResource(HiveObjectType.TABLE, - db, tbl); - return checkPrivilege(currentUser, convertToAccessType(wanted), resource); - } - - @Override - public void checkColsPriv(UserIdentity currentUser, String ctl, String db, String tbl, Set cols, - PrivPredicate wanted) throws AuthorizationException { - if (grantedByGlobalScopeAuthority(currentUser, wanted)) { - return; - } - List resources = new ArrayList<>(); - for (String col : cols) { - RangerHiveResource resource = new RangerHiveResource(HiveObjectType.COLUMN, - db, tbl, col); - resources.add(resource); - } - - checkPrivileges(currentUser, convertToAccessType(wanted), resources); - } - - @Override - public boolean checkCloudPriv(UserIdentity currentUser, String cloudName, - PrivPredicate wanted, ResourceTypeEnum type) { - return false; - } - - @Override - public boolean checkStorageVaultPriv(UserIdentity currentUser, String storageVaultName, PrivPredicate wanted) { - return false; - } - - @Override - public boolean checkResourcePriv(UserIdentity currentUser, String resourceName, PrivPredicate wanted) { - return false; - } - - @Override - public boolean checkWorkloadGroupPriv(UserIdentity currentUser, String workloadGroupName, PrivPredicate wanted) { - // Not support workload group privilege in ranger hive plugin. - // So always return true to pass the check - return true; - } - - @Override - public List evalRowFilterPolicies(UserIdentity currentUser, String ctl, String db, - String tbl) { + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext context) { lifecycleLock.readLock().lock(); try { - return closed ? new ArrayList<>() : super.evalRowFilterPolicies(currentUser, ctl, db, tbl); + return closed ? new ArrayList<>() : super.getRowFilters(subject, table, context); } finally { lifecycleLock.readLock().unlock(); } } @Override - public Optional evalDataMaskPolicy(UserIdentity currentUser, String ctl, String db, String tbl, - String col) { + public Map getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table, + Set columns, AccessContext context) { lifecycleLock.readLock().lock(); try { - return closed ? Optional.empty() : super.evalDataMaskPolicy(currentUser, ctl, db, tbl, col); + return closed ? Collections.emptyMap() + : super.getDataMasks(subject, table, columns, context); } finally { lifecycleLock.readLock().unlock(); } @@ -308,17 +341,4 @@ protected RangerBasePlugin getPlugin() { protected RangerAccessResultProcessor getAccessResultProcessor() { return auditHandler; } - - // For test only - public static void main(String[] args) { - Map properties = Maps.newHashMap(); - properties.put("ranger.service.name", "hive"); - RangerHiveAccessController ac = new RangerHiveAccessController(properties); - UserIdentity user = new UserIdentity("user1", "127.0.0.1"); - user.setIsAnalyzed(); - boolean res = ac.checkDbPriv(user, "hive", "tpcds_bin_partitioned_orc_1", PrivPredicate.SHOW); - System.out.println("res: " + res); - res = ac.checkTblPriv(user, "internal", "tpch1", "customer", PrivPredicate.SELECT); - System.out.println("res: " + res); - } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java index a45632ff9e619c..15613f4567cc65 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java @@ -17,20 +17,26 @@ package org.apache.doris.catalog.authorizer.ranger.hive; -import org.apache.doris.mysql.privilege.AccessControllerFactory; -import org.apache.doris.mysql.privilege.CatalogAccessController; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; import java.util.Map; -public class RangerHiveAccessControllerFactory implements AccessControllerFactory { +public class RangerHiveAccessControllerFactory implements AuthorizationPluginFactory { @Override - public String factoryIdentifier() { - return "ranger-hive"; + public String name() { + return RangerHiveAccessController.NAME; } @Override - public CatalogAccessController createAccessController(Map prop) { - return new RangerHiveAccessController(prop); + public String description() { + return "Authorizes one catalog against the policies of a Ranger service of type hive"; + } + + @Override + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return new RangerHiveAccessController(properties, context); } } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index b521f4180b87af..b3d055110203bb 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -26,6 +26,7 @@ import org.apache.doris.authorization.ResourceKind; import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; import org.apache.doris.catalog.AuthorizationInfo; import org.apache.doris.catalog.Env; import org.apache.doris.catalog.info.TableNameInfo; @@ -76,6 +77,9 @@ public class AccessControllerManager { // A catalog name can be reused after DROP. Keep the catalog id next to the source so cleanup from // an old catalog generation can never remove or close the replacement generation's source. private Map ctlToCtlAccessController = Maps.newConcurrentMap(); + // Factories publishing an authorization source under the current contract, by the name it is selected by + private ConcurrentHashMap authorizationPluginFactories + = new ConcurrentHashMap<>(); // Cache of loaded access controller factories for quick creation of new access controllers private ConcurrentHashMap accessControllerFactoriesCache = new ConcurrentHashMap<>(); @@ -111,19 +115,37 @@ private AuthorizationPlugin loadAccessControllerOrThrow(String accessControllerN if (accessControllerName.equalsIgnoreCase(InternalAuthorizationPlugin.NAME)) { return new InternalAuthorizationPlugin(auth); } - if (accessControllerFactoriesCache.containsKey(accessControllerName)) { - Map prop; - try { - prop = PropertiesUtils.loadAccessControllerPropertiesOrNull(); - } catch (IOException e) { - throw new RuntimeException("Failed to load authorization properties." - + "Please check the configuration file, authorization name is " + accessControllerName, e); - } - return adapt(accessControllerName, - accessControllerFactoriesCache.get(accessControllerName).createAccessController(prop)); + if (!isKnownAuthorizationSource(accessControllerName)) { + throw new RuntimeException("No authorization plugin factory found for " + accessControllerName + + ". Please confirm that your plugin is placed in the correct location."); + } + Map prop; + try { + prop = PropertiesUtils.loadAccessControllerPropertiesOrNull(); + } catch (IOException e) { + throw new RuntimeException("Failed to load authorization properties." + + "Please check the configuration file, authorization name is " + accessControllerName, e); } - throw new RuntimeException("No authorization plugin factory found for " + accessControllerName - + ". Please confirm that your plugin is placed in the correct location."); + return create(accessControllerName, prop); + } + + /** + * Builds the authorization source published under {@code name}, whichever contract publishes it. + * + *

A source written against the current contract is built with the context it may put questions to the + * engine through. It cannot be handed that context any earlier than this - the context has to name the + * source it belongs to, and the source does not exist until its factory has run. + */ + private AuthorizationPlugin create(String name, Map properties) { + AuthorizationPluginFactory factory = authorizationPluginFactories.get(name); + if (factory == null) { + return adapt(name, accessControllerFactoriesCache.get(name).createAccessController(properties)); + } + EngineAuthorizationContext context = new EngineAuthorizationContext(this, auth); + AuthorizationPlugin plugin = factory.create( + properties == null ? Collections.emptyMap() : properties, context); + context.servedBy(plugin); + return plugin; } /** Presents a controller written against the older per-scope interface as an authorization source. */ @@ -131,12 +153,22 @@ private AuthorizationPlugin adapt(String name, CatalogAccessController controlle return new LegacyAccessControllerPlugin(name, controller); } + private boolean isKnownAuthorizationSource(String name) { + return authorizationPluginFactories.containsKey(name) || accessControllerFactoriesCache.containsKey(name); + } + private void loadAccessControllerPlugins() { + // Sources shipped with the FE, and any on its class path. Loading them from a plugin directory is + // what the version gate guards, so that channel opens with the gate rather than before it. + for (AuthorizationPluginFactory factory : ServiceLoader.load(AuthorizationPluginFactory.class)) { + LOG.info("Found authorization plugin factory: {} from class path.", factory.name()); + authorizationPluginFactories.put(factory.name(), factory); + accessControllerClassNameMapping.put(factory.getClass().getName(), factory.name()); + } ServiceLoader loaderFromClasspath = ServiceLoader.load(AccessControllerFactory.class); for (AccessControllerFactory factory : loaderFromClasspath) { LOG.info("Found Authentication Plugin Factories: {} from class path.", factory.factoryIdentifier()); - accessControllerFactoriesCache.put(factory.factoryIdentifier(), factory); - accessControllerClassNameMapping.put(factory.getClass().getName(), factory.factoryIdentifier()); + registerLegacyFactory(factory); } List loader = null; try { @@ -146,11 +178,21 @@ private void loadAccessControllerPlugins() { } for (AccessControllerFactory factory : loader) { LOG.info("Found Access Controller Plugin Factory: {} from directory.", factory.factoryIdentifier()); - accessControllerFactoriesCache.put(factory.factoryIdentifier(), factory); - accessControllerClassNameMapping.put(factory.getClass().getName(), factory.factoryIdentifier()); + registerLegacyFactory(factory); } } + private void registerLegacyFactory(AccessControllerFactory factory) { + String name = factory.factoryIdentifier(); + if (authorizationPluginFactories.containsKey(name)) { + // Both were found, so say which one answers rather than letting the loser look installed. + LOG.warn("Authorization source {} is published both as a plugin and as an access controller" + + " factory; the plugin is the one used.", name); + } + accessControllerFactoriesCache.put(name, factory); + accessControllerClassNameMapping.put(factory.getClass().getName(), name); + } + /** The authorization source governing the objects inside {@code ctl}. */ public AuthorizationPlugin getAccessControllerOrDefault(String ctl) { if (InternalCatalog.INTERNAL_CATALOG_NAME.equals(ctl)) { @@ -223,8 +265,7 @@ public boolean checkIfAccessControllerExist(String ctl) { public void createAccessController(ExternalCatalog catalog, String acFactoryClassName, Map prop, boolean isDryRun) { String pluginIdentifier = getPluginIdentifierForAccessController(acFactoryClassName); - AuthorizationPlugin accessController = adapt(pluginIdentifier, - accessControllerFactoriesCache.get(pluginIdentifier).createAccessController(prop)); + AuthorizationPlugin accessController = create(pluginIdentifier, prop); if (isDryRun) { closeAccessController(catalog.getName(), accessController); return; @@ -264,10 +305,10 @@ private String getPluginIdentifierForAccessController(String acClassName) { if (accessControllerClassNameMapping.containsKey(acClassName)) { pluginIdentifier = accessControllerClassNameMapping.get(acClassName); } - if (accessControllerFactoriesCache.containsKey(acClassName)) { + if (isKnownAuthorizationSource(acClassName)) { pluginIdentifier = acClassName; } - if (null == pluginIdentifier || !accessControllerFactoriesCache.containsKey(pluginIdentifier)) { + if (null == pluginIdentifier || !isKnownAuthorizationSource(pluginIdentifier)) { throw new RuntimeException("Access Controller Plugin Factory not found for " + acClassName); } return pluginIdentifier; @@ -404,18 +445,12 @@ private AuthorizationPlugin systemScopeController() { /** * Whether {@code candidate} is itself the source governing instance scope. * - *

Asked by a source that would otherwise defer to that authority, so that it does not ask itself a - * question it is about to answer - the two would agree, at the price of evaluating the same policies - * twice. Identity is the question, so a controller reached through an adapter is compared against the - * controller, not against its wrapper. + *

Asked on behalf of a source that would otherwise defer to that authority, so that it does not ask + * itself a question it is about to answer - the two would agree, at the price of evaluating the same + * policies twice. */ - public boolean isGlobalScopeAuthority(Object candidate) { - AuthorizationPlugin authority = systemScopeController(); - if (authority == candidate) { - return true; - } - return authority instanceof LegacyAccessControllerPlugin - && ((LegacyAccessControllerPlugin) authority).getController() == candidate; + boolean isGlobalScopeAuthority(AuthorizationPlugin candidate) { + return systemScopeController() == candidate; } // ==== Global ==== diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/EngineAuthorizationContext.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/EngineAuthorizationContext.java new file mode 100644 index 00000000000000..d75cda0cbd67fe --- /dev/null +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/EngineAuthorizationContext.java @@ -0,0 +1,83 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; + +import com.google.common.base.Preconditions; + +import java.util.Objects; +import java.util.Set; + +/** + * What the engine answers when an authorization source asks it something. + * + *

One of these is created per source, so that a question whose answer depends on who is asking - "does + * somebody else already grant this at instance scope?" - can be answered without the source having to + * identify itself on every call. {@link AccessControllerManager} builds them; nothing else needs to, beyond + * a test standing a source up the way the manager would. + */ +public class EngineAuthorizationContext implements AuthorizationContext { + + private final AccessControllerManager manager; + private final Auth auth; + /** + * The source this context serves. Assigned once the factory has built it, which is necessarily after this + * context exists: the factory needs the context to build the source with. + */ + private volatile AuthorizationPlugin servedSource; + + public EngineAuthorizationContext(AccessControllerManager manager, Auth auth) { + this.manager = Objects.requireNonNull(manager, "manager is required"); + this.auth = Objects.requireNonNull(auth, "auth is required"); + } + + /** Records which source this context belongs to; called by the manager right after the source is built. */ + public void servedBy(AuthorizationPlugin source) { + this.servedSource = Objects.requireNonNull(source, "source is required"); + } + + /** + * {@inheritDoc} + * + *

Without the per-user default role: it is an artefact of how the built-in model stores a user's own + * grants, not a role anybody names in a policy, and a source that saw it would be matching on a name the + * administrator never wrote. + */ + @Override + public Set rolesOf(AuthorizedSubject subject) { + return auth.getRolesByUser(AccessTranslation.userIdentityOf(subject), false); + } + + @Override + public boolean grantedByGlobalScopeAuthority(AuthorizedSubject subject, AccessRequirement requirement) { + Preconditions.checkState(servedSource != null, + "an authorization source asked about global scope before the engine knew which source it is"); + if (manager.isGlobalScopeAuthority(servedSource)) { + // Asking would route straight back to the source asking, which is about to answer the same + // question from the same policies. Same verdict, twice the evaluations. + return false; + } + return manager.decide(AccessTranslation.userIdentityOf(subject), AuthorizedResource.global(), + requirement); + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java index 4557cd87ee93ad..7255a328b2264c 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/LegacyAccessControllerPlugin.java @@ -54,11 +54,7 @@ public LegacyAccessControllerPlugin(String name, CatalogAccessController control this.controller = Objects.requireNonNull(controller, "controller is required"); } - /** - * The controller this presents. Needed where an identity, not a behaviour, is the question - a controller - * asking whether it is itself the one governing instance scope has to compare against the object it is, - * not against the wrapper it is reached through. - */ + /** The controller this presents, for where the controller itself is the question rather than its answers. */ public CatalogAccessController getController() { return controller; } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java index 7e23769911520c..b63a344debcf94 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java @@ -17,23 +17,48 @@ package org.apache.doris.mysql.privilege; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; + import java.util.Map; -public class RangerDorisAccessControllerFactory implements AccessControllerFactory { - private static class SingletonHolder { - // Every controller starts a Ranger policy refresher, so all Env instances must share one controller. - private static final RangerDorisAccessController INSTANCE = new RangerDorisAccessController("doris"); +public class RangerDorisAccessControllerFactory implements AuthorizationPluginFactory { + private static final Logger LOG = LogManager.getLogger(RangerDorisAccessControllerFactory.class); + private static final String SERVICE_NAME = "doris"; + + // Every controller starts a Ranger policy refresher, so all Env instances must share one controller. + private static RangerDorisAccessController instance; + + @Override + public String name() { + return RangerDorisAccessController.NAME; } @Override - public String factoryIdentifier() { - return "ranger-doris"; + public String description() { + return "Authorizes against the policies of a Ranger service of type " + SERVICE_NAME; } @Override - public RangerDorisAccessController createAccessController(Map prop) { - return SingletonHolder.INSTANCE; + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return singleton(properties, context); + } + + private static synchronized RangerDorisAccessController singleton(Map properties, + AuthorizationContext context) { + if (instance == null) { + instance = new RangerDorisAccessController(SERVICE_NAME, properties, context); + } else if (!properties.isEmpty()) { + // There is one refresher and therefore one controller, so the configuration it was built with is + // the one in force. Say so rather than let a second, differently configured binding look applied. + LOG.warn("Ranger Doris authorization is already configured; properties {} are ignored, the" + + " configuration the source was created with stays in force.", properties.keySet()); + } + return instance; } } diff --git a/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory b/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory similarity index 98% rename from fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory rename to fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory index e2100cb8b23508..6d19de0830694a 100644 --- a/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory +++ b/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory @@ -16,4 +16,4 @@ # # org.apache.doris.mysql.privilege.RangerDorisAccessControllerFactory -org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory \ No newline at end of file +org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java index a302d56667873e..eec85525daf9a1 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerGlobalScopeDeferenceTest.java @@ -17,39 +17,44 @@ package org.apache.doris.catalog.authorizer.ranger; -import org.apache.doris.analysis.UserIdentity; -import org.apache.doris.catalog.Env; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.apache.doris.common.jmockit.Deencapsulation; import org.apache.doris.mysql.privilege.AccessControllerManager; import org.apache.doris.mysql.privilege.Auth; -import org.apache.doris.mysql.privilege.CatalogAccessController; -import org.apache.doris.mysql.privilege.LegacyAccessControllerPlugin; -import org.apache.doris.mysql.privilege.PrivPredicate; +import org.apache.doris.mysql.privilege.EngineAuthorizationContext; import org.apache.doris.mysql.privilege.StubRangerPolicyEngine; import org.apache.ranger.plugin.policyengine.RangerAccessRequest; import org.apache.ranger.plugin.policyengine.RangerAccessResult; import org.junit.Assert; import org.junit.Test; -import org.mockito.MockedStatic; -import org.mockito.Mockito; +import java.util.Collections; +import java.util.Map; import java.util.concurrent.atomic.AtomicInteger; -import java.util.function.BooleanSupplier; /** - * A catalog bound to Ranger still lets through whoever holds the privilege at global scope. + * A catalog bound to Ranger still lets through whoever holds the privilege at instance scope. * - *

Global scope is not something a Ranger service knows about; it belongs to the controller named by + *

Instance scope is not something a Ranger service knows about; it belongs to the source named by * {@code access_controller_type}. Honouring it is why a cluster administrator can still reach a Ranger-governed - * catalog after the engine stopped establishing privileges on a plugin's behalf. It is the plugin's own choice, - * so it is tested on the plugin, and a plugin that declines it stays a legal plugin. + * catalog after the engine stopped establishing privileges on a source's behalf. It is the source's own choice, + * so it is tested on the source, it is configurable, and a source that declines it stays a legal source. */ public class RangerGlobalScopeDeferenceTest { - private static final UserIdentity ADMIN = UserIdentity.createAnalyzedUserIdentWithIp("admin_user", "%"); - private static final UserIdentity RANGER_USER = - UserIdentity.createAnalyzedUserIdentWithIp(StubRangerPolicyEngine.ALLOWED_USER, "%"); + private static final AuthorizedSubject ADMIN = AuthorizedSubject.of("admin_user", "%"); + private static final AuthorizedSubject RANGER_USER = + AuthorizedSubject.of(StubRangerPolicyEngine.ALLOWED_USER, "%"); + private static final AuthorizedResource.Table ALLOWED_TABLE = AuthorizedResource.table("ctl", + StubRangerPolicyEngine.ALLOWED_DB, StubRangerPolicyEngine.ALLOWED_TABLE); /** Counts what actually reached the policy engine, so "answered without asking Ranger" is observable. */ private static final class CountingPolicyEngine extends StubRangerPolicyEngine { @@ -62,73 +67,123 @@ public RangerAccessResult isAccessAllowed(RangerAccessRequest request) { } } + /** + * Stands for whatever {@code access_controller_type} installs: it grants everything to one account and + * nothing to anyone else, and remembers what it was asked about. + */ + private static final class Authority implements AuthorizationPlugin { + private final AuthorizedSubject allowed; + private ResourceKind askedAbout; + + private Authority(AuthorizedSubject allowed) { + this.allowed = allowed; + } + + @Override + public String name() { + return "authority"; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + askedAbout = resource.getKind(); + if (!subject.equals(allowed)) { + throw AccessDeniedException.of(subject, resource, requirement, name()); + } + } + } + private final CountingPolicyEngine engine = new CountingPolicyEngine(); - private final RangerDorisAccessController controller = new RangerDorisAccessController(engine); @Test - public void testGlobalScopeAuthorityGrantsWithoutConsultingRanger() { - CatalogAccessController authority = Mockito.mock(CatalogAccessController.class); - Mockito.when(authority.checkGlobalPriv(ADMIN, PrivPredicate.SELECT)).thenReturn(true); + public void testGlobalScopeAuthorityGrantsWithoutConsultingRanger() throws Exception { + Authority authority = new Authority(ADMIN); + RangerDorisAccessController controller = rangerDeferringTo(authority, Collections.emptyMap()); - boolean allowed = withGlobalScopeAuthority(authority, - () -> controller.checkTblPriv(ADMIN, "ctl", StubRangerPolicyEngine.ALLOWED_DB, - StubRangerPolicyEngine.ALLOWED_TABLE, PrivPredicate.SELECT)); + controller.checkPrivilege(ADMIN, ALLOWED_TABLE, AccessRequirements.SELECT, AccessContext.NONE); - Assert.assertTrue(allowed); // Not merely an optimisation: the Ranger policy set denies this user everywhere, so had the request // reached the engine the answer would have been the opposite one. Assert.assertEquals(0, engine.requests.get()); + // And what was deferred is the privilege at instance scope, not the table. + Assert.assertEquals(ResourceKind.GLOBAL, authority.askedAbout); } @Test - public void testRangerDecidesWhenTheAuthorityGrantsNothingGlobally() { - CatalogAccessController authority = Mockito.mock(CatalogAccessController.class); - - Assert.assertTrue(withGlobalScopeAuthority(authority, - () -> controller.checkTblPriv(RANGER_USER, "ctl", StubRangerPolicyEngine.ALLOWED_DB, - StubRangerPolicyEngine.ALLOWED_TABLE, PrivPredicate.SELECT))); - Assert.assertFalse(withGlobalScopeAuthority(authority, - () -> controller.checkTblPriv(RANGER_USER, "ctl", StubRangerPolicyEngine.ALLOWED_DB, - "other_tbl", PrivPredicate.SELECT))); + public void testRangerDecidesWhenTheAuthorityGrantsNothingGlobally() throws Exception { + RangerDorisAccessController controller = rangerDeferringTo(new Authority(ADMIN), + Collections.emptyMap()); + + controller.checkPrivilege(RANGER_USER, ALLOWED_TABLE, AccessRequirements.SELECT, AccessContext.NONE); + Assert.assertThrows(AccessDeniedException.class, + () -> controller.checkPrivilege(RANGER_USER, + AuthorizedResource.table("ctl", StubRangerPolicyEngine.ALLOWED_DB, "other_tbl"), + AccessRequirements.SELECT, AccessContext.NONE)); } /** - * With Ranger installed globally, the controller is itself the authority and its own global check already - * answers the question - asking through the manager would evaluate the same Ranger policies a second time - * on every database, table and column check. + * A source configured not to defer answers out of its own policies alone, so an administrator of the + * instance has no access to what it governs beyond what Ranger grants. */ @Test - public void testBeingTheAuthorityCostsNoExtraPolicyEvaluation() { - int asPlugin = requestsWhile(Mockito.mock(CatalogAccessController.class), - () -> controller.checkDbPriv(RANGER_USER, "ctl", "other_db", PrivPredicate.SELECT)); - int asAuthority = requestsWhile(controller, - () -> controller.checkDbPriv(RANGER_USER, "ctl", "other_db", PrivPredicate.SELECT)); + public void testConfiguredNotToDeferRefusesTheGlobalScopeAuthority() throws Exception { + Authority authority = new Authority(ADMIN); + RangerDorisAccessController controller = rangerDeferringTo(authority, Collections.singletonMap( + RangerAccessController.DEFER_TO_GLOBAL_SCOPE_AUTHORITY, "false")); + + Assert.assertThrows(AccessDeniedException.class, + () -> controller.checkPrivilege(ADMIN, ALLOWED_TABLE, AccessRequirements.SELECT, + AccessContext.NONE)); + Assert.assertNull("the authority must not even be asked", authority.askedAbout); + Assert.assertNotEquals("Ranger has to be the one deciding", 0, engine.requests.get()); + } - Assert.assertEquals(asPlugin, asAuthority); + @Test + public void testUnreadableDeferenceSettingIsRejected() { + Assert.assertThrows(IllegalArgumentException.class, + () -> rangerDeferringTo(new Authority(ADMIN), Collections.singletonMap( + RangerAccessController.DEFER_TO_GLOBAL_SCOPE_AUTHORITY, "no"))); } - private int requestsWhile(CatalogAccessController authority, BooleanSupplier check) { + /** + * With Ranger installed for the instance, the source is itself the authority and its own global check + * already answers the question - asking through the engine would evaluate the same Ranger policies a + * second time on every database, table and column check. + */ + @Test + public void testBeingTheAuthorityCostsNoExtraPolicyEvaluation() throws Exception { + int asCatalogSource = requestsWhile(new Authority(ADMIN)); + int asAuthority = requestsWhile(null); + + Assert.assertEquals(asCatalogSource, asAuthority); + } + + private int requestsWhile(Authority authority) throws Exception { engine.requests.set(0); - withGlobalScopeAuthority(authority, check); + RangerDorisAccessController controller = rangerDeferringTo(authority, Collections.emptyMap()); + try { + controller.checkPrivilege(RANGER_USER, AuthorizedResource.database("ctl", "other_db"), + AccessRequirements.SELECT, AccessContext.NONE); + } catch (AccessDeniedException expected) { + // The point is the number of evaluations, not the verdict. + } return engine.requests.get(); } /** - * Runs {@code check} against an FE whose {@code access_controller_type} resolves to {@code authority}. - * - *

Installed the way the engine installs one written against the older interface - behind the adapter - - * because that is what makes "is this authority me?" a question about the controller rather than about - * the object the manager happens to hold. + * Stands a Ranger source up the way the engine does: with a context of the engine's own, on an FE whose + * {@code access_controller_type} resolves to {@code authority} - or to the Ranger source itself when + * there is no other authority. */ - private boolean withGlobalScopeAuthority(CatalogAccessController authority, BooleanSupplier check) { + private RangerDorisAccessController rangerDeferringTo(AuthorizationPlugin authority, + Map properties) { AccessControllerManager manager = new AccessControllerManager(new Auth()); + EngineAuthorizationContext context = new EngineAuthorizationContext(manager, manager.getAuth()); + RangerDorisAccessController controller = new RangerDorisAccessController(engine, properties, context); + context.servedBy(controller); Deencapsulation.setField(manager, "defaultAccessController", - new LegacyAccessControllerPlugin("authority", authority)); - try (MockedStatic mockedEnv = Mockito.mockStatic(Env.class)) { - Env env = Mockito.mock(Env.class); - mockedEnv.when(Env::getCurrentEnv).thenReturn(env); - Mockito.when(env.getAccessManager()).thenReturn(manager); - return check.getAsBoolean(); - } + authority == null ? controller : authority); + return controller; } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java new file mode 100644 index 00000000000000..20ed56f7fcd823 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java @@ -0,0 +1,96 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.catalog.authorizer.ranger.hive; + +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.spi.AuthorizationContext; + +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.ImmutableSet; +import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl; +import org.junit.Assert; +import org.junit.Test; +import org.mockito.MockedConstruction; +import org.mockito.Mockito; + +import java.util.Set; + +public class RangerHiveAccessControllerTest { + + private static final AuthorizedSubject SUBJECT = AuthorizedSubject.of("user1", "%"); + + /** + * A Ranger policy may be written against a role, and which roles a Doris account holds is the engine's to + * know: the source has no user directory of its own to look them up in. + */ + @Test + public void testRequestCarriesTheRolesTheEngineKnows() { + Set roles = ImmutableSet.of("analyst"); + AuthorizationContext context = Mockito.mock(AuthorizationContext.class); + Mockito.when(context.rolesOf(SUBJECT)).thenReturn(roles); + + try (MockedConstruction plugin = Mockito.mockConstruction(RangerHivePlugin.class); + MockedConstruction audit = + Mockito.mockConstruction(RangerHiveAuditHandler.class)) { + RangerHiveAccessController controller = new RangerHiveAccessController( + ImmutableMap.of("ranger.service.name", "hive"), context); + try { + RangerAccessRequestImpl request = controller.createRequest(SUBJECT); + + Assert.assertEquals("user1", request.getUser()); + Assert.assertEquals(roles, request.getUserRoles()); + Assert.assertEquals("%", request.getClientIPAddress()); + } finally { + controller.close(); + } + } + } + + /** + * Only the checks the engine asks by name map onto a Hive access type. Anything else - a requirement put + * together for one statement, for instance - maps to one no Hive policy grants, rather than to a + * neighbouring access type that some policy might. + */ + @Test + public void testAccessTypeIsRecognisedByWhatIsAsked() { + Assert.assertEquals(HiveAccessType.USE, + RangerHiveAccessController.accessTypeOf(AccessRequirements.VISIBILITY)); + Assert.assertEquals(HiveAccessType.SELECT, + RangerHiveAccessController.accessTypeOf(AccessRequirements.SELECT)); + Assert.assertEquals(HiveAccessType.UPDATE, + RangerHiveAccessController.accessTypeOf(AccessRequirements.LOAD)); + Assert.assertEquals(HiveAccessType.ALTER, + RangerHiveAccessController.accessTypeOf(AccessRequirements.ALTER)); + Assert.assertEquals(HiveAccessType.CREATE, + RangerHiveAccessController.accessTypeOf(AccessRequirements.CREATE)); + Assert.assertEquals(HiveAccessType.DROP, + RangerHiveAccessController.accessTypeOf(AccessRequirements.DROP)); + Assert.assertEquals(HiveAccessType.ALL, + RangerHiveAccessController.accessTypeOf(AccessRequirements.ADMINISTRATION)); + Assert.assertEquals(HiveAccessType.ALL, + RangerHiveAccessController.accessTypeOf(AccessRequirements.ANY_PRIVILEGE)); + + Assert.assertEquals(HiveAccessType.NONE, RangerHiveAccessController.accessTypeOf( + AccessRequirement.allOf(AccessAction.SELECT, AccessAction.GRANT))); + Assert.assertEquals(HiveAccessType.NONE, + RangerHiveAccessController.accessTypeOf(AccessRequirement.of(AccessAction.USAGE))); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java index 0c75a4877e850f..f6a4cc39e4cf81 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessControlBehaviorBaselineTest.java @@ -217,11 +217,19 @@ private List renderSnapshot() { AuthorizationPlugin builtin = Deencapsulation.getField(manager, "defaultAccessController"); renderWithDefaultController(lines, manager, "builtin", builtin); - renderWithDefaultController(lines, manager, "ranger", new LegacyAccessControllerPlugin( - "stub-ranger-doris", new RangerDorisAccessController(new StubRangerPolicyEngine()))); + renderWithDefaultController(lines, manager, "ranger", rangerInstalledForTheInstance(manager)); return lines; } + /** A Ranger source standing where {@code access_controller_type} puts one, built as the engine builds it. */ + private AuthorizationPlugin rangerInstalledForTheInstance(AccessControllerManager manager) { + EngineAuthorizationContext context = new EngineAuthorizationContext(manager, manager.getAuth()); + RangerDorisAccessController ranger = + new RangerDorisAccessController(new StubRangerPolicyEngine(), context); + context.servedBy(ranger); + return ranger; + } + /** * Renders the whole matrix under one default controller, i.e. under one value of * {@code fe.conf: access_controller_type}. diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessRequirementVocabularyTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessRequirementVocabularyTest.java new file mode 100644 index 00000000000000..6aa27387d80400 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AccessRequirementVocabularyTest.java @@ -0,0 +1,69 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; + +import com.google.common.collect.ImmutableMap; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.util.Map; + +/** + * The questions the engine asks by name mean the same thing on both sides of the plugin contract. + * + *

A source outside this repository recognises a check by comparing it against {@link AccessRequirements}; + * the engine builds that check out of a {@link PrivPredicate}. If the two ever say different things, no + * compiler and no plugin notices: the source simply stops recognising the question and falls back to whatever + * it does with one it does not know - for the Ranger Hive service, an access type no policy grants. So the + * agreement is asserted here, where both are visible, rather than left to be discovered in production. + */ +public class AccessRequirementVocabularyTest { + + private static final Map VOCABULARY = + ImmutableMap.builder() + .put(AccessRequirements.VISIBILITY, PrivPredicate.SHOW) + .put(AccessRequirements.SELECT, PrivPredicate.SELECT) + .put(AccessRequirements.LOAD, PrivPredicate.LOAD) + .put(AccessRequirements.ALTER, PrivPredicate.ALTER) + .put(AccessRequirements.CREATE, PrivPredicate.CREATE) + .put(AccessRequirements.DROP, PrivPredicate.DROP) + .put(AccessRequirements.ADMINISTRATION, PrivPredicate.ADMIN) + .put(AccessRequirements.ANY_PRIVILEGE, PrivPredicate.ALL) + .build(); + + @Test + public void testEachNamedRequirementAsksWhatItsPredicateAsks() { + VOCABULARY.forEach((requirement, predicate) -> Assertions.assertEquals(requirement, + AccessTranslation.requirementOf(predicate), + "the actions " + predicate + " names have changed; " + requirement + + " has to change with them or plugins stop recognising the question")); + } + + /** + * And back: the engine still compares the predicate it gets against its own constants by identity, so a + * named requirement has to translate to that very constant and not to an equal copy of it. + */ + @Test + public void testEachNamedRequirementTranslatesBackToItsOwnPredicate() { + VOCABULARY.forEach((requirement, predicate) -> Assertions.assertSame(predicate, + AccessTranslation.privPredicateOf(requirement))); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/EngineAuthorizationContextTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/EngineAuthorizationContextTest.java new file mode 100644 index 00000000000000..ccd9fd963fa0e5 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/EngineAuthorizationContextTest.java @@ -0,0 +1,146 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.common.jmockit.Deencapsulation; + +import com.google.common.collect.ImmutableSet; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; + +import java.util.Set; + +/** + * What the engine answers an authorization source that asks it something. + */ +public class EngineAuthorizationContextTest { + + private static final AuthorizedSubject SUBJECT = AuthorizedSubject.of("user1", "%"); + + /** An authorization source that records what it was asked and grants everything. */ + private static final class Recording implements AuthorizationPlugin { + private ResourceKind askedAbout; + private AccessRequirement asked; + + @Override + public String name() { + return "recording"; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + askedAbout = resource.getKind(); + asked = requirement; + } + } + + /** + * Roles come without the per-user default role: it is how the built-in model stores a user's own grants, + * not a role an administrator writes a policy against. + */ + @Test + public void testRolesAreReadWithoutTheInternalDefaultRole() { + Auth auth = Mockito.mock(Auth.class); + Set roles = ImmutableSet.of("analyst"); + Mockito.when(auth.getRolesByUser(Mockito.any(UserIdentity.class), Mockito.eq(false))).thenReturn(roles); + + EngineAuthorizationContext context = new EngineAuthorizationContext( + new AccessControllerManager(auth), auth); + context.servedBy(new Recording()); + + Assertions.assertEquals(roles, context.rolesOf(SUBJECT)); + Mockito.verify(auth).getRolesByUser( + UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"), false); + } + + @Test + public void testAsksWhoeverGovernsInstanceScope() { + Auth auth = Mockito.mock(Auth.class); + AccessControllerManager manager = new AccessControllerManager(auth); + Recording authority = new Recording(); + Deencapsulation.setField(manager, "defaultAccessController", authority); + + EngineAuthorizationContext context = new EngineAuthorizationContext(manager, auth); + context.servedBy(new Recording()); + + Assertions.assertTrue(context.grantedByGlobalScopeAuthority(SUBJECT, AccessRequirements.SELECT)); + Assertions.assertEquals(ResourceKind.GLOBAL, authority.askedAbout); + Assertions.assertEquals(AccessRequirements.SELECT, authority.asked); + } + + /** + * A source that governs instance scope itself is told no rather than being routed back into itself: it is + * about to answer that very question, and answering it twice evaluates the same policies twice. + */ + @Test + public void testTheAuthorityIsNotAskedToDeferToItself() { + Auth auth = Mockito.mock(Auth.class); + AccessControllerManager manager = new AccessControllerManager(auth); + Recording authority = new Recording(); + Deencapsulation.setField(manager, "defaultAccessController", authority); + + EngineAuthorizationContext context = new EngineAuthorizationContext(manager, auth); + context.servedBy(authority); + + Assertions.assertFalse(context.grantedByGlobalScopeAuthority(SUBJECT, AccessRequirements.SELECT)); + Assertions.assertNull(authority.askedAbout); + } + + /** + * The authority may well be a controller written against the older interface - that is what + * {@code access_controller_type} has always named - so deferring to it has to reach through the adapter + * and arrive as the global check that interface offers. + */ + @Test + public void testDeferenceReachesAnAuthorityWrittenAgainstTheOlderInterface() { + Auth auth = Mockito.mock(Auth.class); + AccessControllerManager manager = new AccessControllerManager(auth); + CatalogAccessController controller = Mockito.mock(CatalogAccessController.class); + Mockito.when(controller.checkGlobalPriv( + UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"), PrivPredicate.SELECT)) + .thenReturn(true); + Deencapsulation.setField(manager, "defaultAccessController", + new LegacyAccessControllerPlugin("authority", controller)); + + EngineAuthorizationContext context = new EngineAuthorizationContext(manager, auth); + context.servedBy(new Recording()); + + Assertions.assertTrue(context.grantedByGlobalScopeAuthority(SUBJECT, AccessRequirements.SELECT)); + } + + @Test + public void testAskingBeforeTheEngineKnowsWhichSourceItIsFails() { + Auth auth = Mockito.mock(Auth.class); + EngineAuthorizationContext context = new EngineAuthorizationContext( + new AccessControllerManager(auth), auth); + + Assertions.assertThrows(IllegalStateException.class, + () -> context.grantedByGlobalScopeAuthority(SUBJECT, AccessRequirements.SELECT)); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java index 7029f8e4389f8e..58c9e579276b3f 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java @@ -17,24 +17,41 @@ package org.apache.doris.mysql.privilege; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.junit.Assert; +import org.junit.Before; import org.junit.Test; import org.mockito.MockedConstruction; import org.mockito.Mockito; +import java.lang.reflect.Field; import java.util.Collections; public class RangerDorisAccessControllerFactoryTest { + + @Before + public void forgetPreviouslyCreatedController() throws Exception { + Field instance = RangerDorisAccessControllerFactory.class.getDeclaredField("instance"); + instance.setAccessible(true); + instance.set(null, null); + } + + /** + * One controller per FE, whoever asks for it: it starts a Ranger policy refresher, and a second one would + * mean a second refresher polling the same service. + */ @Test - public void testCreateAccessControllerReturnsSingleton() { + public void testCreateReturnsSingleton() { + AuthorizationContext context = Mockito.mock(AuthorizationContext.class); try (MockedConstruction mockedConstruction = Mockito.mockConstruction(RangerDorisAccessController.class)) { - RangerDorisAccessController first = new RangerDorisAccessControllerFactory() - .createAccessController(Collections.emptyMap()); - RangerDorisAccessController second = new RangerDorisAccessControllerFactory() - .createAccessController(Collections.emptyMap()); + AuthorizationPlugin first = new RangerDorisAccessControllerFactory() + .create(Collections.emptyMap(), context); + AuthorizationPlugin second = new RangerDorisAccessControllerFactory() + .create(Collections.singletonMap("ranger.service.name", "other"), context); Assert.assertEquals(1, mockedConstruction.constructed().size()); Assert.assertSame(first, second); diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java index 9a3f1af03b447c..2d2dd6f0dcf28c 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java @@ -17,12 +17,19 @@ package org.apache.doris.mysql.privilege; -import org.apache.doris.analysis.ResourceTypeEnum; -import org.apache.doris.analysis.UserIdentity; +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.DataMaskSpec; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisResource; -import org.apache.doris.common.AuthorizationException; import com.google.common.base.Strings; import com.google.common.collect.Lists; @@ -36,9 +43,11 @@ import org.junit.jupiter.api.Assertions; import java.util.Collection; +import java.util.Collections; import java.util.List; -import java.util.Optional; +import java.util.Map; import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; public class RangerTest { @@ -129,132 +138,174 @@ private RangerAccessResult returnAccessResult( } } + /** + * Grants one action per level of the hierarchy and counts what was asked, so that an action a level + * already granted being asked about again further down is observable. + */ + public static class LevelledPlugin extends RangerBasePlugin { + private final AtomicInteger requests = new AtomicInteger(); + + public LevelledPlugin() { + super("test", null, null); + } + + @Override + public RangerAccessResult isAccessAllowed(RangerAccessRequest request) { + requests.incrementAndGet(); + RangerAccessResource resource = request.getResource(); + RangerAccessResult result = new RangerAccessResult(1, "test", null, request); + if (resource.getValue(RangerDorisResource.KEY_GLOBAL) != null) { + result.setIsAllowed(false); + } else if (resource.getValue(RangerDorisResource.KEY_TABLE) == null) { + result.setIsAllowed(DorisAccessType.SELECT.name().equals(request.getAccessType())); + } else { + result.setIsAllowed(DorisAccessType.LOAD.name().equals(request.getAccessType())); + } + return result; + } + } + + private static final AuthorizedSubject USER = AuthorizedSubject.of("user1", "%"); + /** Resource and workload group usage, as the engine asks about it. */ + private static final AccessRequirement USAGE = AccessRequirement.anyOf(AccessAction.ADMIN, + AccessAction.USAGE, AccessAction.CLUSTER_USAGE, AccessAction.STAGE_USAGE); + + /** + * An FE where nothing outside Ranger grants anything, so every verdict below is Ranger's own. + */ + private static final AuthorizationContext NOTHING_GRANTED_ELSEWHERE = new AuthorizationContext() { + @Override + public Set rolesOf(AuthorizedSubject subject) { + return Collections.emptySet(); + } + + @Override + public boolean grantedByGlobalScopeAuthority(AuthorizedSubject subject, AccessRequirement requirement) { + return false; + } + }; + + private RangerDorisAccessController controller() { + return new RangerDorisAccessController(new DorisTestPlugin("test"), NOTHING_GRANTED_ELSEWHERE); + } + + private void check(AuthorizedResource resource, AccessRequirement requirement) throws AccessDeniedException { + controller().checkPrivilege(USER, resource, requirement, AccessContext.NONE); + } + + private void assertRefused(AuthorizedResource resource, AccessRequirement requirement) { + Assertions.assertThrows(AccessDeniedException.class, () -> check(resource, requirement)); + } + // Does not have priv on ctl1.db1.tbl1.col3 - @Test(expected = AuthorizationException.class) - public void testNoAuthCol() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col3"); - ac.checkColsPriv(ui, "ctl1", "db1", "tbl1", cols, PrivPredicate.SELECT); + @Test + public void testNoAuthCol() { + assertRefused(AuthorizedResource.columns("ctl1", "db1", "tbl1", Sets.newHashSet("col1", "col3")), + AccessRequirements.SELECT); } // Have priv on ctl1.db1.tbl1.col1 & col2 @Test - public void testAuthCol() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl1", "db1", "tbl1", cols, PrivPredicate.SELECT); + public void testAuthCol() throws AccessDeniedException { + check(AuthorizedResource.columns("ctl1", "db1", "tbl1", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); } // Have priv on ctl2.db2.tbl2, so when checking auth on col1 & col2, can pass @Test - public void testUsingTableAuthAsColAuth() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl2", "db2", "tbl2", cols, PrivPredicate.SELECT); + public void testUsingTableAuthAsColAuth() throws AccessDeniedException { + check(AuthorizedResource.columns("ctl2", "db2", "tbl2", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); } // Does not have priv on ctl2.db2.tbl3, so when checking auth on col1 & col2, can not pass - @Test(expected = AuthorizationException.class) - public void testUsingNoTableAuthAsColAuth() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl2", "db2", "tbl3", cols, PrivPredicate.SELECT); + @Test + public void testUsingNoTableAuthAsColAuth() { + assertRefused(AuthorizedResource.columns("ctl2", "db2", "tbl3", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); } // Have priv on ctl3.db3, so when checking auth on tbl1 and (tbl1.col1 & tbl1.col2), can pass @Test - public void testUsingDbAuthAsColAndTableAuth() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl3", "db3", "tbl1", cols, PrivPredicate.SELECT); - ac.checkTblPriv(ui, "ctl3", "db3", "tbl1", PrivPredicate.SELECT); + public void testUsingDbAuthAsColAndTableAuth() throws AccessDeniedException { + check(AuthorizedResource.columns("ctl3", "db3", "tbl1", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); + check(AuthorizedResource.table("ctl3", "db3", "tbl1"), AccessRequirements.SELECT); } // Does not have priv on ctl2.db3, so when checking auth on col1 & col2, can not pass - @Test(expected = AuthorizationException.class) - public void testNoDbAuthAsColAndTableAuth() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl2", "db3", "tbl3", cols, PrivPredicate.SELECT); + @Test + public void testNoDbAuthAsColAndTableAuth() { + assertRefused(AuthorizedResource.columns("ctl2", "db3", "tbl3", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); } // Have priv on ctl4, so when checking auth on objs under ctl4, can pass @Test - public void testUsingCtlAuthAsColAndTableAndDbAuth() throws AuthorizationException { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - Set cols = Sets.newHashSet(); - cols.add("col1"); - cols.add("col2"); - ac.checkColsPriv(ui, "ctl4", "db1", "tbl1", cols, PrivPredicate.SELECT); - ac.checkTblPriv(ui, "ctl4", "db2", "tbl2", PrivPredicate.SELECT); - ac.checkDbPriv(ui, "ctl4", "db3", PrivPredicate.SELECT); + public void testUsingCtlAuthAsColAndTableAndDbAuth() throws AccessDeniedException { + check(AuthorizedResource.columns("ctl4", "db1", "tbl1", Sets.newHashSet("col1", "col2")), + AccessRequirements.SELECT); + check(AuthorizedResource.table("ctl4", "db2", "tbl2"), AccessRequirements.SELECT); + check(AuthorizedResource.database("ctl4", "db3"), AccessRequirements.SELECT); + } + + /** + * An action granted at an outer level is not asked about again further in. + * + *

Doris checks a whole requirement, and this source answers it one action per request while walking + * the resource down to the table. Carrying what is already granted is what keeps that walk at one request + * per action rather than one per action per level - a difference in cost, not in verdict, so nothing that + * records only allow/deny can hold it still. + */ + @Test + public void testAnActionGrantedAtAnOuterLevelIsNotAskedAboutAgain() throws AccessDeniedException { + LevelledPlugin plugin = new LevelledPlugin(); + RangerDorisAccessController controller = + new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE); + + controller.checkPrivilege(USER, AuthorizedResource.table("ctl", "db", "tbl"), + AccessRequirement.allOf(AccessAction.SELECT, AccessAction.LOAD), AccessContext.NONE); + + // Both actions at global (2) and at catalog (2, where SELECT is granted), then only the outstanding + // LOAD at database (1) and at table (1), where it is granted. + Assertions.assertEquals(6, plugin.requests.get()); + } + + /** The refusal names the column that failed, which is the whole reason columns answer with a message. */ + @Test + public void testRefusalNamesTheColumnThatFailed() { + AccessDeniedException refused = Assertions.assertThrows(AccessDeniedException.class, + () -> check(AuthorizedResource.columns("ctl1", "db1", "tbl1", Sets.newHashSet("col3")), + AccessRequirements.SELECT)); + Assertions.assertEquals("Permission denied: user ['user1'@'%'] does not have privilege for" + + " [ANY[ADMIN, SELECT]] command on [ctl1].[db1].[tbl1].[col3]", refused.getMessage()); } @Test public void testDataMask() { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); + Map masks = controller().getDataMasks(USER, + AuthorizedResource.table("ctl1", "db1", "tbl1"), + Sets.newHashSet("col1", "col2", "col3", "col4"), AccessContext.NONE); // MASK_NULL - Optional policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col1"); - Assertions.assertEquals("NULL", policy.get().getMaskSql()); + Assertions.assertEquals("NULL", masks.get("col1").getMaskSql()); // MASK_NONE - policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col2"); - Assertions.assertTrue(!policy.isPresent()); + Assertions.assertFalse(masks.containsKey("col2")); // CUSTOM - policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col3"); - Assertions.assertEquals("hex(col3)", policy.get().getMaskSql()); + Assertions.assertEquals("hex(col3)", masks.get("col3").getMaskSql()); // Others - policy = ac.evalDataMaskPolicy(ui, "ctl1", "db1", "tbl1", "col4"); - Assertions.assertTrue(!policy.isPresent()); + Assertions.assertFalse(masks.containsKey("col4")); } @Test - public void testComputeGroupAuth() { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - boolean cg1 = ac.checkCloudPriv(ui, "cg1", PrivPredicate.USAGE, ResourceTypeEnum.CLUSTER); - Assertions.assertTrue(cg1); - boolean cg2 = ac.checkCloudPriv(ui, "cg2", PrivPredicate.USAGE, ResourceTypeEnum.CLUSTER); - Assertions.assertFalse(cg2); + public void testComputeGroupAuth() throws AccessDeniedException { + check(AuthorizedResource.cloud(ResourceKind.CLOUD_COMPUTE_GROUP, "cg1"), USAGE); + assertRefused(AuthorizedResource.cloud(ResourceKind.CLOUD_COMPUTE_GROUP, "cg2"), USAGE); } @Test - public void testStorageVaultAuth() { - DorisTestPlugin plugin = new DorisTestPlugin("test"); - RangerDorisAccessController ac = new RangerDorisAccessController(plugin); - UserIdentity ui = UserIdentity.createAnalyzedUserIdentWithIp("user1", "%"); - boolean cg1 = ac.checkStorageVaultPriv(ui, "sv1", PrivPredicate.USAGE); - Assertions.assertTrue(cg1); - boolean cg2 = ac.checkStorageVaultPriv(ui, "sv2", PrivPredicate.USAGE); - Assertions.assertFalse(cg2); + public void testStorageVaultAuth() throws AccessDeniedException { + check(AuthorizedResource.storageVault("sv1"), USAGE); + assertRefused(AuthorizedResource.storageVault("sv2"), USAGE); } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java index 20ad286b412d93..ac7656b4bff76c 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/StubRangerAccessControllerFactory.java @@ -17,22 +17,25 @@ package org.apache.doris.mysql.privilege; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import java.util.Map; /** - * Binds a catalog to the production Ranger controller driven by {@link StubRangerPolicyEngine}, so a test can + * Binds a catalog to the production Ranger source driven by {@link StubRangerPolicyEngine}, so a test can * create a Ranger-governed catalog with {@code "access_controller.class"} without a Ranger server. */ -public class StubRangerAccessControllerFactory implements AccessControllerFactory { +public class StubRangerAccessControllerFactory implements AuthorizationPluginFactory { @Override - public String factoryIdentifier() { + public String name() { return "stub-ranger-doris"; } @Override - public CatalogAccessController createAccessController(Map prop) { - return new RangerDorisAccessController(new StubRangerPolicyEngine()); + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return new RangerDorisAccessController(new StubRangerPolicyEngine(), context); } } diff --git a/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory new file mode 100644 index 00000000000000..a821c437bdb939 --- /dev/null +++ b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory @@ -0,0 +1,18 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# +org.apache.doris.mysql.privilege.StubRangerAccessControllerFactory diff --git a/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory index 776fc40c3f77a0..9908dc00894ba5 100644 --- a/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory +++ b/fe/fe-core/src/test/resources/META-INF/services/org.apache.doris.mysql.privilege.AccessControllerFactory @@ -16,4 +16,3 @@ # # org.apache.doris.nereids.privileges.CustomAccessControllerFactory -org.apache.doris.mysql.privilege.StubRangerAccessControllerFactory \ No newline at end of file From f75b663e9bfee4740323803729d20cebbb3d0869 Mon Sep 17 00:00:00 2001 From: morningman Date: Wed, 12 Aug 2026 23:40:03 +0800 Subject: [PATCH 09/22] [feat](authorization) let an authorization source be installed from a directory Deciding access became a contract in the previous change, but the only way to publish an implementation of it was to be on the FE's class path - which means being built into the FE. A source that ships separately had nowhere to be. It has one now: plugins/authorization//, the layout the connector, filesystem, authentication and lineage families already use, read through the same loader. That loader is what makes the difference from the older channel this one sits beside: it gives each plugin its own classloader, and it takes a version gate as a mandatory argument rather than as something a family remembers to add. So a jar dropped in there has to declare, in its manifest, the authorization plugin API it was built against, and one that declares nothing - or another major - is refused instead of inheriting the kernel's own answer. Both channels read the same directory and cannot collide: the older one lists jars lying loose at the root, this one lists subdirectories. Three things follow from what the loader does rather than from the design: - org.apache.doris.authorization. is loaded parent-first, so the vocabulary crossing the boundary exists exactly once. A plugin bundling its own copy of ResourceKind would otherwise hand back a value the engine refuses to recognise as the type it asked for, which reads as a plugin bug and is not one. - a name already answered by a source shipped with the FE keeps it. Otherwise a jar named ranger-doris, dropped into a directory, would displace the real one - and an allow-everything plugin under a trusted source's name is the whole point of refusing that. - the class path channel is deliberately NOT gated. What is on the class path was built from this tree in this build, so its version would be compared against itself; and the in-tree Ranger sources live in fe-core.jar, which carries no authorization stamp at all, so gating it would refuse them at startup. No unit test can see this - a test loads its classes from target/classes, a directory, where there is no manifest to read. A refusal used to be invisible where it hurt. "No authorization plugin factory found for X" read identically whether X was never installed or was installed and refused on its version, so the reason is now appended to it, naming both versions. The older AccessControllerFactory and CatalogAccessController interfaces keep working and are now deprecated. Registering one logs what to implement instead, where to put it, and which manifest attribute to declare. AuthorizationPluginSurfaceTest freezes the contract - the SPI interfaces and the api types they speak in - so that adding a method or an enum constant fails until the major version moves in the same commit. Copying the authentication family's baseline would not have worked: erasure hides RowFilterSpec inside List unless generic signatures are walked, members inherited from the JDK have to stay out or a JDK upgrade turns it red for nothing, and the self-referential Enum> overflows a recursive closure. Verified: behaviour baseline unchanged; 480 tests across the 88 classes that touch the decision path, no failures, the four skips pre-existing; 28 api plus 8 spi tests; checkstyle clean in all three modules. The two version numbers were checked in the built jars by hand, there being no test that can read a pom's against the attribute name the gate derives. Mutations, five of five: adding a default method to the contract turns the frozen surface red; not recording why the gate refused a plugin turns the two "the message names both versions" tests red while leaving the admitted case green; changing the parent-first prefix turns the shared-vocabulary test red; admitting every version turns six tests red across three families. The fifth - deleting the guard on a name already taken - passed at first, and the escape was the finding: the assertion compared Class objects across classloaders, where they can never be equal, so it could not fail. Comparing class names instead turns it red. A second assertion in the same test, on the inventory table, could not fail either and now checks that a refused plugin is released along with its classloader. Co-Authored-By: Claude Opus 5 (1M context) --- .../spi/AuthenticationPluginSurfaceTest.java | 2 +- fe/fe-authentication/pom.xml | 2 +- .../fe-authorization-spi/README.md | 167 ++++++++++ .../fe-authorization-spi/pom.xml | 14 + .../spi/AuthorizationPluginFactory.java | 23 +- ...uthorization-plugin-api-version.properties | 7 + .../spi/AuthorizationPluginSurfaceTest.java | 305 ++++++++++++++++++ .../authorization-plugin-surface.txt | 221 +++++++++++++ fe/fe-authorization/pom.xml | 54 +++- .../spi/ConnectorPluginSurfaceTest.java | 2 +- fe/fe-connector/pom.xml | 2 +- fe/fe-core/pom.xml | 2 +- .../privilege/AccessControllerFactory.java | 10 + .../privilege/AccessControllerManager.java | 155 ++++++++- .../privilege/CatalogAccessController.java | 5 + .../lineage/LineagePluginSurfaceTest.java | 2 +- .../PluginApiVersionWiringTest.java | 208 +++++++++++- .../ShadowingAuthorizationPluginFactory.java | 58 ++++ ...ersionProbeAuthorizationPluginFactory.java | 63 ++++ .../spi/FileSystemPluginSurfaceTest.java | 2 +- fe/fe-filesystem/pom.xml | 2 +- 21 files changed, 1271 insertions(+), 35 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-spi/README.md create mode 100644 fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties create mode 100644 fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginSurfaceTest.java create mode 100644 fe/fe-authorization/fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt create mode 100644 fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/VersionProbeAuthorizationPluginFactory.java diff --git a/fe/fe-authentication/fe-authentication-spi/src/test/java/org/apache/doris/authentication/spi/AuthenticationPluginSurfaceTest.java b/fe/fe-authentication/fe-authentication-spi/src/test/java/org/apache/doris/authentication/spi/AuthenticationPluginSurfaceTest.java index 1d7585e99a236e..8d028368ce620a 100644 --- a/fe/fe-authentication/fe-authentication-spi/src/test/java/org/apache/doris/authentication/spi/AuthenticationPluginSurfaceTest.java +++ b/fe/fe-authentication/fe-authentication-spi/src/test/java/org/apache/doris/authentication/spi/AuthenticationPluginSurfaceTest.java @@ -47,7 +47,7 @@ * {@code fe/fe-authentication/pom.xml} in the SAME commit. * *

{@code Plugin} / {@code PluginFactory} / {@code PluginContext} from fe-extension-spi are frozen here - * too, and identically in the other three families' baselines. They are loaded parent-first for every family + * too, and identically in the other four families' baselines. They are loaded parent-first for every family * (see {@code ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES}), so a change to them breaks all four * plugin kinds at once — and turns all four baselines red at once, each asking for its own bump. * diff --git a/fe/fe-authentication/pom.xml b/fe/fe-authentication/pom.xml index a8fcbba9ed55b7..556921612b7381 100644 --- a/fe/fe-authentication/pom.xml +++ b/fe/fe-authentication/pom.xml @@ -37,7 +37,7 @@ under the License. is what the FE kernel expects of a plugin. Bump the MAJOR (and zero the minor) in the SAME commit as ANY change to the authentication - SPI surface - additions included. A change to fe-extension-spi means bumping all four + SPI surface - additions included. A change to fe-extension-spi means bumping all five families. See plan-doc/designs/2026-07-29-plugin-api-version-check-design.md. --> 1.0 diff --git a/fe/fe-authorization/fe-authorization-spi/README.md b/fe/fe-authorization/fe-authorization-spi/README.md new file mode 100644 index 00000000000000..385eef3d183934 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/README.md @@ -0,0 +1,167 @@ +# Doris FE Authorization SPI + +## Overview + +`fe-authorization-spi` defines the plugin contract for authorization in Doris FE: an *authorization source* +decides, for the resources it governs, what a user may do with them. + +Plugin authors implement: +- `AuthorizationPlugin` — the decisions +- `AuthorizationPluginFactory` — how the engine builds one + +Both are discovered via Java `ServiceLoader`. The decision vocabulary (`AuthorizedSubject`, +`AuthorizedResource`, `AccessAction`, `AccessRequirement`, …) lives in `fe-authorization-api`, which this +module depends on and which is part of the same frozen contract. + +## What the engine promises + +**One source answers, and its answer is the whole answer.** Which source is asked follows from the resource +alone: the plugin a catalog is bound to answers for everything inside that catalog, the plugin installed for +the instance answers for everything else. Nothing grants access before a plugin is asked, and no second +plugin is consulted after it. Exemptions that used to be the engine's — "an administrator may go anywhere" — +are each plugin's own to grant or refuse, with `AuthorizationContext` there to ask the questions such a +decision needs. + +**Refusing is throwing.** A check that returns has allowed the access; a check that refuses throws +`AccessDeniedException`. There is no third outcome and no boolean for a caller to ignore. + +**Every check method defaults to refusing.** A plugin that implements nothing but `name()` denies +everything — an omission costs you access control you did not think about, never a hole. The two +data-policy methods are the exception: their empty default means "this source defines no policy", which is +not the same as allowing anything. + +## Minimal plugin + +```java +public final class CustomAuthorizationPlugin implements AuthorizationPlugin { + + private final AuthorizationContext context; + + CustomAuthorizationPlugin(AuthorizationContext context) { + this.context = context; + } + + @Override + public String name() { + return "custom-authz"; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext ctx) throws AccessDeniedException { + // Whoever owns instance scope may already have settled this; asking spares a second evaluation + // of the same policies. Answers false when this plugin IS that authority. + if (context.grantedByGlobalScopeAuthority(subject, requirement)) { + return; + } + if (!(resource instanceof AuthorizedResource.Table)) { + // A kind this source does not recognise is a refusal, never a guess. + throw AccessDeniedException.of(subject, resource, requirement, name()); + } + AuthorizedResource.Table table = (AuthorizedResource.Table) resource; + if (!allowed(context.rolesOf(subject), table, requirement)) { + throw AccessDeniedException.of(subject, resource, requirement, name()); + } + } + + @Override + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext ctx) { + // The predicate is SQL in Doris dialect; the engine parses, type-checks and plans it. The ident + // names the policy it came from, and only shows up in diagnostics. Several filters on one table are + // combined as each one's merge type says: RESTRICTIVE ones are ANDed, PERMISSIVE ones ORed. + return Collections.singletonList(RowFilterSpec.restrictive("eu-only", "region = 'EU'")); + } +} +``` + +Factory and `ServiceLoader` registration: + +```java +public final class CustomAuthorizationPluginFactory implements AuthorizationPluginFactory { + + @Override + public String name() { + return "custom-authz"; // the value of access_controller_type / access_controller.class + } + + @Override + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return new CustomAuthorizationPlugin(context); + } +} +``` + +`src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory`: + +```text +com.example.authz.CustomAuthorizationPluginFactory +``` + +## Lifecycle and cost + +A plugin is created once and kept. Unlike an authentication attempt, an authorization decision happens many +times within a single statement — planning one query checks every table it reads, and listing what a user may +see checks every object that exists — so a source that caches policies has to be the same instance +throughout. The engine builds a new one only when what configures it changes, and calls `close()` on the old +one. + +The engine adds no caching of its own and cannot: it does not know when a policy changed. Whatever caching an +external source needs belongs inside the plugin, where it can be invalidated on that source's own terms. + +## Packaging and installation + +Lay the plugin out one plugin per subdirectory of `authorization_plugins_dir` (default +`${DORIS_HOME}/plugins/authorization`): + +```text +plugins/authorization/ +└── custom-authz/ + ├── custom-authz-1.0.jar # the plugin: factory, plugin, service descriptor + └── lib/ + └── some-dependency.jar # whatever it needs, isolated from FE's own classpath +``` + +Then name it in `fe.conf` (`access_controller_type = custom-authz`) to govern the whole instance, or in a +catalog property (`"access_controller.class" = "custom-authz"`) to govern one external catalog. + +Each plugin gets its own child-first classloader, so its dependencies do not collide with the FE's. The +exception is `org.apache.doris.authorization.*` — the api and spi types are always loaded from the FE, so +that the types crossing the boundary exist exactly once. Do not bundle this module in your plugin jar. + +## Plugin API version + +Every plugin jar must declare, in its MANIFEST, the authorization plugin API it was built against: + +```xml + + org.apache.maven.plugins + maven-jar-plugin + + + + 1.0 + + + + +``` + +The FE admits a plugin whose **major** equals the one it serves; minor and patch are ignored. A jar that +declares nothing is refused, so a plugin written with no awareness of this contract cannot slip through. The +version the FE serves is recorded in `META-INF/doris/authorization-plugin-api-version.properties` inside this +module's jar. + +Major is bumped for **any** change to the frozen surface — adding a method or an enum constant just as much +as removing one. That is deliberate: one more `ResourceKind` constant turns every deployed plugin's "a kind I +do not recognise" branch into a refusal of something that used to be allowed. `AuthorizationPluginSurfaceTest` +freezes that surface and fails when it moves. + +This check is a compatibility control, not a security one: a plugin can declare a version it was not built +against. What it prevents is a plugin built for another release loading silently and deciding wrongly. + +## Test + +```bash +mvn -o -f fe/pom.xml -pl fe-authorization/fe-authorization-spi -am test +``` diff --git a/fe/fe-authorization/fe-authorization-spi/pom.xml b/fe/fe-authorization/fe-authorization-spi/pom.xml index ea51ffe4149c15..cc8f1422325fb1 100644 --- a/fe/fe-authorization/fe-authorization-spi/pom.xml +++ b/fe/fe-authorization/fe-authorization-spi/pom.xml @@ -44,6 +44,20 @@ under the License. + + + + src/main/resources + + + + src/main/resources-filtered + true + + org.apache.maven.plugins diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java index a14482b4404edd..cf1be89df43bb7 100644 --- a/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java +++ b/fe/fe-authorization/fe-authorization-spi/src/main/java/org/apache/doris/authorization/spi/AuthorizationPluginFactory.java @@ -17,6 +17,9 @@ package org.apache.doris.authorization.spi; +import org.apache.doris.extension.spi.Plugin; +import org.apache.doris.extension.spi.PluginFactory; + import java.util.Map; /** @@ -25,17 +28,23 @@ *

A plugin is created once and kept: unlike an authentication attempt, an authorization decision happens * many times within a single statement, and a source that caches policies has to be the same instance * throughout. The engine builds a new one only when what configures it changes.

+ * + *

Extending {@link PluginFactory} is what lets a jar in {@code plugins/authorization/} be discovered by + * the same loader the other plugin families use, and therefore be held to the same plugin API version + * contract before any of its code runs.

*/ -public interface AuthorizationPluginFactory { +public interface AuthorizationPluginFactory extends PluginFactory { /** * The name this source is selected by in configuration, and the name its plugin reports. * * @return plugin name, e.g. {@code "ranger-doris"} */ + @Override String name(); /** One line about what this source is, for logs and diagnostics. */ + @Override default String description() { return ""; } @@ -45,4 +54,16 @@ default String description() { * @param context what the engine will answer if this plugin asks; see {@link AuthorizationContext} */ AuthorizationPlugin create(Map properties, AuthorizationContext context); + + /** + * Never called for an authorization source: an authorization plugin cannot be built without the context + * it asks the engine questions through, so the engine only ever calls + * {@link #create(Map, AuthorizationContext)}. Present because {@link PluginFactory} declares it, which is + * what makes this factory discoverable by the shared plugin loader. + */ + @Override + default Plugin create() { + throw new UnsupportedOperationException("AuthorizationPluginFactory does not support no-arg create();" + + " an authorization source is built with create(Map, AuthorizationContext)"); + } } diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties b/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties new file mode 100644 index 00000000000000..20541eeeaca6d1 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties @@ -0,0 +1,7 @@ +# The authorization plugin API version this FE serves, filtered from +# in fe/fe-authorization/pom.xml at build time. +# +# A plugin loaded from plugins/authorization/ must declare the same MAJOR in its jar MANIFEST, under +# Doris-Authorization-Plugin-Api-Version. Do not edit this file: it is generated, and the property in the +# pom is the single place the number is written. +api.version=${authorization.plugin.api.version} diff --git a/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginSurfaceTest.java b/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginSurfaceTest.java new file mode 100644 index 00000000000000..c47ae3917c6c7b --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/test/java/org/apache/doris/authorization/spi/AuthorizationPluginSurfaceTest.java @@ -0,0 +1,305 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorization.spi; + +import org.apache.doris.authorization.AccessRequirements; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; +import java.lang.reflect.GenericArrayType; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.lang.reflect.ParameterizedType; +import java.lang.reflect.Type; +import java.lang.reflect.TypeVariable; +import java.lang.reflect.WildcardType; +import java.nio.charset.StandardCharsets; +import java.util.ArrayDeque; +import java.util.Arrays; +import java.util.Deque; +import java.util.HashSet; +import java.util.List; +import java.util.Set; +import java.util.TreeSet; + +/** + * Freezes the AUTHORIZATION plugin API surface, so that changing it cannot happen without also deciding the + * version consequence. + * + *

Why this exists. A plugin loaded from {@code plugins/authorization/} was compiled against some + * release's version of these types and is admitted by matching majors alone + * ({@code Doris-Authorization-Plugin-Api-Version}). That is only sound if "major" really means any + * change to what a plugin can see - additions included. No unit test can prove somebody bumped the property + * (a test sees the current state, never the delta), so this is a speed bump rather than a gate: it makes the + * change visible in review, in the same commit, with the reason spelled out in the failure message. + * + *

Regenerating. Run this test, copy the "actual" block out of the failure message into + * {@code src/test/resources/authorization-plugin-surface.txt}, and bump the major of + * {@code authorization.plugin.api.version} in {@code fe/fe-authorization/pom.xml} in the SAME commit. + * + *

Why the surface is computed rather than listed

+ * + *

The other families freeze a hand-written list of SPI interfaces. That is enough for them because what + * crosses their boundary is data a plugin passes through. It is not enough here: an authorization plugin + * decides with the fe-authorization-api vocabulary, and one more {@code ResourceKind} or + * {@code AccessAction} constant silently turns every deployed plugin's "a kind I do not recognise" branch - + * which the contract requires to be a refusal - into a denial of something that used to be allowed. Nobody + * rebuilt those plugins, and a hand-written list would have stayed green. + * + *

So the frozen set is a closure: start from what a plugin implements or is handed, and follow every + * signature, field, nested type and supertype that stays inside {@code org.apache.doris}. A type the closure + * cannot reach is a type no plugin can encounter, so leaving it out is not a gap. + * + *

{@code Plugin} / {@code PluginFactory} / {@code PluginContext} from fe-extension-spi are frozen here + * too, and identically in the other families' baselines. They are loaded parent-first for every family, so a + * change to them breaks all plugin kinds at once - and turns all baselines red at once, each asking for its + * own bump. + */ +public class AuthorizationPluginSurfaceTest { + + private static final String BASELINE_RESOURCE = "/authorization-plugin-surface.txt"; + + /** + * Where the closure starts: the two interfaces a plugin implements, the context the engine hands it, the + * three shared plugin types - and the named requirements, which no signature mentions because a plugin + * reads them as constants to recognise which question it is being asked. + */ + private static final List> SEED_TYPES = Arrays.asList( + AuthorizationPluginFactory.class, + AuthorizationPlugin.class, + AuthorizationContext.class, + AccessRequirements.class, + org.apache.doris.extension.spi.Plugin.class, + org.apache.doris.extension.spi.PluginFactory.class, + org.apache.doris.extension.spi.PluginContext.class); + + @Test + public void pluginApiSurfaceMatchesRecordedBaseline() throws IOException { + TreeSet actual = renderSurface(); + TreeSet expected = readBaseline(); + + TreeSet missing = new TreeSet<>(expected); + missing.removeAll(actual); + TreeSet added = new TreeSet<>(actual); + added.removeAll(expected); + + Assertions.assertTrue(missing.isEmpty() && added.isEmpty(), + "The AUTHORIZATION plugin API surface changed.\n" + + " gone from the baseline (removed, renamed, or re-signed): " + missing + "\n" + + " new since the baseline: " + added + "\n" + + "THIS IS A MAJOR CHANGE - the same commit that refreshes src/test/resources" + + BASELINE_RESOURCE + " must increment the major of" + + " in fe/fe-authorization/pom.xml (and zero its" + + " minor).\n" + + "Full actual surface:\n" + String.join("\n", actual)); + } + + @Test + public void theClosureReachesTheDecisionVocabulary() { + // Guards the mechanism above rather than the surface itself: were the walk to stop at the SPI + // interfaces, the baseline would still match on the day someone adds a resource kind, and the + // check this class exists for would be silently gone. + TreeSet surface = renderSurface(); + for (String required : new String[] { + "org.apache.doris.authorization.ResourceKind@TABLE", + "org.apache.doris.authorization.AccessAction@SELECT", + "org.apache.doris.authorization.AuthorizedResource$Table!final class", + "org.apache.doris.authorization.RowFilterMergeType@PERMISSIVE"}) { + Assertions.assertTrue(surface.contains(required), + "the frozen surface no longer reaches " + required + + "; a plugin decides with that type, so it must be frozen. Surface:\n" + + String.join("\n", surface)); + } + } + + /** + * One line per element a plugin can see, keyed by the type it is reachable on rather than by whichever + * supertype declares it: what matters is what a plugin can call on the type it was handed, so moving a + * default method up or down a super-interface chain is not by itself a surface change. + */ + private static TreeSet renderSurface() { + TreeSet rendered = new TreeSet<>(); + Set> visited = new HashSet<>(); + Deque> pending = new ArrayDeque<>(SEED_TYPES); + while (!pending.isEmpty()) { + Class type = pending.poll(); + if (!isFrozen(type) || !visited.add(type)) { + continue; + } + render(type, rendered, pending); + } + return rendered; + } + + /** In scope iff it is one of ours: JDK types are a dependency of the surface, not part of it. */ + private static boolean isFrozen(Class type) { + Class element = type; + while (element.isArray()) { + element = element.getComponentType(); + } + return !element.isPrimitive() && isOurs(element); + } + + /** + * Whether we are the ones who declared it. Members inherited from the JDK are excluded for a practical + * reason as much as a conceptual one: {@code Enum} and {@code Throwable} grow methods between Java + * releases ({@code describeConstable} arrived in Java 12), and a baseline that recorded them would go red + * on a compiler upgrade and demand a major bump nothing about this SPI justifies - training whoever hits + * it to regenerate without reading. + */ + private static boolean isOurs(Class declaringClass) { + return declaringClass.getName().startsWith("org.apache.doris."); + } + + private static void render(Class type, TreeSet rendered, Deque> pending) { + rendered.add(type.getName() + "!" + kindOf(type)); + if (type.isEnum()) { + for (Object constant : type.getEnumConstants()) { + rendered.add(type.getName() + "@" + ((Enum) constant).name()); + } + } + for (Class nested : type.getClasses()) { + // A closed hierarchy is expressed as nested final subclasses, which no signature mentions + // although every plugin casts to them. + pending.add(nested); + } + pending.add(type.getSuperclass() == null ? Object.class : type.getSuperclass()); + pending.addAll(Arrays.asList(type.getInterfaces())); + + for (Field field : type.getFields()) { + if (field.isSynthetic() || !isOurs(field.getDeclaringClass()) || field.isEnumConstant()) { + // Enum constants are recorded above, in the form that says what actually matters about + // them - that the constant exists - rather than restating their type. + continue; + } + rendered.add(type.getName() + "$" + field.getName() + ":" + field.getGenericType().getTypeName()); + expand(field.getGenericType(), pending); + } + for (Constructor constructor : type.getConstructors()) { + if (constructor.isSynthetic()) { + continue; + } + rendered.add(signature(type, "", constructor.getGenericParameterTypes(), void.class)); + expandAll(constructor.getGenericParameterTypes(), pending); + } + for (Method method : type.getMethods()) { + if (method.isSynthetic() || !isOurs(method.getDeclaringClass()) + || (type.isEnum() && Modifier.isStatic(method.getModifiers()) + && ("values".equals(method.getName()) || "valueOf".equals(method.getName())))) { + continue; + } + rendered.add(signature(type, method.getName(), method.getGenericParameterTypes(), + method.getGenericReturnType())); + expandAll(method.getGenericParameterTypes(), pending); + expand(method.getGenericReturnType(), pending); + expandAll(method.getGenericExceptionTypes(), pending); + } + } + + private static void expandAll(Type[] types, Deque> pending) { + for (Type type : types) { + expand(type, pending); + } + } + + /** + * Follows a declared type to every class mentioned in it, type arguments included. Erasure is not good + * enough here: {@code List} erases to {@code java.util.List}, and the payload types a + * plugin builds its answers out of are reachable only through the argument. + * + *

Iterative with a seen set rather than recursive, because a type variable can name itself: + * {@code Enum>} arrives on every enum's {@code compareTo} and walks forever. + */ + private static void expand(Type root, Deque> pending) { + Set seen = new HashSet<>(); + Deque todo = new ArrayDeque<>(); + todo.add(root); + while (!todo.isEmpty()) { + Type type = todo.poll(); + if (type == null || !seen.add(type)) { + continue; + } + if (type instanceof Class) { + Class clazz = (Class) type; + pending.add(clazz.isArray() ? clazz.getComponentType() : clazz); + } else if (type instanceof ParameterizedType) { + ParameterizedType parameterized = (ParameterizedType) type; + todo.add(parameterized.getRawType()); + todo.addAll(Arrays.asList(parameterized.getActualTypeArguments())); + } else if (type instanceof GenericArrayType) { + todo.add(((GenericArrayType) type).getGenericComponentType()); + } else if (type instanceof WildcardType) { + todo.addAll(Arrays.asList(((WildcardType) type).getUpperBounds())); + todo.addAll(Arrays.asList(((WildcardType) type).getLowerBounds())); + } else if (type instanceof TypeVariable) { + todo.addAll(Arrays.asList(((TypeVariable) type).getBounds())); + } + } + } + + /** + * The declaration kind, because it is part of what a plugin may do with a type: an interface that becomes + * a class stops being implementable, and a final class that stops being final stops being a closed set + * the engine can enumerate. + */ + private static String kindOf(Class type) { + if (type.isEnum()) { + return "enum"; + } + if (type.isInterface()) { + return "interface"; + } + if (Modifier.isAbstract(type.getModifiers())) { + return "abstract class"; + } + return Modifier.isFinal(type.getModifiers()) ? "final class" : "class"; + } + + private static String signature(Class owner, String name, Type[] params, Type returnType) { + StringBuilder sb = new StringBuilder(owner.getName()).append('#').append(name).append('('); + for (int i = 0; i < params.length; i++) { + if (i > 0) { + sb.append(','); + } + sb.append(params[i].getTypeName()); + } + return sb.append("):").append(returnType.getTypeName()).toString(); + } + + private static TreeSet readBaseline() throws IOException { + TreeSet baseline = new TreeSet<>(); + try (InputStream in = AuthorizationPluginSurfaceTest.class.getResourceAsStream(BASELINE_RESOURCE)) { + Assertions.assertNotNull(in, "missing test resource " + BASELINE_RESOURCE); + BufferedReader reader = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8)); + String line; + while ((line = reader.readLine()) != null) { + if (!line.trim().isEmpty()) { + baseline.add(line.trim()); + } + } + } + return baseline; + } +} diff --git a/fe/fe-authorization/fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt b/fe/fe-authorization/fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt new file mode 100644 index 00000000000000..0af27b7c5ef973 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt @@ -0,0 +1,221 @@ +org.apache.doris.authorization.AccessAction!enum +org.apache.doris.authorization.AccessAction@ADMIN +org.apache.doris.authorization.AccessAction@ALTER +org.apache.doris.authorization.AccessAction@CLUSTER_USAGE +org.apache.doris.authorization.AccessAction@CREATE +org.apache.doris.authorization.AccessAction@DROP +org.apache.doris.authorization.AccessAction@GRANT +org.apache.doris.authorization.AccessAction@LOAD +org.apache.doris.authorization.AccessAction@NODE +org.apache.doris.authorization.AccessAction@SELECT +org.apache.doris.authorization.AccessAction@SHOW_VIEW +org.apache.doris.authorization.AccessAction@STAGE_USAGE +org.apache.doris.authorization.AccessAction@USAGE +org.apache.doris.authorization.AccessContext!interface +org.apache.doris.authorization.AccessContext#getClientIp():java.util.Optional +org.apache.doris.authorization.AccessContext#getQueryId():java.util.Optional +org.apache.doris.authorization.AccessContext$NONE:org.apache.doris.authorization.AccessContext +org.apache.doris.authorization.AccessDeniedException!class +org.apache.doris.authorization.AccessDeniedException#getDeniedBy():java.util.Optional +org.apache.doris.authorization.AccessDeniedException#getMessage():java.lang.String +org.apache.doris.authorization.AccessDeniedException#getRequirement():java.util.Optional +org.apache.doris.authorization.AccessDeniedException#getResource():org.apache.doris.authorization.AuthorizedResource +org.apache.doris.authorization.AccessDeniedException#of(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AuthorizedResource,org.apache.doris.authorization.AccessRequirement,java.lang.String):org.apache.doris.authorization.AccessDeniedException +org.apache.doris.authorization.AccessDeniedException#withMessage(java.lang.String,org.apache.doris.authorization.AuthorizedResource,java.lang.String):org.apache.doris.authorization.AccessDeniedException +org.apache.doris.authorization.AccessRequirement!final class +org.apache.doris.authorization.AccessRequirement#allOf(org.apache.doris.authorization.AccessAction[]):org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirement#anyOf(org.apache.doris.authorization.AccessAction[]):org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirement#equals(java.lang.Object):boolean +org.apache.doris.authorization.AccessRequirement#getActions():java.util.Set +org.apache.doris.authorization.AccessRequirement#getMatch():org.apache.doris.authorization.ActionMatch +org.apache.doris.authorization.AccessRequirement#hashCode():int +org.apache.doris.authorization.AccessRequirement#isSatisfiedBy(java.util.Set):boolean +org.apache.doris.authorization.AccessRequirement#of(java.util.Collection,org.apache.doris.authorization.ActionMatch):org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirement#of(org.apache.doris.authorization.AccessAction):org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirement#toString():java.lang.String +org.apache.doris.authorization.AccessRequirements!final class +org.apache.doris.authorization.AccessRequirements$ADMINISTRATION:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$ALTER:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$ANY_PRIVILEGE:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$CREATE:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$DROP:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$LOAD:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$SELECT:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.AccessRequirements$VISIBILITY:org.apache.doris.authorization.AccessRequirement +org.apache.doris.authorization.ActionMatch!enum +org.apache.doris.authorization.ActionMatch@ALL +org.apache.doris.authorization.ActionMatch@ANY +org.apache.doris.authorization.AuthorizedResource!abstract class +org.apache.doris.authorization.AuthorizedResource#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Catalog!final class +org.apache.doris.authorization.AuthorizedResource$Catalog#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Catalog#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Catalog#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Catalog#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Catalog#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedResource$Catalog#getCatalog():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Catalog#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Catalog#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Catalog#hashCode():int +org.apache.doris.authorization.AuthorizedResource$Catalog#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Catalog#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Catalog#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Catalog#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Catalog#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Columns!final class +org.apache.doris.authorization.AuthorizedResource$Columns#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Columns#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Columns#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Columns#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Columns#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedResource$Columns#getCatalog():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Columns#getColumns():java.util.Set +org.apache.doris.authorization.AuthorizedResource$Columns#getDatabase():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Columns#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Columns#getTable():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Columns#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Columns#hashCode():int +org.apache.doris.authorization.AuthorizedResource$Columns#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Columns#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Columns#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Columns#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Columns#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Database!final class +org.apache.doris.authorization.AuthorizedResource$Database#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Database#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Database#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Database#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Database#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedResource$Database#getCatalog():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Database#getDatabase():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Database#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Database#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Database#hashCode():int +org.apache.doris.authorization.AuthorizedResource$Database#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Database#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Database#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Database#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Database#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Global!final class +org.apache.doris.authorization.AuthorizedResource$Global#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Global#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Global#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Global#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Global#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Global#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Global#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Global#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Global#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Global#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Global#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Named!final class +org.apache.doris.authorization.AuthorizedResource$Named#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Named#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Named#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Named#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Named#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedResource$Named#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Named#getName():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Named#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Named#hashCode():int +org.apache.doris.authorization.AuthorizedResource$Named#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Named#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Named#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Named#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Named#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Table!final class +org.apache.doris.authorization.AuthorizedResource$Table#catalog(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Catalog +org.apache.doris.authorization.AuthorizedResource$Table#cloud(org.apache.doris.authorization.ResourceKind,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Table#columns(java.lang.String,java.lang.String,java.lang.String,java.util.Set):org.apache.doris.authorization.AuthorizedResource$Columns +org.apache.doris.authorization.AuthorizedResource$Table#database(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Database +org.apache.doris.authorization.AuthorizedResource$Table#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedResource$Table#getCatalog():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Table#getDatabase():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Table#getKind():org.apache.doris.authorization.ResourceKind +org.apache.doris.authorization.AuthorizedResource$Table#getTable():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Table#global():org.apache.doris.authorization.AuthorizedResource$Global +org.apache.doris.authorization.AuthorizedResource$Table#hashCode():int +org.apache.doris.authorization.AuthorizedResource$Table#resource(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Table#storageVault(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedResource$Table#table(java.lang.String,java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedResource$Table +org.apache.doris.authorization.AuthorizedResource$Table#toString():java.lang.String +org.apache.doris.authorization.AuthorizedResource$Table#workloadGroup(java.lang.String):org.apache.doris.authorization.AuthorizedResource$Named +org.apache.doris.authorization.AuthorizedSubject!final class +org.apache.doris.authorization.AuthorizedSubject#equals(java.lang.Object):boolean +org.apache.doris.authorization.AuthorizedSubject#getHost():java.lang.String +org.apache.doris.authorization.AuthorizedSubject#getUser():java.lang.String +org.apache.doris.authorization.AuthorizedSubject#hashCode():int +org.apache.doris.authorization.AuthorizedSubject#isDomain():boolean +org.apache.doris.authorization.AuthorizedSubject#of(java.lang.String,java.lang.String):org.apache.doris.authorization.AuthorizedSubject +org.apache.doris.authorization.AuthorizedSubject#of(java.lang.String,java.lang.String,boolean):org.apache.doris.authorization.AuthorizedSubject +org.apache.doris.authorization.AuthorizedSubject#toString():java.lang.String +org.apache.doris.authorization.DataMaskSpec!final class +org.apache.doris.authorization.DataMaskSpec#(java.lang.String,java.lang.String):void +org.apache.doris.authorization.DataMaskSpec#equals(java.lang.Object):boolean +org.apache.doris.authorization.DataMaskSpec#getMaskSql():java.lang.String +org.apache.doris.authorization.DataMaskSpec#getPolicyIdent():java.lang.String +org.apache.doris.authorization.DataMaskSpec#hashCode():int +org.apache.doris.authorization.DataMaskSpec#toString():java.lang.String +org.apache.doris.authorization.ResourceKind!enum +org.apache.doris.authorization.ResourceKind@CATALOG +org.apache.doris.authorization.ResourceKind@CLOUD_COMPUTE_GROUP +org.apache.doris.authorization.ResourceKind@CLOUD_GENERAL +org.apache.doris.authorization.ResourceKind@CLOUD_STAGE +org.apache.doris.authorization.ResourceKind@CLOUD_STORAGE_VAULT +org.apache.doris.authorization.ResourceKind@COLUMNS +org.apache.doris.authorization.ResourceKind@DATABASE +org.apache.doris.authorization.ResourceKind@GLOBAL +org.apache.doris.authorization.ResourceKind@RESOURCE +org.apache.doris.authorization.ResourceKind@STORAGE_VAULT +org.apache.doris.authorization.ResourceKind@TABLE +org.apache.doris.authorization.ResourceKind@WORKLOAD_GROUP +org.apache.doris.authorization.RowFilterMergeType!enum +org.apache.doris.authorization.RowFilterMergeType@PERMISSIVE +org.apache.doris.authorization.RowFilterMergeType@RESTRICTIVE +org.apache.doris.authorization.RowFilterSpec!final class +org.apache.doris.authorization.RowFilterSpec#(java.lang.String,java.lang.String,org.apache.doris.authorization.RowFilterMergeType):void +org.apache.doris.authorization.RowFilterSpec#equals(java.lang.Object):boolean +org.apache.doris.authorization.RowFilterSpec#getFilterSql():java.lang.String +org.apache.doris.authorization.RowFilterSpec#getMergeType():org.apache.doris.authorization.RowFilterMergeType +org.apache.doris.authorization.RowFilterSpec#getPolicyIdent():java.lang.String +org.apache.doris.authorization.RowFilterSpec#hashCode():int +org.apache.doris.authorization.RowFilterSpec#restrictive(java.lang.String,java.lang.String):org.apache.doris.authorization.RowFilterSpec +org.apache.doris.authorization.RowFilterSpec#toString():java.lang.String +org.apache.doris.authorization.spi.AuthorizationContext!interface +org.apache.doris.authorization.spi.AuthorizationContext#grantedByGlobalScopeAuthority(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AccessRequirement):boolean +org.apache.doris.authorization.spi.AuthorizationContext#ownerOf(org.apache.doris.authorization.AuthorizedResource):java.util.Optional +org.apache.doris.authorization.spi.AuthorizationContext#rolesOf(org.apache.doris.authorization.AuthorizedSubject):java.util.Set +org.apache.doris.authorization.spi.AuthorizationPlugin!interface +org.apache.doris.authorization.spi.AuthorizationPlugin#checkAction(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AuthorizedResource,org.apache.doris.authorization.AccessAction,org.apache.doris.authorization.AccessContext):void +org.apache.doris.authorization.spi.AuthorizationPlugin#checkPrivilege(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AuthorizedResource,org.apache.doris.authorization.AccessRequirement,org.apache.doris.authorization.AccessContext):void +org.apache.doris.authorization.spi.AuthorizationPlugin#close():void +org.apache.doris.authorization.spi.AuthorizationPlugin#getDataMasks(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AuthorizedResource$Table,java.util.Set,org.apache.doris.authorization.AccessContext):java.util.Map +org.apache.doris.authorization.spi.AuthorizationPlugin#getRowFilters(org.apache.doris.authorization.AuthorizedSubject,org.apache.doris.authorization.AuthorizedResource$Table,org.apache.doris.authorization.AccessContext):java.util.List +org.apache.doris.authorization.spi.AuthorizationPlugin#initialize(org.apache.doris.extension.spi.PluginContext):void +org.apache.doris.authorization.spi.AuthorizationPlugin#name():java.lang.String +org.apache.doris.authorization.spi.AuthorizationPluginFactory!interface +org.apache.doris.authorization.spi.AuthorizationPluginFactory#create():org.apache.doris.extension.spi.Plugin +org.apache.doris.authorization.spi.AuthorizationPluginFactory#create(java.util.Map,org.apache.doris.authorization.spi.AuthorizationContext):org.apache.doris.authorization.spi.AuthorizationPlugin +org.apache.doris.authorization.spi.AuthorizationPluginFactory#create(org.apache.doris.extension.spi.PluginContext):org.apache.doris.extension.spi.Plugin +org.apache.doris.authorization.spi.AuthorizationPluginFactory#description():java.lang.String +org.apache.doris.authorization.spi.AuthorizationPluginFactory#name():java.lang.String +org.apache.doris.extension.spi.Plugin!interface +org.apache.doris.extension.spi.Plugin#close():void +org.apache.doris.extension.spi.Plugin#initialize(org.apache.doris.extension.spi.PluginContext):void +org.apache.doris.extension.spi.PluginContext!final class +org.apache.doris.extension.spi.PluginContext#(java.util.Map):void +org.apache.doris.extension.spi.PluginContext#getProperties():java.util.Map +org.apache.doris.extension.spi.PluginFactory!interface +org.apache.doris.extension.spi.PluginFactory#create():org.apache.doris.extension.spi.Plugin +org.apache.doris.extension.spi.PluginFactory#create(org.apache.doris.extension.spi.PluginContext):org.apache.doris.extension.spi.Plugin +org.apache.doris.extension.spi.PluginFactory#description():java.lang.String +org.apache.doris.extension.spi.PluginFactory#name():java.lang.String diff --git a/fe/fe-authorization/pom.xml b/fe/fe-authorization/pom.xml index c627ee7aa88a26..520dffeb69c193 100644 --- a/fe/fe-authorization/pom.xml +++ b/fe/fe-authorization/pom.xml @@ -29,13 +29,53 @@ under the License. fe-authorization pom Doris FE Authorization - + + + + 1.0 + + + + + + + org.apache.maven.plugins + maven-jar-plugin + + + + ${authorization.plugin.api.version} + + + + + + + fe-authorization-api fe-authorization-spi diff --git a/fe/fe-connector/fe-connector-spi/src/test/java/org/apache/doris/connector/spi/ConnectorPluginSurfaceTest.java b/fe/fe-connector/fe-connector-spi/src/test/java/org/apache/doris/connector/spi/ConnectorPluginSurfaceTest.java index 015ba4aaa4cea6..e1b1cd2401d89c 100644 --- a/fe/fe-connector/fe-connector-spi/src/test/java/org/apache/doris/connector/spi/ConnectorPluginSurfaceTest.java +++ b/fe/fe-connector/fe-connector-spi/src/test/java/org/apache/doris/connector/spi/ConnectorPluginSurfaceTest.java @@ -52,7 +52,7 @@ * {@code fe/fe-connector/pom.xml} in the SAME commit. * *

{@code Plugin} / {@code PluginFactory} / {@code PluginContext} from fe-extension-spi are frozen here - * too, and identically in the other three families' baselines. They are loaded parent-first for every family + * too, and identically in the other four families' baselines. They are loaded parent-first for every family * (see {@code ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES}), so a change to them breaks all four * plugin kinds at once — and turns all four baselines red at once, each asking for its own bump. * diff --git a/fe/fe-connector/pom.xml b/fe/fe-connector/pom.xml index dc9ab08118d1f1..b989c02d20a469 100644 --- a/fe/fe-connector/pom.xml +++ b/fe/fe-connector/pom.xml @@ -53,7 +53,7 @@ under the License. The contract this version describes spans fe-connector-spi and - because connector plugins link them too - fe-extension-spi and fe-filesystem-api. Changing either of the latter two means bumping this property as well (and fe-extension-spi means bumping - all four families). + all five families). --> 5.0 diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index 0585cbf0d18e45..8b90a10bf0dd67 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -42,7 +42,7 @@ under the License. (org.apache.doris.nereids.lineage) is part of fe-core itself. Bump the MAJOR (and zero the minor) in the SAME commit as ANY change to the lineage SPI - surface - additions included. A change to fe-extension-spi means bumping all four families. + surface - additions included. A change to fe-extension-spi means bumping all five families. See plan-doc/designs/2026-07-29-plugin-api-version-check-design.md. --> 1.0 diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerFactory.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerFactory.java index 8d1481aa070d39..09e4c9588062c9 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerFactory.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerFactory.java @@ -19,6 +19,16 @@ import java.util.Map; +/** + * Publishes a {@link CatalogAccessController}, the older shape of an authorization source. + * + * @deprecated implement {@link org.apache.doris.authorization.spi.AuthorizationPluginFactory} instead, and + * ship the plugin as a subdirectory of {@code authorization_plugins_dir} with the authorization + * plugin API version declared in its jar manifest. A factory found through this interface is still + * loaded, and a name published both ways resolves to the newer one; but this channel carries no + * declared API version, so a plugin built against an older Doris is admitted with no diagnosis. + */ +@Deprecated public interface AccessControllerFactory { /** * Returns the identifier for the factory, such as "range-doris". diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index b3d055110203bb..47c11576687b89 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -38,6 +38,13 @@ import org.apache.doris.datasource.CatalogMgr; import org.apache.doris.datasource.ExternalCatalog; import org.apache.doris.datasource.InternalCatalog; +import org.apache.doris.extension.loader.ApiVersionGate; +import org.apache.doris.extension.loader.ClassLoadingPolicy; +import org.apache.doris.extension.loader.DirectoryPluginRuntimeManager; +import org.apache.doris.extension.loader.LoadFailure; +import org.apache.doris.extension.loader.LoadReport; +import org.apache.doris.extension.loader.PluginHandle; +import org.apache.doris.extension.loader.PluginRegistry; import org.apache.doris.plugin.PropertiesUtils; import org.apache.doris.qe.ConnectContext; @@ -49,6 +56,9 @@ import org.apache.logging.log4j.Logger; import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; import java.util.Collections; import java.util.List; import java.util.Map; @@ -57,6 +67,7 @@ import java.util.ServiceLoader; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.CopyOnWriteArrayList; /** * AccessControllerManager is the entry point of privilege authentication. @@ -71,6 +82,27 @@ public class AccessControllerManager { private static final Logger LOG = LogManager.getLogger(AccessControllerManager.class); + /** + * The authorization plugin API contract this FE serves. Built from the version filtered into + * fe-authorization-spi at build time, anchored on {@link AuthorizationPluginFactory} so that it is read + * from the very artifact carrying the SPI. A missing or malformed resource is a build defect and fails + * class initialization loudly rather than degrading into a check that admits everything. + */ + private static final ApiVersionGate API_VERSION_GATE = + ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + + /** + * Loaded from the FE rather than from the plugin jar, so that the types crossing the boundary - the + * decision vocabulary in {@code org.apache.doris.authorization} and the contract in its {@code .spi} + * sub-package - exist exactly once. A plugin carrying its own copy would hand back objects the engine + * refuses to recognise as the types it asked for. + */ + private static final List AUTHORIZATION_PARENT_FIRST_PREFIXES = + Collections.singletonList("org.apache.doris.authorization."); + + /** Family label in the process-wide {@link PluginRegistry}, i.e. in information_schema.extensions. */ + private static final String PLUGIN_FAMILY = "AUTHORIZATION"; + private Auth auth; // Governs everything no catalog-bound source governs; the built-in model unless configured otherwise private AuthorizationPlugin defaultAccessController; @@ -85,6 +117,18 @@ public class AccessControllerManager { = new ConcurrentHashMap<>(); // Mapping between access controller class names and their identifiers for easy lookup of factory identifiers private ConcurrentHashMap accessControllerClassNameMapping = new ConcurrentHashMap<>(); + // Holds the classloader of every plugin loaded from a directory, for the lifetime of the FE + private final DirectoryPluginRuntimeManager pluginDirectoryRuntime = + new DirectoryPluginRuntimeManager<>(); + /** + * Plugin directories refused on the API version they declared. + * + *

Kept because the refusal and the complaint happen in different places: the load is a startup sweep + * that logs and carries on, while what an operator sees is "no authorization plugin factory found for + * {@code }" from whoever asked for that name. Without this, a plugin refused on its version would be + * indistinguishable from one that was never installed. + */ + private final List apiVersionRejections = new CopyOnWriteArrayList<>(); public AccessControllerManager(Auth auth) { this.auth = auth; @@ -117,7 +161,8 @@ private AuthorizationPlugin loadAccessControllerOrThrow(String accessControllerN } if (!isKnownAuthorizationSource(accessControllerName)) { throw new RuntimeException("No authorization plugin factory found for " + accessControllerName - + ". Please confirm that your plugin is placed in the correct location."); + + ". Please confirm that your plugin is placed in the correct location." + + apiVersionRejectionHint()); } Map prop; try { @@ -158,13 +203,16 @@ private boolean isKnownAuthorizationSource(String name) { } private void loadAccessControllerPlugins() { - // Sources shipped with the FE, and any on its class path. Loading them from a plugin directory is - // what the version gate guards, so that channel opens with the gate rather than before it. + // Sources shipped with the FE, and any on its class path. Deliberately not held to the plugin API + // version: what is on the class path was built from this same source tree in the same build, so the + // version there would be a number compared against itself. The gate exists for the directory + // channel below, where a jar built against some other Doris release can turn up. for (AuthorizationPluginFactory factory : ServiceLoader.load(AuthorizationPluginFactory.class)) { LOG.info("Found authorization plugin factory: {} from class path.", factory.name()); - authorizationPluginFactories.put(factory.name(), factory); - accessControllerClassNameMapping.put(factory.getClass().getName(), factory.name()); + registerPluginFactory(factory); + PluginRegistry.getInstance().registerBuiltin(PLUGIN_FAMILY, factory); } + loadAuthorizationPluginsFromDirectory(); ServiceLoader loaderFromClasspath = ServiceLoader.load(AccessControllerFactory.class); for (AccessControllerFactory factory : loaderFromClasspath) { LOG.info("Found Authentication Plugin Factories: {} from class path.", factory.factoryIdentifier()); @@ -182,17 +230,111 @@ private void loadAccessControllerPlugins() { } } + /** + * Loads authorization plugins from {@code authorization_plugins_dir}, laid out one plugin per + * subdirectory: {@code

/*.jar} plus {@code /lib/*.jar}. + * + *

A directory that fails is logged and skipped: one unusable plugin must not stop an FE from + * starting, and if the failed one is the very source {@code access_controller_type} names, the + * constructor refuses right afterwards anyway - with the reason attached, see + * {@link #apiVersionRejectionHint()}. + * + *

This is a different layout from the one the deprecated {@link AccessControllerFactory} channel + * reads out of the same directory, which takes jars lying loose at its root. The two cannot collide: + * that channel lists only files, this one lists only subdirectories. + */ + private void loadAuthorizationPluginsFromDirectory() { + List pluginRoots = new ArrayList<>(); + for (Path root : ClassLoaderUtils.parsePluginRootDirectories(Config.authorization_plugins_dir)) { + if (Files.isDirectory(root)) { + pluginRoots.add(root); + } else { + // Having nowhere to put plugins is the normal state of an FE with none, so this is not a + // warning: one that fires on every start teaches operators to skip the ones that matter. + LOG.info("No authorization plugin directory at {}; skipping the directory channel.", root); + } + } + if (pluginRoots.isEmpty()) { + return; + } + LoadReport report = pluginDirectoryRuntime.loadAll( + pluginRoots, + AccessControllerManager.class.getClassLoader(), + AuthorizationPluginFactory.class, + new ClassLoadingPolicy(AUTHORIZATION_PARENT_FIRST_PREFIXES), + API_VERSION_GATE); + + apiVersionRejections.clear(); + for (LoadFailure failure : report.getFailures()) { + LOG.warn("Skip authorization plugin directory: pluginDir={}, stage={}, message={}", + failure.getPluginDir(), failure.getStage(), failure.getMessage(), failure.getCause()); + if (LoadFailure.STAGE_API_VERSION.equals(failure.getStage())) { + apiVersionRejections.add(failure.getMessage()); + } + } + + for (PluginHandle handle : report.getSuccesses()) { + String name = handle.getPluginName(); + if (authorizationPluginFactories.containsKey(name)) { + // Whatever is already installed under this name keeps it, so that dropping a jar into the + // plugin directory can never displace a source shipped with the FE. + LOG.warn("Skip authorization plugin '{}' from {}: that name is already taken by a plugin on" + + " the class path", name, handle.getPluginDir()); + pluginDirectoryRuntime.discard(name); + continue; + } + registerPluginFactory(handle.getFactory()); + // Only a plugin that was actually admitted gets an inventory row, so + // information_schema.extensions never lists an authorization source nothing can reach. + PluginRegistry.getInstance().registerExternal(PLUGIN_FAMILY, handle); + LOG.info("Loaded authorization plugin: name={}, pluginDir={}, jarCount={}", + name, handle.getPluginDir(), handle.getResolvedJars().size()); + } + } + + private void registerPluginFactory(AuthorizationPluginFactory factory) { + String name = factory.name(); + authorizationPluginFactories.put(name, factory); + // Keeps `access_controller.class = ` working for a source published this way, + // which is how a catalog written before plugin names existed still names its source. + accessControllerClassNameMapping.put(factory.getClass().getName(), name); + } + private void registerLegacyFactory(AccessControllerFactory factory) { String name = factory.factoryIdentifier(); if (authorizationPluginFactories.containsKey(name)) { // Both were found, so say which one answers rather than letting the loser look installed. LOG.warn("Authorization source {} is published both as a plugin and as an access controller" + " factory; the plugin is the one used.", name); + } else { + LOG.warn("Authorization source {} implements the deprecated {} interface. It keeps working, but" + + " that interface will be removed: implement {} instead and ship the plugin as" + + " a subdirectory of {}, declaring {}={} in its jar manifest.", + name, AccessControllerFactory.class.getName(), AuthorizationPluginFactory.class.getName(), + Config.authorization_plugins_dir, API_VERSION_GATE.getManifestAttribute(), + API_VERSION_GATE.getExpectedVersion()); } accessControllerFactoriesCache.put(name, factory); accessControllerClassNameMapping.put(factory.getClass().getName(), name); } + /** + * A clause naming any plugin the startup sweep refused on its declared API version, or the empty string + * when there was none. + * + *

Appended to "no authorization plugin factory found for {@code }". A refused plugin never reaches the + * factory table, and the sweep itself does not fail, so without this the version rejection would only + * ever be an FE log line nobody correlates with the failure they are looking at. + */ + private String apiVersionRejectionHint() { + if (apiVersionRejections.isEmpty()) { + return ""; + } + return " Note that " + apiVersionRejections.size() + + " plugin(s) were refused on their declared API version: " + + String.join("; ", apiVersionRejections); + } + /** The authorization source governing the objects inside {@code ctl}. */ public AuthorizationPlugin getAccessControllerOrDefault(String ctl) { if (InternalCatalog.INTERNAL_CATALOG_NAME.equals(ctl)) { @@ -309,7 +451,8 @@ private String getPluginIdentifierForAccessController(String acClassName) { pluginIdentifier = acClassName; } if (null == pluginIdentifier || !isKnownAuthorizationSource(pluginIdentifier)) { - throw new RuntimeException("Access Controller Plugin Factory not found for " + acClassName); + throw new RuntimeException("Access Controller Plugin Factory not found for " + acClassName + + "." + apiVersionRejectionHint()); } return pluginIdentifier; } diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java index 13a17be97a15fd..6b8b57f6bef11e 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java @@ -41,7 +41,12 @@ * one through {@link LegacyAccessControllerPlugin}; a source written today implements * {@link org.apache.doris.authorization.spi.AuthorizationPlugin} instead, which asks a single question about * a typed resource and answers by refusing rather than by returning false. + * + * @deprecated implement {@link org.apache.doris.authorization.spi.AuthorizationPlugin} instead. This + * interface still works and is still what {@code access_controller.class} may name, but it is not + * held to a plugin API version, so nothing detects when a Doris upgrade changes what it means. */ +@Deprecated public interface CatalogAccessController { default void close() { } diff --git a/fe/fe-core/src/test/java/org/apache/doris/nereids/lineage/LineagePluginSurfaceTest.java b/fe/fe-core/src/test/java/org/apache/doris/nereids/lineage/LineagePluginSurfaceTest.java index f3cfff2b3d8ade..dee1130e0e9937 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/nereids/lineage/LineagePluginSurfaceTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/nereids/lineage/LineagePluginSurfaceTest.java @@ -47,7 +47,7 @@ * {@code fe/fe-core/pom.xml} in the SAME commit. * *

{@code Plugin} / {@code PluginFactory} / {@code PluginContext} from fe-extension-spi are frozen here - * too, and identically in the other three families' baselines. They are loaded parent-first for every family + * too, and identically in the other four families' baselines. They are loaded parent-first for every family * (see {@code ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES}), so a change to them breaks all four * plugin kinds at once — and turns all four baselines red at once, each asking for its own bump. * diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java index dee63c1e592325..9a51df27379577 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java @@ -18,13 +18,23 @@ package org.apache.doris.pluginapiversion; import org.apache.doris.authentication.spi.AuthenticationPluginFactory; +import org.apache.doris.authorization.ResourceKind; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; +import org.apache.doris.common.Config; +import org.apache.doris.common.jmockit.Deencapsulation; import org.apache.doris.connector.ConnectorPluginManager; import org.apache.doris.connector.spi.ConnectorProvider; +import org.apache.doris.datasource.InternalCatalog; import org.apache.doris.extension.loader.ApiVersionGate; +import org.apache.doris.extension.loader.DirectoryPluginRuntimeManager; import org.apache.doris.extension.loader.PluginRegistry; import org.apache.doris.filesystem.spi.FileSystemProvider; import org.apache.doris.fs.FileSystemPluginManager; +import org.apache.doris.mysql.privilege.AccessControllerManager; +import org.apache.doris.mysql.privilege.Auth; import org.apache.doris.nereids.lineage.LineagePluginFactory; +import org.apache.doris.pluginapiversion.testplugins.ShadowingAuthorizationPluginFactory; +import org.apache.doris.pluginapiversion.testplugins.VersionProbeAuthorizationPluginFactory; import org.apache.doris.pluginapiversion.testplugins.VersionProbeConnectorProvider; import org.apache.doris.pluginapiversion.testplugins.VersionProbeFileSystemProvider; @@ -41,7 +51,9 @@ import java.nio.file.Path; import java.util.Collections; import java.util.HashSet; +import java.util.List; import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; import java.util.jar.Attributes; import java.util.jar.JarEntry; import java.util.jar.JarOutputStream; @@ -146,23 +158,28 @@ public void filesystemPluginDeclaringNothingIsRefused() throws IOException { @Test public void everyFamilyDeclaresItsOwnIndependentContract() { - // Four properties, four resources, four attributes. The point of keeping them separate is that - // changing one family's SPI must not force plugins of the other three to be rebuilt (design 3.3); + // Five properties, five resources, five attributes. The point of keeping them separate is that + // changing one family's SPI must not force plugins of the other four to be rebuilt (design 3.3); // a shared attribute name or a shared resource would quietly undo that. ApiVersionGate connector = ApiVersionGate.forFamily("connector", ConnectorProvider.class); ApiVersionGate filesystem = ApiVersionGate.forFamily("filesystem", FileSystemProvider.class); ApiVersionGate authentication = ApiVersionGate.forFamily("authentication", AuthenticationPluginFactory.class); + ApiVersionGate authorization = + ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); ApiVersionGate lineage = ApiVersionGate.forFamily("lineage", LineagePluginFactory.class); Assertions.assertEquals("Doris-Connector-Plugin-Api-Version", connector.getManifestAttribute()); Assertions.assertEquals("Doris-Filesystem-Plugin-Api-Version", filesystem.getManifestAttribute()); Assertions.assertEquals("Doris-Authentication-Plugin-Api-Version", authentication.getManifestAttribute()); + Assertions.assertEquals("Doris-Authorization-Plugin-Api-Version", + authorization.getManifestAttribute()); Assertions.assertEquals("Doris-Lineage-Plugin-Api-Version", lineage.getManifestAttribute()); Set attributes = new HashSet<>(); - for (ApiVersionGate gate : new ApiVersionGate[] {connector, filesystem, authentication, lineage}) { + for (ApiVersionGate gate : new ApiVersionGate[] { + connector, filesystem, authentication, authorization, lineage}) { Assertions.assertTrue(gate.getExpectedMajor() >= 1, "a family's major starts at 1; 0 means the resource was read but never set"); Assertions.assertTrue(attributes.add(gate.getManifestAttribute()), @@ -170,6 +187,153 @@ public void everyFamilyDeclaresItsOwnIndependentContract() { } } + @Test + public void authorizationPluginDeclaringTheServedVersionGovernsTheInstance() throws IOException { + ApiVersionGate gate = ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + + AccessControllerManager manager = managerLoadingProbeFrom(gate.getExpectedVersion()); + + // The observable end of the whole channel: the source named in fe.conf is the one deciding. + Assertions.assertEquals(VersionProbeAuthorizationPluginFactory.NAME, + manager.getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME).name()); + } + + @Test + public void authorizationPluginDeclaringAnotherMajorIsRefusedWithBothVersionsNamed() throws IOException { + ApiVersionGate gate = ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + int otherMajor = gate.getExpectedMajor() + 1; + + String failure = refusalOf(otherMajor + ".0"); + + // Refusing is not enough: an operator has to be able to tell "installed but incompatible" from + // "never installed", which is the same message with nothing appended. + Assertions.assertTrue(failure.contains("refused on their declared API version"), failure); + Assertions.assertTrue(failure.contains(otherMajor + ".0"), failure); + Assertions.assertTrue(failure.contains(gate.getExpectedVersion()), failure); + } + + @Test + public void authorizationPluginDeclaringNothingIsRefused() throws IOException { + // The regression the version gate exists for: a plugin that says nothing about the API it was built + // against must not inherit the kernel's own answer and be admitted. + String failure = refusalOf(null); + + Assertions.assertTrue(failure.contains("refused on their declared API version"), failure); + Assertions.assertTrue(failure.contains("Doris-Authorization-Plugin-Api-Version"), failure); + } + + @Test + public void authorizationPluginFromADirectoryCannotTakeTheNameOfAShippedSource() throws IOException { + ApiVersionGate gate = ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + // Declares the version this FE serves, so the version gate is not what refuses it - the name is. + Path root = pluginRoot("authorization-shadow-root", "shadow-authz", + ShadowingAuthorizationPluginFactory.class, AuthorizationPluginFactory.class, + "Doris-Authorization-Plugin-Api-Version", gate.getExpectedVersion()); + String originalDir = Config.authorization_plugins_dir; + AccessControllerManager manager; + try { + Config.authorization_plugins_dir = root.toString(); + manager = new AccessControllerManager(new Auth()); + } finally { + Config.authorization_plugins_dir = originalDir; + } + + // The factory answering to that name must still be the FE's own, not the one from the directory. + ConcurrentHashMap factories = + Deencapsulation.getField(manager, "authorizationPluginFactories"); + AuthorizationPluginFactory installed = factories.get( + ShadowingAuthorizationPluginFactory.SHADOWED_NAME); + Assertions.assertNotNull(installed, "the shipped ranger-doris source disappeared from the test setup," + + " so this test would pass without proving anything"); + // By class NAME, not by Class object. A directory plugin is loaded through its own classloader, so + // its factory class is never the same object as the one this test holds - even when it did displace + // the shipped source. Written as an identity comparison first, this assertion could not fail at all: + // a mutation deleting the guard below left the whole test green. + Assertions.assertNotEquals(ShadowingAuthorizationPluginFactory.class.getName(), + installed.getClass().getName(), + "a jar dropped into the plugin directory displaced a source shipped with the FE; an" + + " allow-everything plugin under a real source's name is the whole point of" + + " refusing this"); + // Refusing it has to release it too: a plugin kept in the directory runtime keeps its classloader, + // and every jar it opened, for the lifetime of the FE. + DirectoryPluginRuntimeManager directoryRuntime = + Deencapsulation.getField(manager, "pluginDirectoryRuntime"); + Assertions.assertFalse( + directoryRuntime.get(ShadowingAuthorizationPluginFactory.SHADOWED_NAME).isPresent(), + "the refused plugin is still held by the directory runtime, and so is its classloader"); + } + + @Test + public void theDecisionVocabularyIsSharedWithADirectoryPlugin() throws IOException { + ApiVersionGate gate = ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + // The jar carries its OWN copy of a vocabulary class. Without org.apache.doris.authorization. being + // parent-first, the plugin's classloader would prefer that copy, and a resource kind it returned + // would not be the resource kind the engine switches on - a ClassCastException with no plugin bug + // behind it. With an empty extras list this assertion would hold trivially, which is why the copy + // is deliberately planted. + Path root = pluginRoot("authorization-shared-types-root", "shared-types-authz", + VersionProbeAuthorizationPluginFactory.class, AuthorizationPluginFactory.class, + "Doris-Authorization-Plugin-Api-Version", gate.getExpectedVersion(), + Collections.singletonList(ResourceKind.class)); + String originalDir = Config.authorization_plugins_dir; + String originalType = Config.access_controller_type; + AccessControllerManager manager; + try { + Config.authorization_plugins_dir = root.toString(); + Config.access_controller_type = VersionProbeAuthorizationPluginFactory.NAME; + manager = new AccessControllerManager(new Auth()); + } finally { + Config.authorization_plugins_dir = originalDir; + Config.access_controller_type = originalType; + } + + ClassLoader pluginClassLoader = manager + .getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME) + .getClass().getClassLoader(); + Assertions.assertNotSame(getClass().getClassLoader(), pluginClassLoader, + "the plugin was not loaded through its own classloader, so nothing about class sharing is" + + " being tested here"); + try { + Assertions.assertSame(ResourceKind.class, + pluginClassLoader.loadClass(ResourceKind.class.getName()), + "the plugin resolved its own copy of the decision vocabulary"); + } catch (ClassNotFoundException e) { + Assertions.fail("the plugin classloader cannot see the decision vocabulary at all", e); + } + } + + /** + * Builds an {@link AccessControllerManager} whose plugin directory holds one probe plugin declaring + * {@code declaredApiVersion}, with {@code access_controller_type} naming it. Construction is where the + * whole channel runs: directory sweep, version gate, factory registration, and installing the named + * source. + */ + private AccessControllerManager managerLoadingProbeFrom(String declaredApiVersion) throws IOException { + Path root = pluginRoot("authorization-root-" + declaredApiVersion, "version-probe-authz", + VersionProbeAuthorizationPluginFactory.class, AuthorizationPluginFactory.class, + "Doris-Authorization-Plugin-Api-Version", declaredApiVersion); + String originalDir = Config.authorization_plugins_dir; + String originalType = Config.access_controller_type; + try { + Config.authorization_plugins_dir = root.toString(); + Config.access_controller_type = VersionProbeAuthorizationPluginFactory.NAME; + return new AccessControllerManager(new Auth()); + } finally { + Config.authorization_plugins_dir = originalDir; + Config.access_controller_type = originalType; + } + } + + private String refusalOf(String declaredApiVersion) throws IOException { + try { + managerLoadingProbeFrom(declaredApiVersion); + } catch (RuntimeException e) { + return e.getMessage(); + } + return Assertions.fail("a plugin declaring " + declaredApiVersion + + " must not become the source governing this instance"); + } + private static Set providerNames(FileSystemPluginManager manager) { Set names = new HashSet<>(); manager.getProviders().forEach(provider -> names.add(provider.name())); @@ -195,6 +359,17 @@ private Path filesystemPluginRoot(String declaredApiVersion) throws IOException */ private Path pluginRoot(String rootName, String pluginDirName, Class providerClass, Class spiInterface, String manifestAttribute, String declaredApiVersion) throws IOException { + return pluginRoot(rootName, pluginDirName, providerClass, spiInterface, manifestAttribute, + declaredApiVersion, Collections.emptyList()); + } + + /** + * @param alsoBundled classes whose bytes are copied into the jar as well, to stand for a plugin that + * ships its own copy of something the FE also has + */ + private Path pluginRoot(String rootName, String pluginDirName, Class providerClass, + Class spiInterface, String manifestAttribute, String declaredApiVersion, + List> alsoBundled) throws IOException { Path root = tempDir.resolve(rootName); Path jarPath = root.resolve(pluginDirName).resolve(pluginDirName + ".jar"); Files.createDirectories(jarPath.getParent()); @@ -204,22 +379,29 @@ private Path pluginRoot(String rootName, String pluginDirName, Class provider if (declaredApiVersion != null) { manifest.getMainAttributes().putValue(manifestAttribute, declaredApiVersion); } - String classEntry = providerClass.getName().replace('.', '/') + ".class"; try (JarOutputStream jar = new JarOutputStream(Files.newOutputStream(jarPath), manifest)) { - jar.putNextEntry(new JarEntry(classEntry)); - try (InputStream classBytes = providerClass.getClassLoader().getResourceAsStream(classEntry)) { - Assertions.assertNotNull(classBytes, "class bytes not found: " + classEntry); - byte[] buffer = new byte[8192]; - int read; - while ((read = classBytes.read(buffer)) != -1) { - jar.write(buffer, 0, read); - } + writeClassEntry(jar, providerClass); + for (Class bundled : alsoBundled) { + writeClassEntry(jar, bundled); } - jar.closeEntry(); jar.putNextEntry(new JarEntry("META-INF/services/" + spiInterface.getName())); jar.write((providerClass.getName() + "\n").getBytes(StandardCharsets.UTF_8)); jar.closeEntry(); } return root; } + + private static void writeClassEntry(JarOutputStream jar, Class clazz) throws IOException { + String classEntry = clazz.getName().replace('.', '/') + ".class"; + jar.putNextEntry(new JarEntry(classEntry)); + try (InputStream classBytes = clazz.getClassLoader().getResourceAsStream(classEntry)) { + Assertions.assertNotNull(classBytes, "class bytes not found: " + classEntry); + byte[] buffer = new byte[8192]; + int read; + while ((read = classBytes.read(buffer)) != -1) { + jar.write(buffer, 0, read); + } + } + jar.closeEntry(); + } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java new file mode 100644 index 00000000000000..f1148c58176629 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java @@ -0,0 +1,58 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.pluginapiversion.testplugins; + +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; + +import java.util.Map; + +/** + * An authorization source that claims a name the FE already ships ({@code ranger-doris}), so that a test can + * check what happens when somebody drops such a jar into the plugin directory. + * + *

Not hypothetical: an allow-everything plugin under the name of a real source would be the cheapest way + * to turn a filesystem write into read access to everything that source governs. + */ +public class ShadowingAuthorizationPluginFactory + implements AuthorizationPluginFactory, AuthorizationPlugin { + + /** Deliberately the name of a source registered on the FE's own class path. */ + public static final String SHADOWED_NAME = "ranger-doris"; + + @Override + public String name() { + return SHADOWED_NAME; + } + + @Override + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return this; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) { + // Allows everything, which is exactly what must never become reachable this way. + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/VersionProbeAuthorizationPluginFactory.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/VersionProbeAuthorizationPluginFactory.java new file mode 100644 index 00000000000000..29bc034ef7c596 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/VersionProbeAuthorizationPluginFactory.java @@ -0,0 +1,63 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.pluginapiversion.testplugins; + +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; + +import java.util.Map; + +/** + * An authorization source whose class bytes are copied into a temporary plugin jar, so that a test can load + * it exactly the way FE loads a third-party one. + * + *

It lives in {@code org.apache.doris.pluginapiversion.testplugins} on purpose. The AUTHORIZATION family + * declares {@code org.apache.doris.authorization.} parent-first, so a probe living there would be loaded + * from the FE's own classpath rather than from the plugin jar - and the loader reads the declared API + * version from the jar that defines the factory class. Such a probe would look undeclared no matter + * what its jar said, and every test here would pass for the wrong reason. + * + *

Factory and plugin are one class so that a single class entry makes a complete plugin jar. + */ +public class VersionProbeAuthorizationPluginFactory + implements AuthorizationPluginFactory, AuthorizationPlugin { + + public static final String NAME = "version_probe_authz"; + + @Override + public String name() { + return NAME; + } + + @Override + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return this; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext context) throws AccessDeniedException { + throw AccessDeniedException.of(subject, resource, requirement, NAME); + } +} diff --git a/fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/FileSystemPluginSurfaceTest.java b/fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/FileSystemPluginSurfaceTest.java index fb67c9790856c7..9c0507cce7d9cb 100644 --- a/fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/FileSystemPluginSurfaceTest.java +++ b/fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/FileSystemPluginSurfaceTest.java @@ -47,7 +47,7 @@ * {@code fe/fe-filesystem/pom.xml} in the SAME commit. * *

{@code Plugin} / {@code PluginFactory} / {@code PluginContext} from fe-extension-spi are frozen here - * too, and identically in the other three families' baselines. They are loaded parent-first for every family + * too, and identically in the other four families' baselines. They are loaded parent-first for every family * (see {@code ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES}), so a change to them breaks all four * plugin kinds at once — and turns all four baselines red at once, each asking for its own bump. * diff --git a/fe/fe-filesystem/pom.xml b/fe/fe-filesystem/pom.xml index 88fe040e7cb880..a8368afa785571 100644 --- a/fe/fe-filesystem/pom.xml +++ b/fe/fe-filesystem/pom.xml @@ -50,7 +50,7 @@ under the License. Bump the MAJOR (and zero the minor) in the SAME commit as ANY change to the filesystem SPI surface - additions included. Note fe-filesystem-api is also linked by connector plugins, so a change there means bumping connector.plugin.api.version too; fe-extension-spi means - bumping all four families. See + bumping all five families. See plan-doc/designs/2026-07-29-plugin-api-version-check-design.md. --> 1.0 From 9285ed3494ec84beec211048adea87a9ca566826 Mon Sep 17 00:00:00 2001 From: morningman Date: Thu, 13 Aug 2026 00:53:38 +0800 Subject: [PATCH 10/22] [test](authorization) prove a plugin installed from a directory really decides An authorization source can be shipped as a jar and dropped into plugins/authorization/ since the previous change, and the tests written for it reach as far as "the source named in fe.conf is the one the manager reports". That is one step short of what the channel exists for. Nothing put a statement through the engine, so nothing would have noticed a plugin that was installed and then never asked, or asked and then overruled. There is a worked example now, and a test that installs it the way a third party installs one - a jar written into the plugin directory, discovered at startup, admitted on the API version it declares, loaded through its own classloader, built by its factory - and then checks three things that are only observable from SQL: - an account the built-in model granted nothing reads a table, because the example allows it; - an account the built-in model granted SELECT on that same table cannot, because the example does not. This is the least intuitive consequence of one source answering for a resource, and the easiest for an implementation to quietly lose; - the row filter the example returns is planned over the table for the account it applies to, and not for the account it does not. The example grants by Doris role rather than allowing everything, which is what the plan had called for. An allow-everything plugin cannot tell "the plugin said yes" apart from "something else said yes first" - both look identical from a passing test. Only a refusal that really refuses pins the channel down, and the second check above is that refusal. Three things this turned up, all written into the example's own comments because a plugin author meets them immediately: - a source installed for the whole instance has to answer for administration itself. grantedByGlobalScopeAuthority answers false when the source asking IS that authority, which an instance-wide source always is, so a source with no admin rule of its own locks out every account - including the one that would put the configuration back. - the properties an instance-wide source is configured with cannot be fed from a unit test: their path is built from the DORIS_HOME environment variable, which the harness never exports. The example runs on its defaults and says so. The per-catalog property channel is a catalog property and is reachable. - the example cannot live under org.apache.doris.authorization, which is loaded parent-first. Placed there it would come off the FE's own class path, with no jar and so no declared version, and every assertion would hold for the wrong reason. A plugin written outside this repository never meets this. The jar-writing helper moves out of the version-gate test into utframe.PluginJarWriter. Both tests depend on the same directory layout, and a layout contract kept in two copies is one that drifts. Deliberately not here: the example is not packaged into a release. It would put a jar on every FE that turns into an allow-everything authorization source one config line later, and no other plugin family ships an example module either. That also rules out a regression-level test, which would need the jar to be lying in a real cluster's plugin directory. Verified: 87 test classes, 480 tests, no failures, the four skips pre-existing; the behaviour baseline unchanged; the legacy channel's own regression still green. Mutations, four of four: never refusing turns the built-in-grant test red; returning no row filter turns the row-filter test red; filtering everyone turns the same test red through its negative control; and a jar declaring no API version stops the FE from starting at all, refused with both versions and the missing attribute named. Co-Authored-By: Claude Opus 5 (1M context) --- .../fe-authorization-spi/README.md | 16 ++ .../AuthorizationPluginFromDirectoryTest.java | 225 ++++++++++++++++++ .../ExampleAuthorizationPlugin.java | 157 ++++++++++++ .../ExampleAuthorizationPluginFactory.java | 60 +++++ .../PluginApiVersionWiringTest.java | 48 +--- .../apache/doris/utframe/PluginJarWriter.java | 111 +++++++++ 6 files changed, 574 insertions(+), 43 deletions(-) create mode 100644 fe/fe-core/src/test/java/org/apache/doris/authorizationexample/AuthorizationPluginFromDirectoryTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPlugin.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPluginFactory.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/utframe/PluginJarWriter.java diff --git a/fe/fe-authorization/fe-authorization-spi/README.md b/fe/fe-authorization/fe-authorization-spi/README.md index 385eef3d183934..07c4c985137319 100644 --- a/fe/fe-authorization/fe-authorization-spi/README.md +++ b/fe/fe-authorization/fe-authorization-spi/README.md @@ -98,6 +98,22 @@ public final class CustomAuthorizationPluginFactory implements AuthorizationPlug com.example.authz.CustomAuthorizationPluginFactory ``` +## Worked example + +The snippets above are elided. A complete source — one that really governs a running FE — lives in the test +tree, together with the test that installs it from a plugin directory and puts SQL through it: + +```text +fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ +├── ExampleAuthorizationPluginFactory.java # the four lines that make a jar a plugin +├── ExampleAuthorizationPlugin.java # privileges by Doris role, plus one row filter +└── AuthorizationPluginFromDirectoryTest.java # installs it and checks what SQL then does +``` + +It is the shortest thing that answers the questions a first plugin runs into: how to decide when the +requirement is not one you recognise, why an instance-wide source needs an administration rule of its own, +and what "returning no row filter" does and does not mean. + ## Lifecycle and cost A plugin is created once and kept. Unlike an authentication attempt, an authorization decision happens many diff --git a/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/AuthorizationPluginFromDirectoryTest.java b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/AuthorizationPluginFromDirectoryTest.java new file mode 100644 index 00000000000000..4cca3b63d0512d --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/AuthorizationPluginFromDirectoryTest.java @@ -0,0 +1,225 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorizationexample; + +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; +import org.apache.doris.catalog.Env; +import org.apache.doris.common.Config; +import org.apache.doris.common.FeConstants; +import org.apache.doris.datasource.InternalCatalog; +import org.apache.doris.extension.loader.ApiVersionGate; +import org.apache.doris.extension.loader.PluginRegistry; +import org.apache.doris.nereids.exceptions.AnalysisException; +import org.apache.doris.nereids.trees.expressions.EqualTo; +import org.apache.doris.nereids.trees.expressions.Expression; +import org.apache.doris.nereids.trees.expressions.literal.Literal; +import org.apache.doris.nereids.trees.plans.Plan; +import org.apache.doris.nereids.trees.plans.logical.LogicalFilter; +import org.apache.doris.nereids.util.PlanChecker; +import org.apache.doris.utframe.PluginJarWriter; +import org.apache.doris.utframe.TestWithFeService; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** + * An authorization source shipped as a jar, installed the way a third party installs one, really decides + * what a running FE allows. + * + *

Everything before this test stops one step short of that. The loader tests prove a plugin directory is + * swept and a version gate applied; the wiring test proves the source named in {@code fe.conf} is the one + * the manager reports. None of them puts a statement through the engine, so none of them would notice if a + * plugin were installed and then never asked, or asked and then overruled. + * + *

So the whole chain runs here: a jar written into {@code authorization_plugins_dir}, discovered at + * startup, admitted on the plugin API version it declares, loaded through its own classloader, built by its + * factory - and then three things are checked that are only observable from SQL: + * + *

    + *
  • an account the built-in model granted nothing can read a table, because the plugin allows it;
    • + *
  • an account the built-in model granted {@code SELECT} on that same table cannot, because the plugin + * does not - the source governing a resource is the whole answer for it;
    • + *
  • the row filter the plugin returns is planned over the table, for the account it applies to and not + * for the one it does not.
    • + *
+ * + *

The second of those records today's behaviour deliberately: {@code GRANT} still succeeds under an + * external source and is then ignored. Making it an error is later work, and this test is where that change + * will announce itself. + * + *

What is not covered here, honestly: the properties an instance-wide source is configured with. Their + * file path is built from the {@code DORIS_HOME} environment variable, which a unit test does not have, so + * {@link ExampleAuthorizationPlugin} runs on its defaults. The per-catalog property channel is a catalog + * property and is exercised by the Ranger tests. + */ +public class AuthorizationPluginFromDirectoryTest extends TestWithFeService { + + private static final String DB = "example_db"; + private static final String TBL = "sales"; + private static final String QUALIFIED_TBL = InternalCatalog.INTERNAL_CATALOG_NAME + "." + DB + "." + TBL; + + /** Holds the role the example source grants reading to, and nothing the built-in model ever granted. */ + private static final String READER = "example_reader_user"; + /** Holds a real built-in SELECT grant on the table, and no role the example source has heard of. */ + private static final String GRANTED_BY_DORIS = "granted_by_doris_user"; + + private Path pluginRoot; + private String originalPluginsDir; + private String originalControllerType; + + /** + * Runs before the FE exists, which is the only moment this is configurable: the manager reading + * {@code access_controller_type} is built once, while the FE starts. + */ + @Override + protected void beforeCreatingConnectContext() throws Exception { + ApiVersionGate gate = ApiVersionGate.forFamily("authorization", AuthorizationPluginFactory.class); + pluginRoot = Files.createTempDirectory("authorization-plugin-e2e"); + PluginJarWriter.writePluginRoot(pluginRoot, ExampleAuthorizationPluginFactory.NAME, + ExampleAuthorizationPluginFactory.class, AuthorizationPluginFactory.class, + gate.getManifestAttribute(), gate.getExpectedVersion(), + Collections.singletonList(ExampleAuthorizationPlugin.class)); + + originalPluginsDir = Config.authorization_plugins_dir; + originalControllerType = Config.access_controller_type; + Config.authorization_plugins_dir = pluginRoot.toString(); + Config.access_controller_type = ExampleAuthorizationPluginFactory.NAME; + } + + /** + * Everything here runs as {@code root}, which the example source lets through because root holds the + * Doris role it treats as its administrator. That is not a detail of the test: a source installed for + * the whole instance answers for administration too, so one without an admin rule of its own would make + * the FE unadministrable from its first statement. + */ + @Override + protected void runBeforeAll() throws Exception { + FeConstants.runningUnitTest = true; + createDatabase(DB); + useDatabase(DB); + createTable("create table " + TBL + " (id int, region varchar(8))" + + " distributed by hash(id) buckets 1 properties(\"replication_num\" = \"1\");"); + + createRole(ExampleAuthorizationPlugin.DEFAULT_READER_ROLE); + addUser(READER, true); + grantRole("GRANT '" + ExampleAuthorizationPlugin.DEFAULT_READER_ROLE + "' TO '" + READER + "'@'%'"); + + addUser(GRANTED_BY_DORIS, true); + grantPriv("GRANT SELECT_PRIV ON " + QUALIFIED_TBL + " TO '" + GRANTED_BY_DORIS + "'@'%'"); + } + + @Override + protected void runBeforeEach() throws Exception { + // Each test names the account it is about; start from root so a leftover identity cannot make one + // of them pass for the wrong reason. + useUser("root"); + } + + @Override + protected void runAfterAll() throws Exception { + Config.authorization_plugins_dir = originalPluginsDir; + Config.access_controller_type = originalControllerType; + // loadPlugins writes rows into a process-wide registry backing information_schema.extensions; + // leaving them there would make other classes' assertions depend on execution order. + PluginRegistry.getInstance().clearForTest(); + } + + @Test + public void theSourceGoverningThisInstanceIsTheOneFromThePluginDirectory() { + AuthorizationPlugin governing = Env.getCurrentEnv().getAccessManager() + .getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME); + + Assertions.assertEquals(ExampleAuthorizationPluginFactory.NAME, governing.name()); + // Without this, every assertion below would hold just as well with the example loaded off the test + // class path - and then nothing here would be about installing a plugin at all. + Assertions.assertNotSame(getClass().getClassLoader(), governing.getClass().getClassLoader(), + "the example was loaded through the FE's own classloader, so it did not come from the jar"); + } + + @Test + public void anAccountTheBuiltInModelGrantedNothingReadsWhatThePluginAllows() throws Exception { + useUser(READER); + + // No GRANT was ever issued to this account. If anything but the plugin had a say, this would fail. + Plan plan = rewrite("select id, region from " + QUALIFIED_TBL); + + Assertions.assertNotNull(plan); + } + + @Test + public void builtInGrantsDoNotCountOnceThePluginGoverns() throws Exception { + useUser(GRANTED_BY_DORIS); + + AnalysisException refused = Assertions.assertThrows(AnalysisException.class, + () -> rewrite("select id, region from " + QUALIFIED_TBL), + "a built-in GRANT was honoured while an external source governs the table; that source's" + + " answer is supposed to be the whole answer"); + + // Naming the source is the difference between an operator finding the policy that refused and + // hunting through a privilege model that has nothing to do with the decision. + Assertions.assertTrue(refused.getMessage().contains(ExampleAuthorizationPluginFactory.NAME), + "the refusal does not say which source refused: " + refused.getMessage()); + } + + @Test + public void theRowFilterTheSourceReturnsIsPlannedOverTheTable() throws Exception { + useUser(READER); + List readerFilters = filterConjunctsOf(rewrite("select id, region from " + QUALIFIED_TBL)); + + Assertions.assertTrue(readerFilters.stream().anyMatch(this::isTheExampleRowFilter), + "the row filter the source returned never reached the plan: " + readerFilters); + + // The same query by an account the source imposes no filter on. Without this the assertion above + // would also pass if every plan carried the predicate regardless of who asked. + useUser("root"); + List adminFilters = filterConjunctsOf(rewrite("select id, region from " + QUALIFIED_TBL)); + + Assertions.assertTrue(adminFilters.stream().noneMatch(this::isTheExampleRowFilter), + "an account the source returns no filter for was filtered anyway: " + adminFilters); + } + + private Plan rewrite(String sql) { + return PlanChecker.from(connectContext).parse(sql).analyze().rewrite().getPlan(); + } + + private List filterConjunctsOf(Plan plan) { + List conjuncts = new ArrayList<>(); + for (Object node : plan.collectToList(LogicalFilter.class::isInstance)) { + conjuncts.addAll(((LogicalFilter) node).getConjuncts()); + } + return conjuncts; + } + + /** Matches {@code region = 'EU'}, whatever the planner has renamed or re-typed around it. */ + private boolean isTheExampleRowFilter(Expression conjunct) { + if (!(conjunct instanceof EqualTo)) { + return false; + } + EqualTo equalTo = (EqualTo) conjunct; + return equalTo.left().toSql().contains("region") + && equalTo.right() instanceof Literal + && "EU".equals(((Literal) equalTo.right()).getStringValue()); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPlugin.java b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPlugin.java new file mode 100644 index 00000000000000..a52d4eb5dd9de8 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPlugin.java @@ -0,0 +1,157 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorizationexample; + +import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; +import org.apache.doris.authorization.AccessDeniedException; +import org.apache.doris.authorization.AccessRequirement; +import org.apache.doris.authorization.AuthorizedResource; +import org.apache.doris.authorization.AuthorizedSubject; +import org.apache.doris.authorization.RowFilterSpec; +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; + +import java.util.Collections; +import java.util.EnumSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * A worked example of an authorization source, complete enough to govern a running FE. + * + *

It grants by Doris role and knows two of them: whoever holds the admin role may do anything, whoever + * holds the reader role may read and nothing else, and then only the rows matching one predicate. Everyone + * else is refused - including an account holding a built-in {@code GRANT}, because the source + * governing a resource is the only thing consulted about it and this source has never heard of that grant. + * + *

Read it together with {@code fe-authorization-spi/README.md}. Three things here are worth copying + * beyond the shape: + * + *

    + *
  • Decide by which actions the subject holds, not by which question was asked. The engine asks + * about a whole requirement, and not every requirement is one of the named ones in + * {@code AccessRequirements}: granting a privilege, for instance, requires holding that privilege + * and the right to grant it, and that requirement is composed at run time. A source that + * matched on the requirements it recognises would refuse those and look, from the outside, like a + * source with a mysteriously incomplete policy.
    • + *
  • A source installed for the whole instance has to answer for administration itself. + * {@link AuthorizationContext#grantedByGlobalScopeAuthority} exists so a catalog-bound source can let + * an instance administrator through, but it answers false when the source asking is that + * authority - which an instance-wide source always is. Such a source with no admin rule of its own + * locks out every account, including the one that would fix the configuration.
    • + *
  • Returning no row filter is not a decision about access. Whether the table may be read at all + * was settled by {@link #checkPrivilege}; the filters only narrow what a read that is already allowed + * returns.
    • + *
+ * + *

This source answers uniformly for every kind of resource, so it has no branch for a kind it does not + * recognise. One that understands only tables must refuse the other kinds rather than guess: the + * kinds are closed and known when the plugin is compiled, so anything else means the plugin was built + * against a different Doris. + * + *

Lastly, a detail that exists only because this example lives inside the Doris source tree: it is + * deliberately not in a package under {@code org.apache.doris.authorization}, which the FE loads from + * itself rather than from the plugin jar. A plugin placed there would be loaded off the FE's own class path, + * with no jar and so no declared API version. Plugins written outside this repository have their own package + * and never meet this. + */ +public class ExampleAuthorizationPlugin implements AuthorizationPlugin { + + /** Doris role whose holders may do anything. Defaults to the role the FE's own {@code root} holds. */ + public static final String ADMIN_ROLE_PROPERTY = "example.admin_role"; + public static final String DEFAULT_ADMIN_ROLE = "operator"; + + /** Doris role whose holders may read, and only read. */ + public static final String READER_ROLE_PROPERTY = "example.reader_role"; + public static final String DEFAULT_READER_ROLE = "example_reader"; + + /** SQL predicate in Doris dialect, restricting which rows a reader sees. */ + public static final String READER_ROW_FILTER_PROPERTY = "example.reader_row_filter"; + public static final String DEFAULT_READER_ROW_FILTER = "region = 'EU'"; + + /** Names the policy in diagnostics; a real source would use whatever its own policies are called. */ + static final String ROW_FILTER_IDENT = "example-reader-rows"; + + private static final Set EVERY_ACTION = + Collections.unmodifiableSet(EnumSet.allOf(AccessAction.class)); + private static final Set READ_ONLY = + Collections.unmodifiableSet(EnumSet.of(AccessAction.SELECT)); + + private final AuthorizationContext context; + private final String adminRole; + private final String readerRole; + private final String readerRowFilter; + + ExampleAuthorizationPlugin(Map properties, AuthorizationContext context) { + this.context = context; + this.adminRole = properties.getOrDefault(ADMIN_ROLE_PROPERTY, DEFAULT_ADMIN_ROLE); + this.readerRole = properties.getOrDefault(READER_ROLE_PROPERTY, DEFAULT_READER_ROLE); + this.readerRowFilter = + properties.getOrDefault(READER_ROW_FILTER_PROPERTY, DEFAULT_READER_ROW_FILTER); + } + + @Override + public String name() { + return ExampleAuthorizationPluginFactory.NAME; + } + + @Override + public void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource, + AccessRequirement requirement, AccessContext accessContext) throws AccessDeniedException { + if (!requirement.isSatisfiedBy(actionsHeldBy(subject))) { + // Refusing is throwing. Naming this source is what later lets the operator reading the error + // know which of the configured sources said no. + throw AccessDeniedException.of(subject, resource, requirement, name()); + } + } + + @Override + public List getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table, + AccessContext accessContext) { + if (!isReader(subject)) { + return Collections.emptyList(); + } + // One policy for every table this source governs, which keeps the example about the contract. A + // real source looks the policy up by table, and returns several when several apply - RESTRICTIVE + // ones are ANDed together, PERMISSIVE ones ORed. + return Collections.singletonList(RowFilterSpec.restrictive(ROW_FILTER_IDENT, readerRowFilter)); + } + + /** + * What this source grants the subject, as a set of actions. Roles come from the engine because only it + * can resolve them, and asking for them lazily - rather than carrying them on every subject - matters + * because listing what an account may see asks thousands of questions per statement. + */ + private Set actionsHeldBy(AuthorizedSubject subject) { + Set roles = context.rolesOf(subject); + if (roles.contains(adminRole)) { + return EVERY_ACTION; + } + if (roles.contains(readerRole)) { + return READ_ONLY; + } + return Collections.emptySet(); + } + + private boolean isReader(AuthorizedSubject subject) { + Set roles = context.rolesOf(subject); + return !roles.contains(adminRole) && roles.contains(readerRole); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPluginFactory.java b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPluginFactory.java new file mode 100644 index 00000000000000..e5226305921b45 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/authorizationexample/ExampleAuthorizationPluginFactory.java @@ -0,0 +1,60 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.authorizationexample; + +import org.apache.doris.authorization.spi.AuthorizationContext; +import org.apache.doris.authorization.spi.AuthorizationPlugin; +import org.apache.doris.authorization.spi.AuthorizationPluginFactory; + +import java.util.Map; + +/** + * Publishes {@link ExampleAuthorizationPlugin} under the name configuration selects it by. + * + *

A jar becomes a plugin by naming a class like this one in + * {@code META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory}. That file is the + * whole of the discovery mechanism; there is nothing to register and nothing to declare in {@code fe.conf} + * beyond the name below. + * + *

The factory exists separately from the plugin because the plugin needs two things that only exist at + * the moment the engine builds it: whatever configures this source, and the {@link AuthorizationContext} it + * puts questions to the engine through. + */ +public class ExampleAuthorizationPluginFactory implements AuthorizationPluginFactory { + + /** + * What to write in {@code fe.conf} as {@code access_controller_type} to have this source govern the + * whole instance, or in a catalog's {@code "access_controller.class"} to have it govern that catalog. + */ + public static final String NAME = "example-authz"; + + @Override + public String name() { + return NAME; + } + + @Override + public String description() { + return "Worked example of an authorization source: privileges by Doris role, plus a row filter"; + } + + @Override + public AuthorizationPlugin create(Map properties, AuthorizationContext context) { + return new ExampleAuthorizationPlugin(properties, context); + } +} diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java index 9a51df27379577..fbcbd0b35df914 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java @@ -37,6 +37,7 @@ import org.apache.doris.pluginapiversion.testplugins.VersionProbeAuthorizationPluginFactory; import org.apache.doris.pluginapiversion.testplugins.VersionProbeConnectorProvider; import org.apache.doris.pluginapiversion.testplugins.VersionProbeFileSystemProvider; +import org.apache.doris.utframe.PluginJarWriter; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.Assertions; @@ -45,19 +46,12 @@ import org.junit.jupiter.api.io.TempDir; import java.io.IOException; -import java.io.InputStream; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; import java.nio.file.Path; import java.util.Collections; import java.util.HashSet; import java.util.List; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; -import java.util.jar.Attributes; -import java.util.jar.JarEntry; -import java.util.jar.JarOutputStream; -import java.util.jar.Manifest; /** * Each fe-core plugin family really enforces its own plugin API version, on real plugin jars. @@ -353,9 +347,8 @@ private Path filesystemPluginRoot(String declaredApiVersion) throws IOException } /** - * Writes {@code //.jar} the way the assembly really lays a plugin - * out: the provider's class bytes, its ServiceLoader registration, and a MANIFEST declaring the plugin - * API version. A null {@code declaredApiVersion} omits the attribute entirely. + * Writes one plugin under {@code tempDir/rootName} and returns that root, the way the assembly really + * lays a plugin out. A null {@code declaredApiVersion} omits the attribute entirely. */ private Path pluginRoot(String rootName, String pluginDirName, Class providerClass, Class spiInterface, String manifestAttribute, String declaredApiVersion) throws IOException { @@ -370,38 +363,7 @@ private Path pluginRoot(String rootName, String pluginDirName, Class provider private Path pluginRoot(String rootName, String pluginDirName, Class providerClass, Class spiInterface, String manifestAttribute, String declaredApiVersion, List> alsoBundled) throws IOException { - Path root = tempDir.resolve(rootName); - Path jarPath = root.resolve(pluginDirName).resolve(pluginDirName + ".jar"); - Files.createDirectories(jarPath.getParent()); - - Manifest manifest = new Manifest(); - manifest.getMainAttributes().put(Attributes.Name.MANIFEST_VERSION, "1.0"); - if (declaredApiVersion != null) { - manifest.getMainAttributes().putValue(manifestAttribute, declaredApiVersion); - } - try (JarOutputStream jar = new JarOutputStream(Files.newOutputStream(jarPath), manifest)) { - writeClassEntry(jar, providerClass); - for (Class bundled : alsoBundled) { - writeClassEntry(jar, bundled); - } - jar.putNextEntry(new JarEntry("META-INF/services/" + spiInterface.getName())); - jar.write((providerClass.getName() + "\n").getBytes(StandardCharsets.UTF_8)); - jar.closeEntry(); - } - return root; - } - - private static void writeClassEntry(JarOutputStream jar, Class clazz) throws IOException { - String classEntry = clazz.getName().replace('.', '/') + ".class"; - jar.putNextEntry(new JarEntry(classEntry)); - try (InputStream classBytes = clazz.getClassLoader().getResourceAsStream(classEntry)) { - Assertions.assertNotNull(classBytes, "class bytes not found: " + classEntry); - byte[] buffer = new byte[8192]; - int read; - while ((read = classBytes.read(buffer)) != -1) { - jar.write(buffer, 0, read); - } - } - jar.closeEntry(); + return PluginJarWriter.writePluginRoot(tempDir.resolve(rootName), pluginDirName, providerClass, + spiInterface, manifestAttribute, declaredApiVersion, alsoBundled); } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/utframe/PluginJarWriter.java b/fe/fe-core/src/test/java/org/apache/doris/utframe/PluginJarWriter.java new file mode 100644 index 00000000000000..f287a73420b79d --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/utframe/PluginJarWriter.java @@ -0,0 +1,111 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.utframe; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.jar.Attributes; +import java.util.jar.JarEntry; +import java.util.jar.JarOutputStream; +import java.util.jar.Manifest; + +/** + * Writes a plugin jar the way a release really lays one out, for tests that need the FE to load a plugin + * the way it loads a third-party one instead of finding it on the test class path. + * + *

The difference matters more than it looks. A class on the test class path is loaded by the test's own + * classloader from a directory of {@code .class} files, so it has no jar and therefore no MANIFEST: the + * plugin API version it declares cannot be read, and every type it exchanges with the engine is already the + * engine's own. A plugin loaded from here has neither of those properties, which is the whole of what the + * loader has to get right. + * + *

Layout written, one plugin per subdirectory of the root: + * + * 

+ * <root>/<pluginDirName>/<pluginDirName>.jar
+ *      META-INF/MANIFEST.MF                  # the declared plugin API version
+ *      META-INF/services/<spi interface>     # naming the factory class
+ *      <factory class>.class
+ *      <each of alsoBundled>.class
+ *
+ */ +public final class PluginJarWriter { + + private PluginJarWriter() { + } + + /** + * Writes one plugin into {@code root} and returns {@code root}, which is what a plugin directory + * configuration names. + * + * @param pluginDirName the subdirectory, and the jar's base name. Deliberately not required to equal the + * plugin's name: the loader takes the name from the factory, and a test passing a different one + * here is what keeps that true. + * @param manifestAttribute the family's plugin API version attribute, e.g. + * {@code Doris-Authorization-Plugin-Api-Version} + * @param declaredApiVersion the version to declare, or null to omit the attribute entirely - which is + * what a plugin written with no awareness of the contract looks like + * @param alsoBundled further classes whose bytes go into the jar: the rest of the plugin, or a copy of + * something the FE also has, to test which copy wins + */ + public static Path writePluginRoot(Path root, String pluginDirName, Class factoryClass, + Class spiInterface, String manifestAttribute, String declaredApiVersion, + List> alsoBundled) throws IOException { + Path jarPath = root.resolve(pluginDirName).resolve(pluginDirName + ".jar"); + Files.createDirectories(jarPath.getParent()); + + Manifest manifest = new Manifest(); + manifest.getMainAttributes().put(Attributes.Name.MANIFEST_VERSION, "1.0"); + if (declaredApiVersion != null) { + manifest.getMainAttributes().putValue(manifestAttribute, declaredApiVersion); + } + try (JarOutputStream jar = new JarOutputStream(Files.newOutputStream(jarPath), manifest)) { + writeClassEntry(jar, factoryClass); + for (Class bundled : alsoBundled) { + writeClassEntry(jar, bundled); + } + jar.putNextEntry(new JarEntry("META-INF/services/" + spiInterface.getName())); + jar.write((factoryClass.getName() + "\n").getBytes(StandardCharsets.UTF_8)); + jar.closeEntry(); + } + return root; + } + + private static void writeClassEntry(JarOutputStream jar, Class clazz) throws IOException { + String classEntry = clazz.getName().replace('.', '/') + ".class"; + jar.putNextEntry(new JarEntry(classEntry)); + try (InputStream classBytes = clazz.getClassLoader().getResourceAsStream(classEntry)) { + if (classBytes == null) { + // Silently writing an empty entry would produce a jar whose plugin fails to load for a + // reason that has nothing to do with what the test is checking. + throw new IllegalStateException("class bytes not found for " + clazz.getName() + + "; a nested or synthetic class has to be bundled by name too"); + } + byte[] buffer = new byte[8192]; + int read; + while ((read = classBytes.read(buffer)) != -1) { + jar.write(buffer, 0, read); + } + } + jar.closeEntry(); + } +} From 90d5814c0130e9755332d9c47c8343faef0a6136 Mon Sep 17 00:00:00 2001 From: morningman Date: Thu, 13 Aug 2026 09:09:35 +0800 Subject: [PATCH 11/22] [improvement](authorization) move the Ranger sources out of the kernel and into plugins of their own The Ranger sources answer the authorization plugin contract already, but they still lived inside fe-core, which meant every FE compiled and shipped ranger-plugins-common and its hadoop closure whether or not anyone authorized against Ranger. They move out into their own modules under fe-authorization/fe-authorization-plugins/. Three modules rather than one: a plugin directory admits exactly one factory (DirectoryPluginRuntimeManager refuses a directory whose jars declare more than one), and Ranger publishes two sources - one for a whole instance, one bound to a catalog - so each needs its own directory and its own jar carrying its own service descriptor. What they share goes in a third module that deliberately declares no descriptor, so it can sit in both plugins' lib/ without either directory looking like it publishes two sources. Same shape as fe-filesystem-s3-base. Package names are unchanged, so a catalog naming its source by class name keeps working. The one exception is the doris-service factory, which used to live in fe-core's privilege package; leaving it there would have split that package across two jars. Two references to fe-core had to go, both unrelated to authorizing anything: the shared audit-log flush timer is now built in the plugin (which costs its entry in the FE thread-pool metrics - no plugin loaded from a directory can register there), and the name of the workload group everyone may use is a constant in the plugin (the behaviour baseline catches it drifting from the engine's). The lookup behind row filters and data masks used to ask with the Doris service's spelling of the read access type for both services. That is what tied the shared code to one of them, so each source now names its own; both spell it SELECT, so nothing changes on the wire. Left alone, and now recorded: the Hive privilege checks lower case the access type while these lookups do not. fe-core keeps a test-scope dependency on the doris-service plugin, as it already does for the connector and filesystem plugin modules. The access-control behaviour baseline records what a Ranger-governed instance decides for every user and every privilege by running the production controller, and that is the only thing that can show this move changed no decision. Co-Authored-By: Claude Opus 5 (1M context) --- .../pom.xml | 81 +++++++++++++ .../ranger/RangerAccessController.java | 22 ++-- .../pom.xml | 95 +++++++++++++++ .../ranger/doris/DorisAccessType.java | 0 .../ranger/doris/DorisObjectType.java | 0 .../doris/RangerDorisAccessController.java | 18 ++- .../RangerDorisAccessControllerFactory.java | 3 +- .../ranger/doris/RangerDorisPlugin.java | 0 .../ranger/doris/RangerDorisResource.java | 0 ...thorization.spi.AuthorizationPluginFactory | 18 +++ ...angerDorisAccessControllerFactoryTest.java | 3 +- .../authorizer/ranger/doris}/RangerTest.java | 23 +++- .../pom.xml | 108 ++++++++++++++++++ .../ranger/hive/HiveAccessType.java | 0 .../ranger/hive/HiveObjectType.java | 0 .../hive/RangerHiveAccessController.java | 30 ++++- .../RangerHiveAccessControllerFactory.java | 0 .../ranger/hive/RangerHiveAuditHandler.java | 0 .../hive/RangerHiveAuditLogFlusher.java | 0 .../hive/RangerHiveAuthorizerProvider.java | 0 .../ranger/hive/RangerHivePlugin.java | 0 .../ranger/hive/RangerHiveResource.java | 0 ...thorization.spi.AuthorizationPluginFactory | 3 +- .../hive/RangerHiveAccessControllerTest.java | 35 ++++++ .../hive/RangerHiveAuditLogFlusherTest.java | 0 .../fe-authorization-plugins/pom.xml | 46 ++++++++ fe/fe-authorization/pom.xml | 1 + fe/fe-core/pom.xml | 15 ++- .../doris/common/util/PropertyAnalyzer.java | 3 +- .../doris/datasource/ExternalCatalog.java | 2 +- .../privilege/CatalogAccessController.java | 7 +- .../PluginApiVersionWiringTest.java | 6 +- .../ShadowingAuthorizationPluginFactory.java | 13 ++- 33 files changed, 497 insertions(+), 35 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common}/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java (93%) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris}/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris}/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisObjectType.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris}/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java (95%) rename fe/{fe-core/src/main/java/org/apache/doris/mysql/privilege => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris}/RangerDorisAccessControllerFactory.java (95%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris}/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris}/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisResource.java (100%) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory rename fe/{fe-core/src/test/java/org/apache/doris/mysql/privilege => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris}/RangerDorisAccessControllerFactoryTest.java (94%) rename fe/{fe-core/src/test/java/org/apache/doris/mysql/privilege => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris}/RangerTest.java (93%) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveAccessType.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveObjectType.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java (91%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditHandler.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusher.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuthorizerProvider.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveResource.java (100%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory (92%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java (69%) rename fe/{fe-core => fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive}/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusherTest.java (100%) create mode 100644 fe/fe-authorization/fe-authorization-plugins/pom.xml diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml new file mode 100644 index 00000000000000..4727479e9ae26c --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml @@ -0,0 +1,81 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization-plugins + ../pom.xml + + fe-authorization-plugin-ranger-common + jar + Doris FE Authorization Plugin - Ranger Common + What the Ranger authorization sources share: asking a Ranger policy engine and reading its + answer. A library, not a plugin - it publishes no service descriptor and ships no zip of its own. + + + + + + org.apache.doris + fe-authorization-spi + ${project.version} + provided + + + org.apache.doris + fe-authorization-api + ${project.version} + provided + + + + org.apache.ranger + ranger-plugins-common + + + com.google.guava + guava + + + org.apache.commons + commons-lang3 + + + org.apache.logging.log4j + log4j-api + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + + + diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java similarity index 93% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java index 2a008e284098c3..202584f5fc1009 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java @@ -26,7 +26,6 @@ import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.authorization.spi.AuthorizationPlugin; -import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; import com.google.common.collect.Lists; import org.apache.commons.lang3.StringUtils; @@ -171,11 +170,7 @@ public List getRowFilters(AuthorizedSubject subject, AuthorizedRe RangerAccessResourceImpl resource = createResource(table.getCatalog(), table.getDatabase(), table.getTable()); RangerAccessRequestImpl request = createRequest(subject); - // If the access type is not set here, it defaults to ANY1 ACCESS. - // The internal logic of the ranger is to traverse all permission items. - // Since the ranger UI will set the access type to 'SELECT', - // we will keep it consistent with the UI here to avoid performance issues - request.setAccessType(DorisAccessType.SELECT.name()); + request.setAccessType(readAccessTypeName()); request.setResource(resource); if (LOG.isDebugEnabled()) { @@ -215,7 +210,7 @@ private Optional evalDataMaskPolicy(AuthorizedSubject subject, Aut RangerAccessResourceImpl resource = createResource(table.getCatalog(), table.getDatabase(), table.getTable(), col); RangerAccessRequestImpl request = createRequest(subject); - request.setAccessType(DorisAccessType.SELECT.name()); + request.setAccessType(readAccessTypeName()); request.setResource(resource); if (LOG.isDebugEnabled()) { @@ -252,6 +247,19 @@ private Optional evalDataMaskPolicy(AuthorizedSubject subject, Aut } } + /** + * How this Ranger service type spells the access type a read is asked with. + * + *

Row-filter and data-mask lookups are made with it rather than left unset: unset means "any access", + * which makes Ranger walk every permission item, and the Ranger UI writes these policies against the read + * access type anyway, so asking with anything else costs time without changing the answer. + * + *

Every service type Doris talks to happens to spell it {@code SELECT}, which is why this used to be + * one hard-coded value for both. It is asked of the subclass because the spelling belongs to the Ranger + * service definition, not to Doris. + */ + protected abstract String readAccessTypeName(); + protected abstract RangerAccessRequestImpl createRequest(AuthorizedSubject subject); protected abstract RangerAccessResourceImpl createResource(String ctl, String db, String tbl); diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml new file mode 100644 index 00000000000000..f4d35487e445ac --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml @@ -0,0 +1,95 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization-plugins + ../pom.xml + + fe-authorization-plugin-ranger-doris + jar + Doris FE Authorization Plugin - Ranger Doris + Authorizes a whole Doris instance against a Ranger service of type doris. Selected by name + in fe.conf (access_controller_type = ranger-doris). + + + + + org.apache.doris + fe-authorization-plugin-ranger-common + ${project.version} + + + + org.apache.doris + fe-authorization-spi + ${project.version} + provided + + + org.apache.doris + fe-authorization-api + ${project.version} + provided + + + + org.apache.ranger + ranger-plugins-common + + + com.google.guava + guava + + + org.apache.logging.log4j + log4j-api + + + + org.junit.jupiter + junit-jupiter + test + + + org.mockito + mockito-core + test + + + + org.mockito + mockito-inline + test + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + + + diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisAccessType.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisObjectType.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisObjectType.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisObjectType.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/DorisObjectType.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java similarity index 95% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java index d32f5a7639b2e5..ec2b49e9964442 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java @@ -26,7 +26,6 @@ import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; -import org.apache.doris.resource.workloadgroup.WorkloadGroupMgr; import com.google.common.annotations.VisibleForTesting; import org.apache.logging.log4j.LogManager; @@ -56,6 +55,16 @@ public class RangerDorisAccessController extends RangerAccessController { // ranger must set name, we agreed that this name must be used private static final String GLOBAL_PRIV_FIXED_NAME = "*"; + /** + * The workload group every user may use without holding a privilege on it, spelled out here rather than + * read from the engine so that this plugin compiles against the authorization contract alone. + * + *

It has to stay equal to {@code WorkloadGroupMgr.DEFAULT_GROUP_NAME}. Nothing enforces that across + * the two, so if the engine ever renames its default group, users of a Ranger-governed instance start + * needing an explicit Ranger policy for the group they were silently allowed before. + */ + private static final String ENGINE_DEFAULT_WORKLOAD_GROUP = "normal"; + private RangerBasePlugin dorisPlugin; // private static ScheduledThreadPoolExecutor logFlushTimer = ThreadPoolManager.newDaemonScheduledThreadPool(1, // "ranger-doris-audit-log-flusher-timer", true); @@ -129,7 +138,7 @@ DorisObjectType.RESOURCE, named(resource))), case WORKLOAD_GROUP: { // For compatibility with older versions, it is not needed to check the privileges of the // default group. - if (WorkloadGroupMgr.DEFAULT_GROUP_NAME.equals(named(resource))) { + if (ENGINE_DEFAULT_WORKLOAD_GROUP.equals(named(resource))) { return; } EnumSet granted = EnumSet.noneOf(AccessAction.class); @@ -185,6 +194,11 @@ private RangerAccessRequestImpl createRequest(AuthorizedSubject subject, DorisAc return request; } + @Override + protected String readAccessTypeName() { + return DorisAccessType.SELECT.name(); + } + @Override protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject) { RangerAccessRequestImpl request = new RangerAccessRequestImpl(); diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java similarity index 95% rename from fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java index b63a344debcf94..28963f00850272 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactory.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java @@ -15,12 +15,11 @@ // specific language governing permissions and limitations // under the License. -package org.apache.doris.mysql.privilege; +package org.apache.doris.catalog.authorizer.ranger.doris; import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.authorization.spi.AuthorizationPlugin; import org.apache.doris.authorization.spi.AuthorizationPluginFactory; -import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisResource.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisResource.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisResource.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisResource.java diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory new file mode 100644 index 00000000000000..45fe48fa74c92f --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory @@ -0,0 +1,18 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessControllerFactory diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java similarity index 94% rename from fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java index 58c9e579276b3f..ce733fc42e4e62 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerDorisAccessControllerFactoryTest.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java @@ -15,11 +15,10 @@ // specific language governing permissions and limitations // under the License. -package org.apache.doris.mysql.privilege; +package org.apache.doris.catalog.authorizer.ranger.doris; import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.authorization.spi.AuthorizationPlugin; -import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; import org.junit.Assert; import org.junit.Before; diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java similarity index 93% rename from fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java index 2d2dd6f0dcf28c..7630381979e679 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/RangerTest.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java @@ -15,7 +15,7 @@ // specific language governing permissions and limitations // under the License. -package org.apache.doris.mysql.privilege; +package org.apache.doris.catalog.authorizer.ranger.doris; import org.apache.doris.authorization.AccessAction; import org.apache.doris.authorization.AccessContext; @@ -27,9 +27,6 @@ import org.apache.doris.authorization.DataMaskSpec; import org.apache.doris.authorization.ResourceKind; import org.apache.doris.authorization.spi.AuthorizationContext; -import org.apache.doris.catalog.authorizer.ranger.doris.DorisAccessType; -import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessController; -import org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisResource; import com.google.common.base.Strings; import com.google.common.collect.Lists; @@ -52,6 +49,9 @@ public class RangerTest { public static class DorisTestPlugin extends RangerBasePlugin { + /** The access type the last masking lookup asked with; see testDataMaskLookupAsksWithTheReadAccessType. */ + private String lastDataMaskAccessType; + public DorisTestPlugin(String serviceName) { super(serviceName, null, null); // super.init(); @@ -87,6 +87,7 @@ public RangerAccessResult isAccessAllowed(RangerAccessRequest request) { @Override public RangerAccessResult evalDataMaskPolicies(RangerAccessRequest request, RangerAccessResultProcessor resultProcessor) { + lastDataMaskAccessType = request.getAccessType(); RangerAccessResource resource = request.getResource(); String ctl = (String) resource.getValue(RangerDorisResource.KEY_CATALOG); String db = (String) resource.getValue(RangerDorisResource.KEY_DATABASE); @@ -297,6 +298,20 @@ public void testDataMask() { Assertions.assertFalse(masks.containsKey("col4")); } + /** + * Which masking policies a Doris service returns depends on the access type the lookup asks with, and + * that string reaches a Ranger server nobody rebuilds when Doris changes. The masking stub above answers + * by resource alone, so without this nothing here would notice the access type changing at all. + */ + @Test + public void testDataMaskLookupAsksWithTheReadAccessType() { + DorisTestPlugin plugin = new DorisTestPlugin("test"); + new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE).getDataMasks(USER, + AuthorizedResource.table("ctl1", "db1", "tbl1"), Sets.newHashSet("col1"), AccessContext.NONE); + + Assertions.assertEquals(DorisAccessType.SELECT.name(), plugin.lastDataMaskAccessType); + } + @Test public void testComputeGroupAuth() throws AccessDeniedException { check(AuthorizedResource.cloud(ResourceKind.CLOUD_COMPUTE_GROUP, "cg1"), USAGE); diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml new file mode 100644 index 00000000000000..4a964b6a87e342 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml @@ -0,0 +1,108 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization-plugins + ../pom.xml + + fe-authorization-plugin-ranger-hive + jar + Doris FE Authorization Plugin - Ranger Hive + Authorizes one external catalog against a Ranger service of type hive. Selected in that + catalog's properties (access_controller.class). + + + + + org.apache.doris + fe-authorization-plugin-ranger-common + ${project.version} + + + + org.apache.doris + fe-authorization-spi + ${project.version} + provided + + + org.apache.doris + fe-authorization-api + ${project.version} + provided + + + + org.apache.ranger + ranger-plugins-common + + + + org.apache.hadoop + hadoop-client-api + + + com.google.guava + guava + + + org.apache.commons + commons-lang3 + + + org.apache.logging.log4j + log4j-api + + + + org.junit.jupiter + junit-jupiter + test + + + org.mockito + mockito-core + test + + + + org.mockito + mockito-inline + test + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + + + diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveAccessType.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveAccessType.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveAccessType.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveAccessType.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveObjectType.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveObjectType.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveObjectType.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/HiveObjectType.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java similarity index 91% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java index d6f1bc9e7a30e8..1bd92cd8c229e2 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java @@ -27,9 +27,9 @@ import org.apache.doris.authorization.RowFilterSpec; import org.apache.doris.authorization.spi.AuthorizationContext; import org.apache.doris.catalog.authorizer.ranger.RangerAccessController; -import org.apache.doris.common.ThreadPoolManager; import com.google.common.annotations.VisibleForTesting; +import com.google.common.util.concurrent.ThreadFactoryBuilder; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.ranger.plugin.policyengine.RangerAccessRequest; @@ -60,8 +60,22 @@ */ public class RangerHiveAccessController extends RangerAccessController { private static final Logger LOG = LogManager.getLogger(RangerHiveAccessController.class); - private static final ScheduledThreadPoolExecutor LOG_FLUSH_TIMER = ThreadPoolManager.newDaemonScheduledThreadPool(1, - "ranger-hive-audit-log-flusher-timer", true); + + /** + * Drains the audit buffers. One thread for the process: every catalog bound to a Hive Ranger service + * schedules its own task on this timer. + * + *

Built here rather than through the engine's thread pool manager, which a plugin outside fe-core + * cannot reach. What that costs is the pool's entry in the FE thread-pool metrics + * ({@code doris_fe_thread_pool} with name {@code ranger-hive-audit-log-flusher-timer}), which no plugin + * loaded from its own directory can register into; for a fixed single-thread timer those gauges never + * moved. + */ + private static final ScheduledThreadPoolExecutor LOG_FLUSH_TIMER = new ScheduledThreadPoolExecutor(1, + new ThreadFactoryBuilder() + .setDaemon(true) + .setNameFormat("ranger-hive-audit-log-flusher-timer-%d") + .build()); /** The name this source is selected by in catalog properties. */ public static final String NAME = "ranger-hive"; @@ -214,6 +228,16 @@ private RangerAccessRequestImpl createRequest(AuthorizedSubject subject, HiveAcc return request; } + /** + * Upper case, unlike the privilege checks above, which lower case the access type they ask with. That + * asymmetry is what a Hive service has always been asked, so it stays: which policies match is decided by + * the deployed Ranger service definition, and this is not the place to find out the hard way. + */ + @Override + protected String readAccessTypeName() { + return HiveAccessType.SELECT.name(); + } + @Override protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject) { RangerAccessRequestImpl request = new RangerAccessRequestImpl(); diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditHandler.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditHandler.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditHandler.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditHandler.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusher.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusher.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusher.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusher.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuthorizerProvider.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuthorizerProvider.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuthorizerProvider.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuthorizerProvider.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveResource.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveResource.java similarity index 100% rename from fe/fe-core/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveResource.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveResource.java diff --git a/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory similarity index 92% rename from fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory index 6d19de0830694a..a6b56f882e257f 100644 --- a/fe/fe-core/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory @@ -14,6 +14,5 @@ # See the License for the specific language governing permissions and # limitations under the License. # -# -org.apache.doris.mysql.privilege.RangerDorisAccessControllerFactory + org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java similarity index 69% rename from fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java index 20ed56f7fcd823..f47f76cb6491ca 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java @@ -18,16 +18,20 @@ package org.apache.doris.catalog.authorizer.ranger.hive; import org.apache.doris.authorization.AccessAction; +import org.apache.doris.authorization.AccessContext; import org.apache.doris.authorization.AccessRequirement; import org.apache.doris.authorization.AccessRequirements; +import org.apache.doris.authorization.AuthorizedResource; import org.apache.doris.authorization.AuthorizedSubject; import org.apache.doris.authorization.spi.AuthorizationContext; import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableSet; +import org.apache.ranger.plugin.policyengine.RangerAccessRequest; import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl; import org.junit.Assert; import org.junit.Test; +import org.mockito.ArgumentCaptor; import org.mockito.MockedConstruction; import org.mockito.Mockito; @@ -93,4 +97,35 @@ public void testAccessTypeIsRecognisedByWhatIsAsked() { Assert.assertEquals(HiveAccessType.NONE, RangerHiveAccessController.accessTypeOf(AccessRequirement.of(AccessAction.USAGE))); } + + /** + * Which row-filter and masking policies a Hive service returns depends on the access type the lookup + * asks with, and that string reaches a Ranger server nobody rebuilds when Doris changes. + * + *

Pinned upper case on purpose. The privilege checks above ask with the access type lower cased, so + * the two paths disagree - and they have disagreed for as long as Doris has had a Hive Ranger source, + * because the shared lookup used to be written against the Doris service's spelling for both services. + * Making them agree is a change to which policies match on a deployed Ranger, not a tidy-up. + */ + @Test + public void testRowFilterLookupAsksWithTheReadAccessType() { + AuthorizationContext context = Mockito.mock(AuthorizationContext.class); + try (MockedConstruction plugin = Mockito.mockConstruction(RangerHivePlugin.class); + MockedConstruction audit = + Mockito.mockConstruction(RangerHiveAuditHandler.class)) { + RangerHiveAccessController controller = new RangerHiveAccessController( + ImmutableMap.of("ranger.service.name", "hive"), context); + try { + controller.getRowFilters(SUBJECT, AuthorizedResource.table("ctl", "db", "tbl"), + AccessContext.NONE); + + ArgumentCaptor asked = ArgumentCaptor.forClass(RangerAccessRequest.class); + Mockito.verify(plugin.constructed().get(0)) + .evalRowFilterPolicies(asked.capture(), Mockito.any()); + Assert.assertEquals("SELECT", asked.getValue().getAccessType()); + } finally { + controller.close(); + } + } + } } diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusherTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusherTest.java similarity index 100% rename from fe/fe-core/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusherTest.java rename to fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditLogFlusherTest.java diff --git a/fe/fe-authorization/fe-authorization-plugins/pom.xml b/fe/fe-authorization/fe-authorization-plugins/pom.xml new file mode 100644 index 00000000000000..eab78eb5aed1d2 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/pom.xml @@ -0,0 +1,46 @@ + + + + 4.0.0 + + org.apache.doris + ${revision} + fe-authorization + ../pom.xml + + fe-authorization-plugins + pom + Doris FE Authorization Plugins + Authorization sources shipped with Doris, each installed from its own plugin directory. + + + + fe-authorization-plugin-ranger-common + fe-authorization-plugin-ranger-doris + fe-authorization-plugin-ranger-hive + + diff --git a/fe/fe-authorization/pom.xml b/fe/fe-authorization/pom.xml index 520dffeb69c193..139757ab20542d 100644 --- a/fe/fe-authorization/pom.xml +++ b/fe/fe-authorization/pom.xml @@ -79,5 +79,6 @@ under the License. fe-authorization-api fe-authorization-spi + fe-authorization-plugins diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index 8b90a10bf0dd67..b7c1ca69c9deb5 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -779,9 +779,20 @@ under the License. okhttp-jvm + - org.apache.ranger - ranger-plugins-common + ${project.groupId} + fe-authorization-plugin-ranger-doris + ${project.version} + test diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java index 0b56c1a244dda5..75b8308fde7786 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java +++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java @@ -1953,7 +1953,8 @@ public static void checkCatalogProperties(Map properties, boolea // validate access controller properties // eg: // ( - // "access_controller.class" = "org.apache.doris.mysql.privilege.RangerHiveAccessControllerFactory", + // "access_controller.class" = + // "org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory", // "access_controller.properties.prop1" = "xxx", // "access_controller.properties.prop2" = "yyy", // ) diff --git a/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java b/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java index 86476b830da832..fe1342db98b012 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java +++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java @@ -469,7 +469,7 @@ public boolean validatePropertiesBeforeUpdate( /** * eg: * ( - * ""access_controller.class" = "org.apache.doris.mysql.privilege.RangerHiveAccessControllerFactory", + * ""access_controller.class" = "org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory", * "access_controller.properties.prop1" = "xxx", * "access_controller.properties.prop2" = "yyy", * ) diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java index 6b8b57f6bef11e..bb16f28ab8f97a 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/CatalogAccessController.java @@ -32,9 +32,10 @@ * *

A controller is asked only about the resources it governs, and its answer is final: nothing outside it * grants first. In particular the engine no longer establishes a global privilege before routing, so an - * implementation that wants "holding the privilege globally is enough" has to say so itself - see - * {@link org.apache.doris.catalog.authorizer.ranger.RangerAccessController}, which defers to whichever - * source owns global scope. + * implementation that wants "holding the privilege globally is enough" has to say so itself - as the Ranger + * sources do, deferring to whichever source owns global scope + * ({@code org.apache.doris.catalog.authorizer.ranger.RangerAccessController}, in the ranger plugin and so + * not on this module's class path). * *

This is the older shape of that contract, kept because a catalog's {@code access_controller.class} * names an implementation of it and such implementations exist outside this repository. The engine reaches diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java index fbcbd0b35df914..671cafac671652 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/PluginApiVersionWiringTest.java @@ -237,15 +237,15 @@ public void authorizationPluginFromADirectoryCannotTakeTheNameOfAShippedSource() Deencapsulation.getField(manager, "authorizationPluginFactories"); AuthorizationPluginFactory installed = factories.get( ShadowingAuthorizationPluginFactory.SHADOWED_NAME); - Assertions.assertNotNull(installed, "the shipped ranger-doris source disappeared from the test setup," - + " so this test would pass without proving anything"); + Assertions.assertNotNull(installed, "the class-path ranger-doris source disappeared from the test" + + " setup, so this test would pass without proving anything"); // By class NAME, not by Class object. A directory plugin is loaded through its own classloader, so // its factory class is never the same object as the one this test holds - even when it did displace // the shipped source. Written as an identity comparison first, this assertion could not fail at all: // a mutation deleting the guard below left the whole test green. Assertions.assertNotEquals(ShadowingAuthorizationPluginFactory.class.getName(), installed.getClass().getName(), - "a jar dropped into the plugin directory displaced a source shipped with the FE; an" + "a jar dropped into the plugin directory displaced a source on the FE's class path; an" + " allow-everything plugin under a real source's name is the whole point of" + " refusing this"); // Refusing it has to release it too: a plugin kept in the directory runtime keeps its classloader, diff --git a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java index f1148c58176629..d03fdad344c268 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java +++ b/fe/fe-core/src/test/java/org/apache/doris/pluginapiversion/testplugins/ShadowingAuthorizationPluginFactory.java @@ -28,8 +28,8 @@ import java.util.Map; /** - * An authorization source that claims a name the FE already ships ({@code ranger-doris}), so that a test can - * check what happens when somebody drops such a jar into the plugin directory. + * An authorization source that claims a name another source on the FE's class path already answers to, so + * that a test can check what happens when somebody drops such a jar into the plugin directory. * *

Not hypothetical: an allow-everything plugin under the name of a real source would be the cheapest way * to turn a filesystem write into read access to everything that source governs. @@ -37,7 +37,14 @@ public class ShadowingAuthorizationPluginFactory implements AuthorizationPluginFactory, AuthorizationPlugin { - /** Deliberately the name of a source registered on the FE's own class path. */ + /** + * Deliberately the name of a source registered through the class-path channel. + * + *

Doris no longer ships one: the Ranger sources are installed from {@code plugins/authorization/} + * like any third-party plugin, and here {@code ranger-doris} is on the class path only because the + * ranger plugin module is a test dependency of fe-core. That still exercises the guard for what does + * reach the class-path channel in production - a jar an operator put in {@code fe/lib}. + */ public static final String SHADOWED_NAME = "ranger-doris"; @Override From 712b5cc5d5d52897f8ab081cb6fc1e1c92ee7770 Mon Sep 17 00:00:00 2001 From: morningman Date: Thu, 13 Aug 2026 09:09:58 +0800 Subject: [PATCH 12/22] [feat](authorization) ship the Ranger sources as plugins the release installs The previous commit took Ranger out of fe-core. Nothing put it back into a release, so a build from that commit alone produces an FE where access_controller_type = ranger-doris refuses to start, without anything in the build having failed. Each of the two sources now assembles the plugin zip DirectoryPluginRuntimeManager reads - its own jar at the root, everything it needs at runtime in lib/ - and build.sh unpacks them into plugins/authorization/ranger-doris/ and plugins/authorization/ranger-hive/. The build list and the deploy list are the same list, written next to each other, because a module in one and not the other does not fail: the deploy step unpacks whatever archive an earlier build left behind, and ships stale. plugins/authorization/ is created whether or not either module built, since it is also where an administrator drops a third-party source. The api and spi jars are provided and so absent from the zips; they are loaded parent-first from the FE, which is what makes the types a plugin hands back the ones the engine asked for. Ranger and its hadoop closure are bundled instead of borrowed from fe/lib, which costs about 69MB per plugin and buys a plugin whose behaviour does not change when the host's dependencies do. Co-Authored-By: Claude Opus 5 (1M context) --- build.sh | 33 +++++++++ .../pom.xml | 19 +++++ .../src/main/assembly/plugin-zip.xml | 69 +++++++++++++++++++ .../pom.xml | 19 +++++ .../src/main/assembly/plugin-zip.xml | 69 +++++++++++++++++++ 5 files changed, 209 insertions(+) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/assembly/plugin-zip.xml create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/assembly/plugin-zip.xml diff --git a/build.sh b/build.sh index 41ad003e28e8dc..6c267610cf6232 100755 --- a/build.sh +++ b/build.sh @@ -833,6 +833,17 @@ if [[ "${BUILD_FE}" -eq 1 ]]; then fi done unset _conn_mod + # Authorization plugin modules (loaded at runtime from plugins/authorization/). Keep this list + # identical to the deploy loop's (search AUTHZ_PLUGIN_DIR), for the same reason as the connectors: + # the deploy step unzips whatever archive is left in the module's target/, so a module built here + # but not deployed there - or the other way round - ships a stale plugin without failing anything. + # ranger-common is a library the two below depend on; -am builds it, nothing deploys it alone. + for _authz_mod in ranger-doris ranger-hive; do + if [[ -d "${DORIS_HOME}/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-${_authz_mod}" ]]; then + modules+=("fe-authorization/fe-authorization-plugins/fe-authorization-plugin-${_authz_mod}") + fi + done + unset _authz_mod for extra_module_path in "${FE_EXTRA_MODULE_PATHS[@]}"; do modules+=("${extra_module_path}") done @@ -1245,6 +1256,28 @@ if [[ "${BUILD_FE}" -eq 1 ]]; then done unset CONN_PLUGIN_DIR conn_module conn_plugin_target conn_module_dir conn_zip conn_conf_tpl + # Deploy authorization sources as independent plugin directories. + # Each sub-directory is one source AccessControllerManager can install, named in fe.conf by + # access_controller_type or in a catalog's access_controller.class. Created even when no module + # produced a zip, because it is also where an administrator drops a third-party source. + # Keep the module list identical to the build list's (search _authz_mod). + AUTHZ_PLUGIN_DIR="${DORIS_OUTPUT}/fe/plugins/authorization" + mkdir -p "${AUTHZ_PLUGIN_DIR}" + for authz_module in ranger-doris ranger-hive; do + authz_plugin_target="${AUTHZ_PLUGIN_DIR}/${authz_module}" + authz_module_dir="${DORIS_HOME}/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-${authz_module}" + if [ ! -d "${authz_module_dir}" ]; then + continue + fi + authz_zip="${authz_module_dir}/target/doris-fe-authorization-${authz_module}.zip" + if [ ! -f "${authz_zip}" ]; then + continue + fi + mkdir -p "${authz_plugin_target}" + unzip -o "${authz_zip}" -d "${authz_plugin_target}/" + done + unset AUTHZ_PLUGIN_DIR authz_module authz_plugin_target authz_module_dir authz_zip + # RC-4: self-contain the paimon connector plugin for OSS. The connector sets # fs.oss.impl=com.aliyun.jindodata.oss.JindoOssFileSystem; that impl lives in the jindofs jars, # which are packaged from thirdparty by post-build.sh into fe/lib/jindofs (NOT a maven artifact). diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml index f4d35487e445ac..ba849823d2aba4 100644 --- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/pom.xml @@ -85,11 +85,30 @@ under the License. + doris-fe-authorization-ranger-doris org.apache.maven.plugins maven-compiler-plugin + + maven-assembly-plugin + + false + + src/main/assembly/plugin-zip.xml + + + + + make-assembly + package + + single + + + + diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/assembly/plugin-zip.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/assembly/plugin-zip.xml new file mode 100644 index 00000000000000..1a0744b47bd5b4 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/assembly/plugin-zip.xml @@ -0,0 +1,69 @@ + + + + + plugin + + zip + + false + + + + + ${project.build.directory}/${project.build.finalName}.jar + / + + + + + + /lib + false + runtime + + + org.apache.logging.log4j:* + org.slf4j:* + + + + diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml index 4a964b6a87e342..2828abce8a68c5 100644 --- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/pom.xml @@ -98,11 +98,30 @@ under the License. + doris-fe-authorization-ranger-hive org.apache.maven.plugins maven-compiler-plugin + + maven-assembly-plugin + + false + + src/main/assembly/plugin-zip.xml + + + + + make-assembly + package + + single + + + + diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/assembly/plugin-zip.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/assembly/plugin-zip.xml new file mode 100644 index 00000000000000..1a0744b47bd5b4 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/assembly/plugin-zip.xml @@ -0,0 +1,69 @@ + + + + + plugin + + zip + + false + + + + + ${project.build.directory}/${project.build.finalName}.jar + / + + + + + + /lib + false + runtime + + + org.apache.logging.log4j:* + org.slf4j:* + + + + From b5c2dda8c7a902c0e4b43605821a1a19a74fe5b2 Mon Sep 17 00:00:00 2001 From: morningman Date: Thu, 13 Aug 2026 10:02:53 +0800 Subject: [PATCH 13/22] [fix](authorization) keep naming an authorization source the way an older release did A catalog names the source governing it in access_controller.class, and that value is persisted with the catalog: an upgraded FE reads back whatever was written when that catalog was created. Moving the Ranger sources into plugins kept every package name except one - the doris-service factory, which lived in fe-core's privilege package and could not stay there without splitting that package across two jars. A catalog naming that class became unreachable at the first statement touching it. A table of the factory classes fe-core used to publish, and the names publishing them now, answers after the runtime table built from the classes that still exist. It lives in the engine rather than in the plugin because the class it names was the engine's own: whoever owned an identifier owns remembering it. Trino, renaming its Hive connector, put the superseded name in the Hive plugin for that same reason - there the identifier that went stale was the plugin's. Both "no plugin factory found" failures now name the directory an authorization plugin is installed in. That is the message an upgrade produces when the plugin directory was not carried across, and without the path it sends an operator looking in fe/lib, where the source used to be. The selectors are pinned as literals, never derived from the classes publishing them: derived, an expectation travels with the code it is meant to pin, and the next package move would leave it green. Co-Authored-By: Claude Opus 5 (1M context) --- ...angerDorisAccessControllerFactoryTest.java | 19 ++++ ...RangerHiveAccessControllerFactoryTest.java | 47 +++++++++ .../privilege/AccessControllerManager.java | 56 ++++++++++- ...zationSourceSelectorCompatibilityTest.java | 97 +++++++++++++++++++ 4 files changed, 215 insertions(+), 4 deletions(-) create mode 100644 fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AuthorizationSourceSelectorCompatibilityTest.java diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java index ce733fc42e4e62..c68db7be04d9cf 100644 --- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactoryTest.java @@ -38,6 +38,25 @@ public void forgetPreviouslyCreatedController() throws Exception { instance.set(null, null); } + /** + * The two strings an operator selects this source by, frozen. + * + *

The name is what {@code access_controller_type} in fe.conf holds; the factory's class name is what a + * catalog's {@code access_controller.class} may hold, and that one is persisted with the catalog and read + * back verbatim by later releases. Written as literals on purpose: derived from the class, they would + * travel with it and a package move would leave this green. Moving this class again means adding the name + * it has today to the table of superseded class names in {@code AccessControllerManager}. + */ + @Test + public void testTheSelectorsThisSourceIsNamedBy() { + RangerDorisAccessControllerFactory factory = new RangerDorisAccessControllerFactory(); + + Assert.assertEquals("ranger-doris", factory.name()); + Assert.assertEquals( + "org.apache.doris.catalog.authorizer.ranger.doris.RangerDorisAccessControllerFactory", + factory.getClass().getName()); + } + /** * One controller per FE, whoever asks for it: it starts a Ranger policy refresher, and a second one would * mean a second refresher polling the same service. diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java new file mode 100644 index 00000000000000..b162f8139cc361 --- /dev/null +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java @@ -0,0 +1,47 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.catalog.authorizer.ranger.hive; + +import org.junit.Assert; +import org.junit.Test; + +public class RangerHiveAccessControllerFactoryTest { + + /** + * The two strings an operator selects this source by, frozen. + * + *

The class name matters most here: this source governs one catalog, and a catalog names it in + * {@code access_controller.class}, which is persisted with the catalog and read back verbatim by every + * later release - the regression suite for Hive catalogs writes exactly this string. It survived the move + * out of fe-core unchanged, which is the only reason those catalogs kept working, so nothing but this + * test stands between the next package move and a catalog nobody can query. + * + *

Written as literals on purpose: derived from the class, they would travel with it and a package move + * would leave this green. Moving this class means adding the name it has today to the table of superseded + * class names in {@code AccessControllerManager}. + */ + @Test + public void testTheSelectorsThisSourceIsNamedBy() { + RangerHiveAccessControllerFactory factory = new RangerHiveAccessControllerFactory(); + + Assert.assertEquals("ranger-hive", factory.name()); + Assert.assertEquals( + "org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory", + factory.getClass().getName()); + } +} diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index 47c11576687b89..c6f774d0921c69 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -50,6 +50,7 @@ import com.google.common.base.Preconditions; import com.google.common.base.Strings; +import com.google.common.collect.ImmutableMap; import com.google.common.collect.Maps; import org.apache.commons.lang3.StringUtils; import org.apache.logging.log4j.LogManager; @@ -103,6 +104,23 @@ public class AccessControllerManager { /** Family label in the process-wide {@link PluginRegistry}, i.e. in information_schema.extensions. */ private static final String PLUGIN_FAMILY = "AUTHORIZATION"; + /** + * Authorization sources that used to be published by a factory class of fe-core's own, and the name each + * is published under now. + * + *

A catalog names its source in {@code access_controller.class}, and that value is persisted with the + * catalog: an FE upgraded across the release that moved a source out of fe-core reads back the class name + * written when the catalog was created. {@link #accessControllerClassNameMapping} answers for every + * factory class that still exists, being filled from the classes actually registered; this table answers + * for the ones that no longer do. + * + *

It lives here, and not in the plugin, because the class it names was fe-core's: whoever owned an + * identifier owns remembering it. Trino, renaming its Hive connector, put the superseded name in the Hive + * plugin for that same reason - there the identifier that went stale was the plugin's own. + */ + private static final Map SOURCES_THAT_LEFT_THE_KERNEL = ImmutableMap.of( + "org.apache.doris.mysql.privilege.RangerDorisAccessControllerFactory", "ranger-doris"); + private Auth auth; // Governs everything no catalog-bound source governs; the built-in model unless configured otherwise private AuthorizationPlugin defaultAccessController; @@ -161,8 +179,7 @@ private AuthorizationPlugin loadAccessControllerOrThrow(String accessControllerN } if (!isKnownAuthorizationSource(accessControllerName)) { throw new RuntimeException("No authorization plugin factory found for " + accessControllerName - + ". Please confirm that your plugin is placed in the correct location." - + apiVersionRejectionHint()); + + "." + pluginLocationHint() + apiVersionRejectionHint()); } Map prop; try { @@ -442,7 +459,15 @@ private boolean isCurrentCatalog(ExternalCatalog catalog) { return currentCatalog == catalog && currentCatalog.getId() == catalog.getId(); } - private String getPluginIdentifierForAccessController(String acClassName) { + /** + * The authorization source a catalog's {@code access_controller.class} names, whichever way it names it: + * by the name the source is published under, by the class name of the factory publishing it, or - for a + * source that has since moved out of fe-core - by the class name that used to publish it. + * + *

Package-private so that the strings older releases accepted here can be pinned by a test; this is + * the only place that decides what any of them mean. + */ + String getPluginIdentifierForAccessController(String acClassName) { String pluginIdentifier = null; if (accessControllerClassNameMapping.containsKey(acClassName)) { pluginIdentifier = accessControllerClassNameMapping.get(acClassName); @@ -450,13 +475,36 @@ private String getPluginIdentifierForAccessController(String acClassName) { if (isKnownAuthorizationSource(acClassName)) { pluginIdentifier = acClassName; } + if (pluginIdentifier == null) { + pluginIdentifier = SOURCES_THAT_LEFT_THE_KERNEL.get(acClassName); + if (pluginIdentifier != null) { + LOG.warn("Catalog property {} = {} names a class this FE no longer has; that source is now" + + " published as '{}'. It is still resolved, but set the property to '{}'.", + CatalogMgr.ACCESS_CONTROLLER_CLASS_PROP, acClassName, pluginIdentifier, + pluginIdentifier); + } + } if (null == pluginIdentifier || !isKnownAuthorizationSource(pluginIdentifier)) { throw new RuntimeException("Access Controller Plugin Factory not found for " + acClassName - + "." + apiVersionRejectionHint()); + + "." + pluginLocationHint() + apiVersionRejectionHint()); } return pluginIdentifier; } + /** + * Where an authorization plugin has to be for this FE to find it. + * + *

Spelled out because the commonest way to reach either "not found" is an upgrade rather than a typo: + * sources that used to be part of the FE itself are installed like any other plugin now, so a deployment + * whose lib directory was assembled by hand loses them with nothing else having changed. + */ + private static String pluginLocationHint() { + return " An authorization plugin is installed as a subdirectory of " + Config.authorization_plugins_dir + + " holding the plugin jar and its lib/ directory, and the release package ships the sources" + + " that used to be part of the FE there. An FE upgraded in place needs that directory" + + " copied across too."; + } + public void removeAccessController(String ctl, long catalogId) { detachAccessController(ctl, catalogId).run(); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AuthorizationSourceSelectorCompatibilityTest.java b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AuthorizationSourceSelectorCompatibilityTest.java new file mode 100644 index 00000000000000..1870cec98fc093 --- /dev/null +++ b/fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/AuthorizationSourceSelectorCompatibilityTest.java @@ -0,0 +1,97 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.mysql.privilege; + +import org.apache.doris.common.Config; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +/** + * Every string an older release let an operator select an authorization source by still selects it. + * + *

One of the two channels is not typed but persisted: a catalog's {@code access_controller.class} is + * written into the catalog's properties when the catalog is created, and read back verbatim by every FE + * afterwards - including one upgraded across the release that moved the Ranger sources out of fe-core into + * plugins of their own. Nothing rewrites those properties, so a value that ever worked has to keep working + * or the catalog stops being reachable at the first statement that touches it. + * + *

Every selector below is written as a literal. Derived instead from the class that publishes the + * source - {@code SomeFactory.class.getName()} - the expectation would travel with the code it is supposed to + * pin, and moving a factory to another package, which is exactly the change this test exists for, would leave + * it green. + */ +public class AuthorizationSourceSelectorCompatibilityTest { + + /** + * The one identifier this work invalidated. That factory lived in this very package until the Ranger + * sources moved out of fe-core; leaving it here would have split the package across two jars. + */ + private static final String DORIS_RANGER_FACTORY_BEFORE_THE_MOVE = + "org.apache.doris.mysql.privilege.RangerDorisAccessControllerFactory"; + + @Test + public void theClassNameASourceHadBeforeLeavingTheKernelStillNamesIt() { + AccessControllerManager manager = new AccessControllerManager(new Auth()); + + Assertions.assertEquals("ranger-doris", + manager.getPluginIdentifierForAccessController(DORIS_RANGER_FACTORY_BEFORE_THE_MOVE), + "a catalog created before the Ranger sources became plugins names a class that is gone;" + + " it has to keep resolving to the source now publishing that behaviour"); + } + + @Test + public void sourceStillPublishedByItsOriginalClassNeedsNoAlias() { + // What keeps the Hive source's class name working - it did not move, so the runtime table filled at + // registration answers for it. That plugin is not on this module's class path, so the source used + // here stands in for it; the literal that matters is pinned in the Hive plugin's own module. + AccessControllerManager manager = new AccessControllerManager(new Auth()); + + Assertions.assertEquals("stub-ranger-doris", manager.getPluginIdentifierForAccessController( + "org.apache.doris.mysql.privilege.StubRangerAccessControllerFactory"), + "a factory class that still exists must resolve without the alias table"); + } + + @Test + public void theNameASourceIsPublishedUnderIsASelectorToo() { + AccessControllerManager manager = new AccessControllerManager(new Auth()); + + // The form fe.conf uses. Its instance-wide channel never accepted a class name - access_controller_type + // has always been a name - so for that channel this is the whole of the compatibility question, once + // the plugin directory is in place. + Assertions.assertEquals("ranger-doris", + manager.getPluginIdentifierForAccessController("ranger-doris")); + Assertions.assertEquals("default", InternalAuthorizationPlugin.NAME, + "the value access_controller_type has meant 'the built-in privilege model' since before" + + " any of this, and is what the setting still defaults to"); + } + + @Test + public void anUnknownSourceSaysWhereAPluginHasToBeInstalled() { + AccessControllerManager manager = new AccessControllerManager(new Auth()); + + RuntimeException e = Assertions.assertThrows(RuntimeException.class, + () -> manager.getPluginIdentifierForAccessController("org.example.NoSuchFactory")); + + // The message an upgraded deployment gets when its plugin directory was not carried across. Naming + // the directory is the whole of its usefulness: the source it is asking for exists, in a place the + // operator has not looked. + Assertions.assertTrue(e.getMessage().contains(Config.authorization_plugins_dir), + "the failure does not say where an authorization plugin belongs: " + e.getMessage()); + } +} From e2aa4e0abb35bb185c3a1053f5d440bac28c6449 Mon Sep 17 00:00:00 2001 From: morningman Date: Thu, 13 Aug 2026 10:03:05 +0800 Subject: [PATCH 14/22] [test](extension) pin that a plugin can still read the FE's own configuration files A plugin's configuration lives in $DORIS_HOME/conf, which is on the FE's class path and therefore the parent of every plugin classloader, and a library inside a plugin looks its configuration up through the classloader that loaded it - its plugin's. The Ranger sources are the live example: RangerConfiguration asks its own classloader for ranger--security.xml, and now that those classes are a plugin, that call reaches ChildFirstClassLoader. It works because only the plural getResources is overridden; the singular one keeps the JDK's parent-first search. Nothing said so, and overriding that one too for symmetry with the class-loading policy is an easy change to make. Ranger does not treat a missing configuration as an error - it carries on with an empty one, authorizing against no policies at all - so this is pinned rather than left to be noticed. Co-Authored-By: Claude Opus 5 (1M context) --- .../loader/ChildFirstClassLoaderTest.java | 69 +++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 fe/fe-extension-loader/src/test/java/org/apache/doris/extension/loader/ChildFirstClassLoaderTest.java diff --git a/fe/fe-extension-loader/src/test/java/org/apache/doris/extension/loader/ChildFirstClassLoaderTest.java b/fe/fe-extension-loader/src/test/java/org/apache/doris/extension/loader/ChildFirstClassLoaderTest.java new file mode 100644 index 00000000000000..f1d1aad46e8143 --- /dev/null +++ b/fe/fe-extension-loader/src/test/java/org/apache/doris/extension/loader/ChildFirstClassLoaderTest.java @@ -0,0 +1,69 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.extension.loader; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.io.IOException; +import java.net.URL; +import java.net.URLClassLoader; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Collections; + +class ChildFirstClassLoaderTest { + + @TempDir + Path tempDir; + + /** + * A plugin's own configuration file lives in the FE's conf directory, not in the plugin, and the plugin + * has to be able to read it. + * + *

{@code $DORIS_HOME/conf} is on the FE's class path, which is the parent of every plugin classloader, + * and a library inside a plugin looks its configuration up through the classloader that loaded it - its + * plugin's. The Ranger sources are the live example: {@code RangerConfiguration.getFileLocation} calls + * {@code RangerConfiguration.class.getClassLoader().getResource("ranger--security.xml")}, and + * since those classes moved into a plugin that call reaches this classloader. Finding nothing is not an + * error there - Ranger carries on with an empty configuration, which authorizes against no policies at + * all - so this has to be pinned rather than left to be noticed. + * + *

It works because {@link ChildFirstClassLoader} overrides only the plural {@code getResources}; the + * singular one keeps the JDK's parent-first search. Overriding that one too, for symmetry with the class + * loading policy, is the change this test exists to fail. + */ + @Test + void configurationFileOnlyTheFeHasIsVisibleToAPlugin() throws IOException { + Path feConf = Files.createDirectory(tempDir.resolve("conf")); + Files.write(feConf.resolve("probe-plugin-security.xml"), + "".getBytes(StandardCharsets.UTF_8)); + Path pluginJars = Files.createDirectory(tempDir.resolve("plugin")); + + try (URLClassLoader fe = new URLClassLoader(new URL[] {feConf.toUri().toURL()}, null); + ChildFirstClassLoader plugin = new ChildFirstClassLoader( + new URL[] {pluginJars.toUri().toURL()}, fe, Collections.emptyList())) { + Assertions.assertNotNull(plugin.getResource("probe-plugin-security.xml"), + "a plugin can no longer read a configuration file out of the FE's conf directory"); + Assertions.assertTrue(plugin.getResources("probe-plugin-security.xml").hasMoreElements(), + "the child-first plural lookup lost its fall-back to the parent"); + } + } +} From 993ab54d30a1c96443d0a0b33859782fcea8ac1d Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 15:19:35 +0800 Subject: [PATCH 15/22] [fix](docker) bring the Ranger stack up without GNU sed or a network inside the container start_ranger patched the bucket into two tracked scripts with `sed -i`, which BSD sed rejects outright and which left the working tree dirty on every run. Those scripts then curl'ed the Doris plugin jars and the service definition from inside ranger-admin, where one flaky download meets `set -e` and takes the whole stack down with it. Fetch all three on the host into the gitignored ranger/cache/ and bind mount it read-only: no substitution in tracked files, and the container needs no network at all. Three more things the ranger_p2 suites need before they can run: - ranger_settings.env exported RANGER_SOLAR_PORT while the template reads RANGER_SOLR_PORT, so the solr service rendered its port mapping as `- :8983`. - Only the service *definition* was registered. The suites write policies to a service *instance* named doris, the name RangerDorisAccessControllerFactory asks for, and nothing created one. - Both installs now look before they POST. On a container restart the duplicate used to fail under `set -e` and kill the entrypoint. Co-Authored-By: Claude Opus 5 (1M context) --- .../docker-compose/ranger/ranger.yaml.tpl | 8 ++- .../docker-compose/ranger/ranger_settings.env | 2 +- .../script/install_doris_ranger_plugins.sh | 15 +++--- .../script/install_doris_service_def.sh | 52 ++++++++++++++++--- .../thirdparties/run-thirdparties-docker.sh | 29 ++++++++++- 5 files changed, 88 insertions(+), 18 deletions(-) diff --git a/docker/thirdparties/docker-compose/ranger/ranger.yaml.tpl b/docker/thirdparties/docker-compose/ranger/ranger.yaml.tpl index 3de94d6fb98f7c..7cd0badabc20ce 100644 --- a/docker/thirdparties/docker-compose/ranger/ranger.yaml.tpl +++ b/docker/thirdparties/docker-compose/ranger/ranger.yaml.tpl @@ -38,11 +38,17 @@ services: interval: 30s timeout: 10s retries: 10 + # setup.sh rebuilds the whole admin install on every boot; on a cold + # machine that runs well past 10 * 30s. + start_period: 600s volumes: - ./ranger-admin/ranger-entrypoint.sh:/opt/ranger-entrypoint.sh - ./script/install_doris_ranger_plugins.sh:/opt/install_doris_ranger_plugins.sh - ./script/install_doris_service_def.sh:/opt/install_doris_service_def.sh - + # Doris plugin jars + service definition, fetched on the host so that a + # flaky download cannot kill the container's `set -e` entrypoint. + - ./cache:/opt/doris-ranger-artifacts:ro + entrypoint : ["bash", "-c", "bash /opt/ranger-entrypoint.sh"] ranger-mysql: diff --git a/docker/thirdparties/docker-compose/ranger/ranger_settings.env b/docker/thirdparties/docker-compose/ranger/ranger_settings.env index 13dd93d517dd1d..2b2fe8b5a18100 100644 --- a/docker/thirdparties/docker-compose/ranger/ranger_settings.env +++ b/docker/thirdparties/docker-compose/ranger/ranger_settings.env @@ -16,6 +16,6 @@ # specific language governing permissions and limitations # under the License. -export RANGER_SOLAR_PORT=8983 +export RANGER_SOLR_PORT=8983 export RANGER_PORT=6081 export RANGER_MYSQL_PORT=33061 diff --git a/docker/thirdparties/docker-compose/ranger/script/install_doris_ranger_plugins.sh b/docker/thirdparties/docker-compose/ranger/script/install_doris_ranger_plugins.sh index c3a1cf428b005c..c2ab4085bbb67a 100755 --- a/docker/thirdparties/docker-compose/ranger/script/install_doris_ranger_plugins.sh +++ b/docker/thirdparties/docker-compose/ranger/script/install_doris_ranger_plugins.sh @@ -12,13 +12,14 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - #!/bin/bash set -ex -if [ ! -d "${RANGER_HOME}/ews/webapp/WEB-INF/classes/ranger-plugins/doris" ]; then - mkdir -p "${RANGER_HOME}/ews/webapp/WEB-INF/classes/ranger-plugins/doris" -fi -cd "${RANGER_HOME}/ews/webapp/WEB-INF/classes/ranger-plugins/doris" -curl -O https://s3BucketName.s3Endpoint/regression/docker/ranger-plugins/mysql-connector-java-8.0.25.jar -curl -O https://s3BucketName.s3Endpoint/regression/docker/ranger-plugins/ranger-doris-plugin-3.0.0-SNAPSHOT.jar \ No newline at end of file +# The jars are fetched on the host into docker-compose/ranger/cache and bind +# mounted here, so this container never needs to reach the network. +ARTIFACTS=/opt/doris-ranger-artifacts +PLUGIN_DIR="${RANGER_HOME}/ews/webapp/WEB-INF/classes/ranger-plugins/doris" + +mkdir -p "${PLUGIN_DIR}" +cp "${ARTIFACTS}/mysql-connector-java-8.0.25.jar" "${PLUGIN_DIR}/" +cp "${ARTIFACTS}/ranger-doris-plugin-3.0.0-SNAPSHOT.jar" "${PLUGIN_DIR}/" diff --git a/docker/thirdparties/docker-compose/ranger/script/install_doris_service_def.sh b/docker/thirdparties/docker-compose/ranger/script/install_doris_service_def.sh index c5eeaa60002adf..2a871d3c70ee55 100755 --- a/docker/thirdparties/docker-compose/ranger/script/install_doris_service_def.sh +++ b/docker/thirdparties/docker-compose/ranger/script/install_doris_service_def.sh @@ -15,13 +15,51 @@ #!/bin/bash set -ex -curl -O https://s3BucketName.s3Endpoint/regression/docker/ranger-plugins/ranger-servicedef-doris.json -until curl -f http://localhost:6080; do +ADMIN=http://localhost:6080 +AUTH=admin:Ranger1234 +# Bind mounted from docker-compose/ranger/cache, fetched on the host. +SERVICE_DEF=/opt/doris-ranger-artifacts/ranger-servicedef-doris.json +# Only used by the admin UI's resource lookup, never by policy evaluation, so a +# Doris that is not up yet (the usual case here) costs nothing. +DORIS_JDBC_URL="${DORIS_JDBC_URL:-jdbc:mysql://host.docker.internal:9030}" +DORIS_JDBC_USER="${DORIS_JDBC_USER:-root}" + +until curl -f "${ADMIN}"; do echo "Waiting for service to be healthy..." sleep 30 done -curl -u admin:Ranger1234 -X POST \ - -H "Accept: application/json" \ - -H "Content-Type: application/json" \ - http://localhost:6080/service/plugins/definitions \ - -d@ranger-servicedef-doris.json \ No newline at end of file + +# Both steps are idempotent: this script reruns on every container restart, and +# a 400 "duplicate" out of `set -e` would take the whole stack down. +if curl -sf -u "${AUTH}" "${ADMIN}/service/plugins/definitions/name/doris" >/dev/null; then + echo "Doris service definition already registered" +else + curl -sS -u "${AUTH}" -X POST \ + -H "Accept: application/json" \ + -H "Content-Type: application/json" \ + "${ADMIN}/service/plugins/definitions" \ + -d@"${SERVICE_DEF}" +fi + +# The regression suites create policies under a service *instance* named +# `doris` -- that is the name RangerDorisAccessControllerFactory asks for. +if curl -sf -u "${AUTH}" "${ADMIN}/service/plugins/services/name/doris" >/dev/null; then + echo "Doris service instance already exists" +else + curl -sS -u "${AUTH}" -X POST \ + -H "Accept: application/json" \ + -H "Content-Type: application/json" \ + "${ADMIN}/service/plugins/services" \ + -d "{ + \"name\": \"doris\", + \"type\": \"doris\", + \"description\": \"Doris regression test service\", + \"isEnabled\": true, + \"configs\": { + \"username\": \"${DORIS_JDBC_USER}\", + \"password\": \"\", + \"jdbc.driver_class\": \"com.mysql.cj.jdbc.Driver\", + \"jdbc.url\": \"${DORIS_JDBC_URL}\" + } + }" +fi diff --git a/docker/thirdparties/run-thirdparties-docker.sh b/docker/thirdparties/run-thirdparties-docker.sh index c4a9079e486d8d..0d1d19bfc88ba3 100755 --- a/docker/thirdparties/run-thirdparties-docker.sh +++ b/docker/thirdparties/run-thirdparties-docker.sh @@ -1736,11 +1736,36 @@ start_polaris() { fi } +# The Doris plugin jars and service definition used to be curl'ed from inside +# ranger-admin, with the bucket patched into the tracked scripts by `sed -i`. +# That both broke on BSD sed and left the working tree dirty, and one flaky +# download killed the container's `set -e` entrypoint. Fetch them here instead, +# into the gitignored cache/ dir that the container bind mounts read-only. +download_ranger_artifacts() { + local dest="${ROOT}/docker-compose/ranger/cache" + local url_prefix="https://${s3BucketName}.${s3Endpoint}/regression/docker/ranger-plugins" + local name + + mkdir -p "${dest}" + for name in ranger-servicedef-doris.json \ + mysql-connector-java-8.0.25.jar \ + ranger-doris-plugin-3.0.0-SNAPSHOT.jar; do + if [[ -s "${dest}/${name}" ]]; then + echo "ranger artifact cached: ${name}" + continue + fi + echo "downloading ${url_prefix}/${name}" + curl -fsSL --retry 10 --retry-all-errors --retry-delay 5 \ + --connect-timeout 30 --speed-limit 1024 --speed-time 120 \ + -o "${dest}/${name}.part" "${url_prefix}/${name}" + mv "${dest}/${name}.part" "${dest}/${name}" + done +} + start_ranger() { echo "RUN_RANGER" export CONTAINER_UID=${CONTAINER_UID} - find "${ROOT}/docker-compose/ranger/script" -type f -exec sed -i "s/s3Endpoint/${s3Endpoint}/g" {} \; - find "${ROOT}/docker-compose/ranger/script" -type f -exec sed -i "s/s3BucketName/${s3BucketName}/g" {} \; + download_ranger_artifacts . "${ROOT}/docker-compose/ranger/ranger_settings.env" envsubst <"${ROOT}"/docker-compose/ranger/ranger.yaml.tpl >"${ROOT}"/docker-compose/ranger/ranger.yaml register_stack_metadata "ranger" "${ROOT}/docker-compose/ranger/ranger.yaml" "${ROOT}/docker-compose/ranger/ranger_settings.env" From 4698c59ab8ec3430c3c6885de1631d96a8af2bec Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 15:19:52 +0800 Subject: [PATCH 16/22] [fix](authorization) give the Ranger sources the Jersey their admin client needs RangerBasePlugin.init() constructs a RangerAdminRESTClient, which is Jersey 1.x, and fe/pom.xml excludes the jersey-bundle Ranger declares it against (#37575). Nothing in fe-core ever built one, so the gap only surfaced once the Ranger sources became plugins of their own: the FE dies on startup with NoClassDefFoundError: javax/ws/rs/core/Cookie. Declaring it on the plugin alone does not fix it. com.sun. and javax. are parent-first for every plugin classloader, so the host's jersey-core still wins for com.sun.jersey.core while javax.ws.rs remains invisible from there, and the failure only moves to NoClassDefFoundError: javax/ws/rs/ext/ContextResolver. fe-core declares it too, which makes the host's copy the whole stack and the plugin's bundle a findClass fallback - the shape FileSystemPluginManager.FS_PARENT_FIRST_PREFIXES already gives hadoop. Co-Authored-By: Claude Opus 5 (1M context) --- .../pom.xml | 13 +++++++++++++ fe/fe-core/pom.xml | 16 ++++++++++++++++ fe/pom.xml | 14 ++++++++++++++ 3 files changed, 43 insertions(+) diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml index 4727479e9ae26c..a2197c9947b1af 100644 --- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml +++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/pom.xml @@ -56,6 +56,19 @@ under the License. org.apache.ranger ranger-plugins-common + + + com.sun.jersey + jersey-client + com.google.guava guava diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index b7c1ca69c9deb5..3e62a39d4ac062 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -153,6 +153,22 @@ under the License. + + + com.sun.jersey + jersey-client + ${project.groupId} fe-foundation diff --git a/fe/pom.xml b/fe/pom.xml index 60371c37bc9d08..95eabce18a95f5 100644 --- a/fe/pom.xml +++ b/fe/pom.xml @@ -394,6 +394,8 @@ under the License. 3.9.3 2.4 2.8.0 + + 1.19.4 1.70 6.5.1 2.0.3 @@ -1208,6 +1210,18 @@ under the License. + + + com.sun.jersey + jersey-client + ${jersey.version} + org.objenesis objenesis From e9d2ea8d1b3c6543780e85aa72f825f0d5fde387 Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 15:20:06 +0800 Subject: [PATCH 17/22] [fix](authorization) build an authorization source under its own plugin's classloader A plugin that bundles a library resolving class names through the thread context classloader gets the engine's copy of that library rather than its own. Hadoop's Configuration is the one both Ranger sources drag in, and org.apache.hadoop. is child-first for this family, so SecurityUtil and DomainNameResolver came from the plugin while DNSDomainNameResolver was resolved through the FE's hadoop-common, and startup died with RuntimeException: class org.apache.hadoop.net.DNSDomainNameResolver not org.apache.hadoop.net.DomainNameResolver Swap the context classloader for the duration of the factory call. The refresher threads such a library starts are created inside it and inherit the loader, so they go on working once the swap is undone. Co-Authored-By: Claude Opus 5 (1M context) --- .../privilege/AccessControllerManager.java | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java index c6f774d0921c69..21448afcc0d8b0 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java @@ -197,6 +197,13 @@ private AuthorizationPlugin loadAccessControllerOrThrow(String accessControllerN *

A source written against the current contract is built with the context it may put questions to the * engine through. It cannot be handed that context any earlier than this - the context has to name the * source it belongs to, and the source does not exist until its factory has run. + * + *

The factory runs under its own plugin's classloader as the thread context one. A plugin that bundles + * a library which resolves class names through the context classloader - Hadoop's Configuration is the + * one both Ranger sources drag in - would otherwise load the name from the engine's copy of that library + * and get back a class the plugin's own copy does not recognise as implementing its interface. The + * refresher threads such a library starts inherit this classloader too, which is what keeps them working + * after the swap is undone. */ private AuthorizationPlugin create(String name, Map properties) { AuthorizationPluginFactory factory = authorizationPluginFactories.get(name); @@ -204,8 +211,15 @@ private AuthorizationPlugin create(String name, Map properties) return adapt(name, accessControllerFactoriesCache.get(name).createAccessController(properties)); } EngineAuthorizationContext context = new EngineAuthorizationContext(this, auth); - AuthorizationPlugin plugin = factory.create( - properties == null ? Collections.emptyMap() : properties, context); + Thread current = Thread.currentThread(); + ClassLoader callerLoader = current.getContextClassLoader(); + AuthorizationPlugin plugin; + try { + current.setContextClassLoader(factory.getClass().getClassLoader()); + plugin = factory.create(properties == null ? Collections.emptyMap() : properties, context); + } finally { + current.setContextClassLoader(callerLoader); + } context.servedBy(plugin); return plugin; } From a8f054678da523fc374800202c89be1ba27311dd Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 17:24:10 +0800 Subject: [PATCH 18/22] [test](authorization) cover an authorization source bound to one catalog Every suite in ranger_p2 exercised the instance wide source named by access_controller_type. The other way a source is selected, a catalog naming it in access_controller.class, had no coverage beyond one negative case in external_table_p0 that only checks a misspelled service is refused. What makes the catalog bound case its own thing is that the decision is split: catalog level questions stay with the instance wide source and everything inside the catalog goes to the bound one, so a table needs a grant in each of two Ranger services. The suite pins that split from both sides. A Hive policy alone leaves SWITCH refused by the instance wide source; the catalog grant alone leaves the table refused by the bound one, and the two refusals do not even have the same shape, the Hive one naming no catalog because a Hive service has no such scope. Row filters and column masks are asked of whoever answers for the table, so the last two cases read one Hive table through two catalogs at once: filtered and masked through the bound one, whole and in the clear through a catalog with no source of its own. Also covers naming the source by its factory class, which is what a catalog created before the sources became plugins still has persisted. Needs ranger-hive-security.xml in the FE conf directory, which is what decides the Ranger service instance; the catalog property of that name is handed to RangerBasePlugin as the service type and so only picks which file is read. The Ranger service itself is this suite's fixture and is created here. Co-Authored-By: Claude Opus 5 (1M context) --- ...st_ranger_catalog_access_controller.groovy | 309 ++++++++++++++++++ 1 file changed, 309 insertions(+) create mode 100644 regression-test/suites/ranger_p2/test_ranger_catalog_access_controller.groovy diff --git a/regression-test/suites/ranger_p2/test_ranger_catalog_access_controller.groovy b/regression-test/suites/ranger_p2/test_ranger_catalog_access_controller.groovy new file mode 100644 index 00000000000000..875600823111bd --- /dev/null +++ b/regression-test/suites/ranger_p2/test_ranger_catalog_access_controller.groovy @@ -0,0 +1,309 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +import org.apache.ranger.RangerClient +import org.apache.ranger.plugin.model.RangerPolicy +import org.apache.ranger.plugin.model.RangerService + +/** + * An authorization source bound to one catalog rather than to the whole instance. + * + * The sibling suites all exercise the instance wide source named by fe.conf's access_controller_type. + * This one covers the other way a Ranger source is selected: a catalog naming it in + * access_controller.class. That splits the decision in two, which is what the cases below pin: + * catalog level questions stay with the instance wide source, everything inside the catalog goes to + * the source the catalog is bound to. Reaching a table therefore needs a grant in each of the two + * Ranger services, and taking either one away is enough to close it again. + * + * The last two cases carry that split into row filters and column masks, which are asked of whoever + * answers for the table: the same table read through a bound catalog and through one without a source + * of its own comes back filtered and masked in the first, whole and in the clear in the second. + * + * Prerequisites beyond the ones the sibling suites need: + * + * - ranger-hive-security.xml in the FE's conf directory. RangerBasePlugin is handed the catalog's + * access_controller.properties.ranger.service.name as the service *type*, so that property picks + * the file (ranger--security.xml) while the file's ranger.plugin.hive.service.name picks + * the Ranger service instance. HIVE_SERVICE_NAME below has to match that property. + * - The ranger-hive plugin installed under plugins/authorization, which the release ships. + * + * The Ranger service instance is created here: it is this suite's fixture, not the environment's. + */ +suite("test_ranger_catalog_access_controller", "p2,ranger,external") { + String enabled = context.config.otherConfigs.get("enableRangerTest") + String rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint") + String rangerUser = context.config.otherConfigs.get("rangerUser") + String rangerPassword = context.config.otherConfigs.get("rangerPassword") + String dorisServiceName = context.config.otherConfigs.get("rangerServiceName") + String externalEnvIp = context.config.otherConfigs.get("externalEnvIp") + String hmsPort = context.config.otherConfigs.get("hive3HmsPort") + + if (enabled == null || !enabled.equalsIgnoreCase("true")) { + logger.info("skip test_ranger_catalog_access_controller because enableRangerTest is not true") + return + } + + // Must equal ranger.plugin.hive.service.name in the FE's ranger-hive-security.xml. + String HIVE_SERVICE_NAME = 'hive' + String FACTORY_CLASS = 'org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory' + + def tokens = context.config.jdbcUrl.split('/') + def defaultJdbcUrl = tokens[0] + "//" + tokens[2] + "/?" + + String boundCatalog = 'ranger_ctl_bound' + String fqcnCatalog = 'ranger_ctl_fqcn' + String plainCatalog = 'ranger_ctl_plain' + String dbName = 'ranger_ctl_db' + String tblName = 'ranger_ctl_tbl' + String user = 'ranger_ctl_user' + String pwd = 'C123_567p' + String hivePolicyName = 'ranger_ctl_hive_policy' + String hiveRowFilterName = 'ranger_ctl_hive_row_filter' + String hiveMaskName = 'ranger_ctl_hive_mask' + String dorisPolicyName = 'ranger_ctl_doris_policy' + + RangerClient rangerClient = new RangerClient("http://${rangerEndpoint}", "simple", rangerUser, rangerPassword, null) + + def dropPolicyQuietly = { String service, String name -> + try { + rangerClient.deletePolicy(service, name) + } catch (Exception e) { + logger.info("policy ${service}/${name} not found: ${e.getMessage()}") + } + } + + // Grants `user` `accesses` on `resources` of `service`, and waits for the plugin to pick it up. + def grant = { String service, String name, Map resources, + List accesses -> + dropPolicyQuietly(service, name) + RangerPolicy policy = new RangerPolicy() + policy.setService(service) + policy.setName(name) + policy.setResources(resources) + RangerPolicy.RangerPolicyItem item = new RangerPolicy.RangerPolicyItem() + item.setUsers([user]) + item.setAccesses(accesses.collect { new RangerPolicy.RangerPolicyItemAccess(it) }) + policy.setPolicyItems([item]) + logger.info("created policy ${name} with id ${rangerClient.createPolicy(policy).getId()}") + waitPolicyEffect() + } + + // ---- the Ranger service the bound catalog answers to ---- + // Its default policies grant the configured user everything, which is what lets root build the + // fixture below through a catalog this service governs. + try { + rangerClient.getService(HIVE_SERVICE_NAME) + logger.info("ranger service ${HIVE_SERVICE_NAME} already exists") + } catch (Exception e) { + logger.info("creating ranger service ${HIVE_SERVICE_NAME}: ${e.getMessage()}") + RangerService service = new RangerService() + service.setName(HIVE_SERVICE_NAME) + service.setType('hive') + service.setDescription('Doris catalog level authorization regression service') + // jdbc.* are read only by the admin UI's resource lookup, but the service definition requires + // them, so a service cannot be created without values here. + service.setConfigs([ + 'username' : 'root', + 'password' : 'root', + 'jdbc.driverClassName': 'org.apache.hive.jdbc.HiveDriver', + 'jdbc.url' : "jdbc:hive2://${externalEnvIp}:10000".toString() + ]) + rangerClient.createService(service) + } + + // ---- fixture ---- + // Built through a catalog with no source of its own, so root's grants in the instance wide + // service cover all of it. Both catalogs are the same metastore, so it is the same table. + sql """DROP CATALOG IF EXISTS ${plainCatalog}""" + sql """CREATE CATALOG ${plainCatalog} PROPERTIES ( + "type"="hms", + 'hive.metastore.uris' = 'thrift://${externalEnvIp}:${hmsPort}' + )""" + sql """DROP DATABASE IF EXISTS ${plainCatalog}.${dbName} FORCE""" + sql """CREATE DATABASE ${plainCatalog}.${dbName}""" + sql """CREATE TABLE ${plainCatalog}.${dbName}.${tblName} ( + id BIGINT, + username VARCHAR(20) + ) ENGINE=hive PROPERTIES ('file_format'='parquet')""" + sql """INSERT INTO ${plainCatalog}.${dbName}.${tblName} VALUES + (1, 'alice'), (2, 'bob'), (3, 'carol'), (4, 'dave')""" + + sql """DROP CATALOG IF EXISTS ${boundCatalog}""" + sql """CREATE CATALOG ${boundCatalog} PROPERTIES ( + "type"="hms", + 'hive.metastore.uris' = 'thrift://${externalEnvIp}:${hmsPort}', + 'access_controller.class' = 'ranger-hive', + 'access_controller.properties.ranger.service.name' = '${HIVE_SERVICE_NAME}' + )""" + + sql """DROP USER IF EXISTS ${user}""" + sql """CREATE USER '${user}' IDENTIFIED BY '${pwd}'""" + + dropPolicyQuietly(HIVE_SERVICE_NAME, hivePolicyName) + dropPolicyQuietly(dorisServiceName, dorisPolicyName) + + Map hiveResources = [ + 'database': new RangerPolicy.RangerPolicyResource(dbName), + 'table' : new RangerPolicy.RangerPolicyResource(tblName), + 'column' : new RangerPolicy.RangerPolicyResource('*') + ] + // The plain catalog is in here so that the last two cases can read the same table through both + // authorities. It has no source of its own, so this grant covers all of it; the two bound ones + // route everything below the catalog elsewhere, which is what case 4 turns on. + Map dorisResources = [ + 'catalog': new RangerPolicy.RangerPolicyResource([boundCatalog, fqcnCatalog, plainCatalog], + false, false) + ] + List dorisAccesses = ["SELECT", "LOAD", "ALTER", "CREATE", "DROP", "SHOW_VIEW"] + + try { + // case 1: what is inside the catalog is the bound source's to answer, and it has been given + // nothing. The refusal names the table, in the shape a Hive service phrases it: no catalog, + // because a Hive service has no such scope. + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + test { + sql """SELECT * FROM ${boundCatalog}.${dbName}.${tblName}""" + exception "does not have privilege" + } + } + + // case 2: a policy in that service, and only in that service, opens the table. + // Hive access types are lower case; RangerHiveAccessController maps a Doris SELECT onto this. + grant(HIVE_SERVICE_NAME, hivePolicyName, hiveResources, ["select"]) + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + def rows = sql """SELECT * FROM ${boundCatalog}.${dbName}.${tblName}""" + assertEquals(4, rows.size()) + } + + // case 3: the catalog itself is a different question, and the bound source never sees it. + // Reading a qualified table above asked about the table alone; SWITCH asks about the catalog, + // which routes to the instance wide source, where nothing has been granted yet. + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + test { + sql """SWITCH ${boundCatalog}""" + exception "to catalog" + } + } + grant(dorisServiceName, dorisPolicyName, dorisResources, dorisAccesses) + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + sql """SWITCH ${boundCatalog}""" + } + + // case 4: and that catalog grant reaches no further than the catalog. It is still in force + // here, so the table closing again when the Hive policy goes away is the bound source's + // doing: the two services each answer their own half and neither covers for the other. + dropPolicyQuietly(HIVE_SERVICE_NAME, hivePolicyName) + waitPolicyEffect() + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + test { + sql """SELECT * FROM ${boundCatalog}.${dbName}.${tblName}""" + exception "does not have privilege" + } + } + + // case 5: the same source can be named by the class of the factory publishing it, which is + // what releases before the sources became plugins wrote, and what a catalog created then + // still has persisted. + grant(HIVE_SERVICE_NAME, hivePolicyName, hiveResources, ["select"]) + sql """DROP CATALOG IF EXISTS ${fqcnCatalog}""" + sql """CREATE CATALOG ${fqcnCatalog} PROPERTIES ( + "type"="hms", + 'hive.metastore.uris' = 'thrift://${externalEnvIp}:${hmsPort}', + 'access_controller.class' = '${FACTORY_CLASS}', + 'access_controller.properties.ranger.service.name' = '${HIVE_SERVICE_NAME}' + )""" + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + def rows = sql """SELECT * FROM ${fqcnCatalog}.${dbName}.${tblName}""" + assertEquals(4, rows.size()) + } + + // case 6: a row filter is decided by the same source as the read it applies to, so writing + // one in the bound catalog's service changes what that catalog returns and leaves the plain + // catalog, whose authority is the instance wide service, returning the whole table. Same + // metastore, same table, two answers. + RangerPolicy rowFilter = new RangerPolicy() + rowFilter.setService(HIVE_SERVICE_NAME) + rowFilter.setName(hiveRowFilterName) + rowFilter.setPolicyType(RangerPolicy.POLICY_TYPE_ROWFILTER) + // A Hive row filter is written against the table; a column resource is not part of that def. + rowFilter.setResources([ + 'database': new RangerPolicy.RangerPolicyResource(dbName), + 'table' : new RangerPolicy.RangerPolicyResource(tblName) + ]) + RangerPolicy.RangerRowFilterPolicyItem rowFilterItem = new RangerPolicy.RangerRowFilterPolicyItem() + rowFilterItem.setUsers([user]) + rowFilterItem.setAccesses([new RangerPolicy.RangerPolicyItemAccess("select")]) + rowFilterItem.setRowFilterInfo(new RangerPolicy.RangerPolicyItemRowFilterInfo("id >= 3")) + rowFilter.setRowFilterPolicyItems([rowFilterItem]) + dropPolicyQuietly(HIVE_SERVICE_NAME, hiveRowFilterName) + logger.info("created row filter policy id ${rangerClient.createPolicy(rowFilter).getId()}") + waitPolicyEffect() + + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + def filtered = sql """SELECT id FROM ${boundCatalog}.${dbName}.${tblName}""" + assertEquals(2, filtered.size()) + filtered.each { assertTrue(it[0] >= 3, "row ${it[0]} should have been filtered out") } + + def whole = sql """SELECT id FROM ${plainCatalog}.${dbName}.${tblName}""" + assertEquals(4, whole.size()) + } + dropPolicyQuietly(HIVE_SERVICE_NAME, hiveRowFilterName) + waitPolicyEffect() + + // case 7: and the same for a column mask. MASK_NULL is the one whose effect needs no + // agreement about string shapes: the column comes back null through the bound catalog and + // intact through the plain one. + RangerPolicy mask = new RangerPolicy() + mask.setService(HIVE_SERVICE_NAME) + mask.setName(hiveMaskName) + mask.setPolicyType(RangerPolicy.POLICY_TYPE_DATAMASK) + // A mask is written against one column, which is why Ranger evaluates masks a column at a time. + mask.setResources([ + 'database': new RangerPolicy.RangerPolicyResource(dbName), + 'table' : new RangerPolicy.RangerPolicyResource(tblName), + 'column' : new RangerPolicy.RangerPolicyResource('username') + ]) + RangerPolicy.RangerDataMaskPolicyItem maskItem = new RangerPolicy.RangerDataMaskPolicyItem() + maskItem.setUsers([user]) + maskItem.setAccesses([new RangerPolicy.RangerPolicyItemAccess("select")]) + maskItem.setDataMaskInfo(new RangerPolicy.RangerPolicyItemDataMaskInfo("MASK_NULL", "", "")) + mask.setDataMaskPolicyItems([maskItem]) + dropPolicyQuietly(HIVE_SERVICE_NAME, hiveMaskName) + logger.info("created data mask policy id ${rangerClient.createPolicy(mask).getId()}") + waitPolicyEffect() + + connect("${user}", "${pwd}", "${defaultJdbcUrl}") { + def masked = sql """SELECT username FROM ${boundCatalog}.${dbName}.${tblName}""" + assertEquals(4, masked.size()) + masked.each { assertTrue(it[0] == null, "username should have been masked, got ${it[0]}") } + + def clear = sql """SELECT username FROM ${plainCatalog}.${dbName}.${tblName}""" + assertEquals(4, clear.size()) + clear.each { assertTrue(it[0] != null, "username should not be masked here") } + } + } finally { + dropPolicyQuietly(HIVE_SERVICE_NAME, hivePolicyName) + dropPolicyQuietly(HIVE_SERVICE_NAME, hiveRowFilterName) + dropPolicyQuietly(HIVE_SERVICE_NAME, hiveMaskName) + dropPolicyQuietly(dorisServiceName, dorisPolicyName) + sql """DROP CATALOG IF EXISTS ${boundCatalog}""" + sql """DROP CATALOG IF EXISTS ${fqcnCatalog}""" + sql """DROP DATABASE IF EXISTS ${plainCatalog}.${dbName} FORCE""" + sql """DROP CATALOG IF EXISTS ${plainCatalog}""" + sql """DROP USER IF EXISTS ${user}""" + } +} From f1a93e5bb9b46a05c0d8d9c61cab61ed4b2bc4f6 Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 21:36:14 +0800 Subject: [PATCH 19/22] [fix](license) satisfy the header check for the new authorization resources license-eye flagged three files this branch adds: authorization-plugin-api-version.properties is a normal resource and just lacked the header - the authentication family's equivalent carries it, so this now matches. The other two are golden files a test reads back line by line, where a header would be parsed as content: authorization-plugin-surface.txt joins the four *-plugin-surface.txt baselines already listed in paths-ignore for exactly that reason, and access-control-behavior-baseline.txt is the privilege matrix AccessControlBehaviorBaselineTest regenerates and compares line for line. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ZKZiwpFJ7sQNJEAdFN11g --- .licenserc.yaml | 5 +++++ .../authorization-plugin-api-version.properties | 17 +++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/.licenserc.yaml b/.licenserc.yaml index f4d89c444a067f..8af6eab88804f3 100644 --- a/.licenserc.yaml +++ b/.licenserc.yaml @@ -58,7 +58,12 @@ header: - "fe/fe-connector/fe-connector-spi/src/test/resources/connector-plugin-surface.txt" - "fe/fe-filesystem/fe-filesystem-spi/src/test/resources/filesystem-plugin-surface.txt" - "fe/fe-authentication/fe-authentication-spi/src/test/resources/authentication-plugin-surface.txt" + - "fe/fe-authorization/fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt" - "fe/fe-core/src/test/resources/lineage-plugin-surface.txt" + # Golden access-control decision matrix. AccessControlBehaviorBaselineTest regenerates + # the whole file and compares it to this one line for line, so a header would read back + # as extra rows; it prints the regenerated path on mismatch instead. + - "fe/fe-core/src/test/resources/access-control-behavior-baseline.txt" # Connector plugin settings templates. build.sh seeds each connector's live # .conf from its template verbatim (cp -n), so the template's content IS # the file an administrator edits in plugins/connector/

/. Matched by name diff --git a/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties b/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties index 20541eeeaca6d1..8c59eb148d898e 100644 --- a/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties +++ b/fe/fe-authorization/fe-authorization-spi/src/main/resources-filtered/META-INF/doris/authorization-plugin-api-version.properties @@ -1,3 +1,20 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + # The authorization plugin API version this FE serves, filtered from # in fe/fe-authorization/pom.xml at build time. # From 0c9bf9964e1dc23cb65442a944ce9bb6da91ba92 Mon Sep 17 00:00:00 2001 From: morningman Date: Sun, 16 Aug 2026 21:36:46 +0800 Subject: [PATCH 20/22] [fix](license) let the dependency gate accept the Jersey client Ranger needs Dependency License Review rejects com.sun.jersey:jersey-client@1.19.4, whose CDDL-1.1 OR GPL-2.0 WITH Classpath-exception-2.0 is not on the allow-licenses list. There is no placement that avoids it: the check reads every changed pom, so declaring the version in the plugin instead of fe/pom.xml only moves which file trips, and javax.ws.rs:jsr311-api - the other half of what RangerAdminRESTClient needs - is CDDL too. Exempting the package is the honest resolution rather than a workaround. CDDL-1.1 is ASF Category B, dist/LICENSE-dist.txt has listed com.sun.jersey:jersey-client under "CDDL + GPLv2 with classpath exception" since before this branch, and the FE already ships jersey-core under the same dual license - fe/lib carries jersey-core, jersey-json, jersey-server and jersey-servlet today. Only the client half was missing, because #37575 excluded the jersey-bundle Ranger declares against. Follows the shape of the allow-ghsas entry above it: a named exemption with the reason recorded next to it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ZKZiwpFJ7sQNJEAdFN11g --- .github/workflows/third_party_review.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/workflows/third_party_review.yml b/.github/workflows/third_party_review.yml index 226706c13e6b65..f08ca37a365751 100644 --- a/.github/workflows/third_party_review.yml +++ b/.github/workflows/third_party_review.yml @@ -37,6 +37,15 @@ jobs: # ([String]). Only allow these licenses (optional) # Possible values: Any SPDX-compliant license identifiers or expressions from https://spdx.org/licenses/ allow-licenses: BSD-2-Clause, BSD-3-Clause, MIT, Apache-2.0, EPL-2.0, MPL-2.0, CC0-1.0 + # ([String]). Packages exempted from the license check above, as PURLs (optional) + # com.sun.jersey:jersey-client (CDDL-1.1 OR GPL-2.0 WITH Classpath-exception-2.0): + # CDDL-1.1 is ASF Category B, usable in binary form as long as the distribution records + # it, and dist/LICENSE-dist.txt already lists com.sun.jersey:jersey-client under + # "CDDL + GPLv2 with classpath exception". The FE also already ships jersey-core under + # that same dual license; only the client half is missing, because #37575 excluded the + # jersey-bundle Ranger declares against. Ranger's RangerAdminRESTClient is Jersey 1.x, + # so the Ranger authorization plugins need it to reach a Ranger admin at all. + allow-dependencies-licenses: pkg:maven/com.sun.jersey/jersey-client # ([String]). Acknowledged advisories that must not fail the review (optional) # org.codehaus.jackson:jackson-mapper-asl (GHSA-c27h-mcmw-48hv, GHSA-r6j9-8759-g62w): # legacy Jackson 1.x is EOL and neither advisory has a fixed version. Hive's metastore From ef177f3eff6ef50f5a689b29063afc9f54735589 Mon Sep 17 00:00:00 2001 From: morningman Date: Mon, 17 Aug 2026 09:16:52 +0800 Subject: [PATCH 21/22] [fix](authorization) ask about the database when a routine load names no table A multi table routine load carries no table name, and createRoutineLoadJob handed that null straight to checkTblPriv. That used to mean "decide this at the database level or above": the old checkTblPriv passed the name down to Role.checkTblPriv, which answers global || catalog || db || table and simply finds no table level grant when the name is absent. AuthorizedResource.Table rejects a null name, so the same call now dies with java.lang.NullPointerException: table is required and every CREATE ROUTINE LOAD without an ON clause fails. The suites that do this wrap the statement in try/finally and stop the job on the way out, so the stop failure replaced the real one and the P0 failures read as six unrelated "There is not operable routine load job with name ..." errors. Ask about the database instead when the job is multi table. Role.checkDbPriv is global || catalog || db, which is exactly what the null table name reduced to, so the decision is unchanged - including the one test_dml_multi_routine_load_auth pins, where table level grants stay denied and load_priv on db.* is what lets the job through. This is also the rule checkPrivAndGetJob already applies to the same job later on; only the create path was left behind. ShowCreateRoutineLoadCommand reads the table name off the job itself, where a multi table job likewise answers null, and would have failed the same way as soon as a multi table job existed in the database. Fixed alongside. Co-Authored-By: Claude Opus 5 (1M context) --- .../load/routineload/RoutineLoadManager.java | 15 ++++++++++++++- .../load/ShowCreateRoutineLoadCommand.java | 9 ++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java index df5615016bfa54..f0099aa9a2bd9a 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java @@ -181,7 +181,20 @@ private Map getBeCurrentTasksNumMap() { public void createRoutineLoadJob(CreateRoutineLoadInfo info, ConnectContext ctx) throws UserException { // check load auth - if (!Env.getCurrentEnv().getAccessManager().checkTblPriv(ConnectContext.get(), + if (info.isMultiTable()) { + // A multi table job names no table at all, so LOAD has to be held on the database or above. + // This is the same rule the job is checked against later on, in checkPrivAndGetJob(). + if (!Env.getCurrentEnv().getAccessManager().checkDbPriv(ConnectContext.get(), + InternalCatalog.INTERNAL_CATALOG_NAME, + info.getDBName(), + PrivPredicate.LOAD)) { + // todo add new error code + ErrorReport.reportAnalysisException(ErrorCode.ERR_TABLEACCESS_DENIED_ERROR, "LOAD", + ConnectContext.get().getQualifiedUser(), + ConnectContext.get().getRemoteIP(), + info.getDBName()); + } + } else if (!Env.getCurrentEnv().getAccessManager().checkTblPriv(ConnectContext.get(), InternalCatalog.INTERNAL_CATALOG_NAME, info.getDBName(), info.getTableName(), diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/load/ShowCreateRoutineLoadCommand.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/load/ShowCreateRoutineLoadCommand.java index f3cfa96d7f75d8..83e9b898c08dee 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/load/ShowCreateRoutineLoadCommand.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/load/ShowCreateRoutineLoadCommand.java @@ -114,7 +114,14 @@ private ShowResultSet handleShowCreateRoutineLoad() throws AnalysisException { .add("error_msg", "The table name for this routine load does not exist") .build(), e); } - if (!Env.getCurrentEnv().getAccessManager() + // A multi table job names no table, so LOAD has to be held on the database or above. + if (job.isMultiTable()) { + if (!Env.getCurrentEnv().getAccessManager() + .checkDbPriv(ConnectContext.get(), InternalCatalog.INTERNAL_CATALOG_NAME, dbName, + PrivPredicate.LOAD)) { + continue; + } + } else if (!Env.getCurrentEnv().getAccessManager() .checkTblPriv(ConnectContext.get(), InternalCatalog.INTERNAL_CATALOG_NAME, dbName, tableName, PrivPredicate.LOAD)) { continue; From 83dd084918e561df735045dcb3d21fe637c14b99 Mon Sep 17 00:00:00 2001 From: morningman Date: Mon, 17 Aug 2026 10:05:12 +0800 Subject: [PATCH 22/22] [doc](authorization) add a developer guide for the authorization plugin framework ### What problem does this PR solve? Issue Number: None Problem Summary: fe/fe-authorization/ has no directory-level guide. The contract's own README (fe-authorization-spi/README.md) tells a plugin author what to implement and how to install it, but nothing describes the framework around it: which module owns what, how the FE discovers a source and routes a check to it, what the classloading rules are, and which obligations a change to the contract carries. Anyone adding an authorization source has to reconstruct all of that from AccessControllerManager and the poms - the same gap fe-connector closed with its README.md / AGENTS.md pair. This adds that pair, in the same shape: - README.md: the design rules, the module map (including why api and spi are two modules here where fe-connector has one), how a source is discovered, selected, routed to, built, classloaded and packaged, how to read the API, and a step-by-step walkthrough for adding a new source. - AGENTS.md: build and test recipes, the machine-checked obligations (the frozen surface baseline and the major bump it forces, the behaviour baseline, the selector literals, the license gates) and the invariants no gate expresses. Two recipes are recorded from what this tree actually does rather than copied from fe-connector's: - `install` on a plugin module fails while the build cache is on, at "fe-extension-spi: The packaging for this project did not assign a file to the build artifact", because a restored module has no artifact file to install. `package` is the recipe. - the build cache key does not include -DskipTests, so a `package` run right after a -DskipTests one restores the module and reports SUCCESS with its test classes never executed. fe/AGENTS.md gets a pointer to the new pair next to the connector one, and the spi README a line pointing up at it. ### Release note None ### Check List (For Author) - Test: No need to test (documentation only). The recipes the new AGENTS.md documents were each run against this tree: the fe-authorization-spi surface test passes; `-pl :fe-authorization-plugin-ranger-doris -am package -Dmaven.build.cache.enabled=false` builds the plugin zip and runs the module's 15 tests; and the `install` failure documented above was reproduced. - Behavior changed: No - Does this need documentation: No (this is documentation) Co-Authored-By: Claude Opus 5 (1M context) --- fe/AGENTS.md | 8 + fe/fe-authorization/AGENTS.md | 202 +++++++++ fe/fe-authorization/README.md | 383 ++++++++++++++++++ .../fe-authorization-spi/README.md | 6 + 4 files changed, 599 insertions(+) create mode 100644 fe/fe-authorization/AGENTS.md create mode 100644 fe/fe-authorization/README.md diff --git a/fe/AGENTS.md b/fe/AGENTS.md index eab55873cc8dc0..5a31a4f1f83255 100644 --- a/fe/AGENTS.md +++ b/fe/AGENTS.md @@ -25,3 +25,11 @@ cd fe && mvn clean install -DskipTests -Dskip.doc=true -T 1C For work under `fe-connector/` (connector plugins / external catalogs), start with `fe-connector/README.md` (architecture, adding a new connector) and `fe-connector/AGENTS.md` (build/test recipes, gates, invariants). + +## Authorization Framework + +For work under `fe-authorization/` (authorization sources / access control), +start with `fe-authorization/README.md` (architecture, adding a new +authorization plugin) and `fe-authorization/AGENTS.md` (build/test recipes, +obligations, invariants). `fe-authorization/fe-authorization-spi/README.md` is +the plugin author's quickstart. diff --git a/fe/fe-authorization/AGENTS.md b/fe/fe-authorization/AGENTS.md new file mode 100644 index 00000000000000..c8e8b781ffb4dd --- /dev/null +++ b/fe/fe-authorization/AGENTS.md @@ -0,0 +1,202 @@ +# AGENTS.md — fe-authorization + +Architecture, the module map and the new-plugin walkthrough live in `README.md` +next to this file — read that first. This file is operational: how to build and +test, and which obligations and invariants you must not break. + +## Build and Test Recipes + +Run from the repository root; `-am` also builds upstream reactor deps. + +```bash +# One module and its tests +mvn -f fe/pom.xml -pl :fe-authorization-spi -am test + +# A single test class. Keep -DfailIfNoTests=false: with -am, upstream modules +# have no matching tests and would fail the build otherwise. +mvn -f fe/pom.xml -pl :fe-authorization-spi -am test \ + -Dtest=AuthorizationPluginSurfaceTest -DfailIfNoTests=false + +# A plugin module: package, NOT test. The plugin zip binds to the package +# phase, so a test-level run skips what actually ships. Disable the build cache +# or the module is restored wholesale and its tests never run - see below. +mvn -f fe/pom.xml -pl :fe-authorization-plugin-ranger-doris -am package \ + -Dmaven.build.cache.enabled=false + +# The engine side: routing, the behaviour baseline, the installed-plugin e2e +mvn -f fe/pom.xml -pl :fe-core -am test \ + -Dtest=AccessControlBehaviorBaselineTest -DfailIfNoTests=false +``` + +Three things about the build cache (`fe/.mvn/maven-build-cache-config.xml`), +each of which produces a green build that proved nothing: + +- **A restored module does not run its tests.** The cache key does not include + `-DskipTests`, so a `package` run right after a `-DskipTests` one restores the + module and reports SUCCESS with its test classes never executed. Pass + `-Dmaven.build.cache.enabled=false` whenever the point of the run is the + tests. +- **`install` fails outright while the cache is on**, at + `fe-extension-spi: The packaging for this project did not assign a file to the + build artifact` — a restored module has no artifact file to install. Use + `package`, or `install -Dmaven.build.cache.enabled=false`. +- It is **mandatory** to disable the cache when you bump the plugin API + version — see obligation 2 — and worth disabling when validating deletions or + refactors, where a cached artifact can mask a stale one. + +And three about the modules themselves: + +- fe-core carries `fe-authorization-plugin-ranger-doris` as a **test** + dependency (the behaviour baseline runs the production controller), so a + breaking change in that plugin fails fe-core's tests, not only the plugin's. + Build order is therefore api → spi → ranger-common → ranger-doris → fe-core. +- The api and spi modules carry no mocking framework: the parent pom + contributes JUnit only, and the contract tests use hand-written fakes. Keep it + that way — what `AuthorizationPluginContractTest` proves is what the SPI's + *default* methods do for a real implementation, and a mock of + `AuthorizationPlugin` would stub out the very defaults under test. Mockito + (`mockito-inline`, needed for `mockConstruction`) is declared per plugin + module, where the thing being faked is a Ranger policy engine. +- Checkstyle is part of the build (`validate` phase) and scans test sources too. + +## Machine-Checked Obligations + +1. **The frozen plugin API surface.** `AuthorizationPluginSurfaceTest` + (fe-authorization-spi) freezes + `src/test/resources/authorization-plugin-surface.txt`. Any drift is a MAJOR + change: the SAME commit must refresh the baseline (run the test, copy the + "actual" block from the failure message) **and** increment the major of + `` in `fe/fe-authorization/pom.xml`, + zeroing its minor. Additions count. One more `ResourceKind` or + `AccessAction` constant turns every deployed plugin's "a kind I do not + recognise" branch — which the contract requires to be a refusal — into a + denial of something that used to be allowed, in plugins nobody rebuilt. + The frozen set is a computed closure over everything a plugin can see, so a + change to `fe-authorization-api` moves it exactly as much as one to the spi. + A change to `fe-extension-spi` turns all five families' baselines red at once + and means bumping all five. +2. **The version property and the build cache.** The version reaches a jar + through a filtered resource whose *source text* is the literal `${...}` + placeholder, and maven-build-cache-extension (enabled in `fe/.mvn`) hashes a + module from `src/**`, its dependencies and its `` — never + from ``. The `maven-jar-plugin` `` block in + `fe/fe-authorization/pom.xml` exists solely to put the value inside + ``; do not remove it. Verify any bump with + `-Dmaven.build.cache.enabled=false`, or the cached jar ships the old number + in `META-INF/doris/authorization-plugin-api-version.properties` and the FE + serves a contract nobody declared. This was reproduced in + `fe/fe-authentication/pom.xml`. +3. **The behaviour baseline.** `AccessControlBehaviorBaselineTest` (fe-core) + records every decision the manager makes over (resource kind × action × + source × caller) into + `fe/fe-core/src/test/resources/access-control-behavior-baseline.txt`. After + a change meant to be structural, `git diff` on that file must be empty. A + line that does change is a behaviour change: read each changed cell, then + say in the PR description what now decides differently and why. Never + regenerate it to make a build green. +4. **The selectors a source is named by.** `AuthorizationSourceSelectorCompatibilityTest` + (fe-core) and each plugin's own factory test pin, as literals, every string + an operator may select a source by. `access_controller.class` is persisted + with the catalog and read back verbatim by later releases, so a value that + ever worked has to keep working. Moving or renaming a factory class means + adding the old fully-qualified name to `SOURCES_THAT_LEFT_THE_KERNEL` in + `AccessControllerManager` in the same commit. +5. **Per-family version wiring.** `PluginApiVersionWiringTest` (fe-core) proves + this family passes a gate at all, that the gate is built from its own kernel + resource, and that its version moves independently of the other families'. + What no test can check is that the `` element name in the + pom equals the attribute name the gate derives — those are pinned as + literals there, to be read against the pom in review. +6. **License gates (CI).** + - The ASF header check (`license-eye`, `.licenserc.yaml`) runs on every new + file. Golden files a test reads back line by line must stay header-free and + be listed in `paths-ignore` instead — + `authorization-plugin-surface.txt` and + `access-control-behavior-baseline.txt` are there for that reason. + - `Dependency License Review` + (`.github/workflows/third_party_review.yml`) reads every changed pom and + rejects anything outside `allow-licenses`. Moving a declaration between + poms does not avoid it. A Category-B dependency needs a named + `allow-dependencies-licenses` PURL with the reason recorded next to it, and + `dist/LICENSE-dist.txt` must already cover it — `com.sun.jersey:jersey-client` + is the worked precedent. + +## Invariants Without a Gate + +Guarded by tests and reviewed comments rather than build gates. Every one has a +concrete failure mode. + +- **Routing stays one function.** `AccessControllerManager.controllerOf` is the + whole of it. No source-name branching anywhere else, no second opinion, no + privilege established before a source is asked. Combining two sources or + granting first would have to happen there, and deliberately does not — the + behaviour baseline is what would catch it. +- **A source installed instance-wide answers for administration itself.** + `grantedByGlobalScopeAuthority` answers false when the asking source *is* the + instance-scope authority, on purpose: it would otherwise ask itself a question + it is about to answer. A source without an administration rule of its own + makes the FE unadministrable from its first statement. +- **The api and spi jars are never bundled in a plugin.** `provided` scope, and + absent from the zip. `org.apache.doris.authorization.` is parent-first, so a + bundled copy is a second set of types the engine refuses to recognise as the + ones it asked for. +- **TCCL is pinned for the factory call.** `AccessControllerManager.create` + swaps the thread context classloader to the plugin's own for the duration. + Without it, a bundled library resolving class names through the TCCL (Hadoop's + `Configuration`, which both Ranger sources drag in) loads half its classes + from the engine's copy and startup dies with "class X not Y". Library worker + threads created inside the call inherit the loader, which is what keeps them + working afterwards. +- **A refusal stays cheap.** `AccessDeniedException` is built with no stack + trace, no cause and no suppression, and composes its message lazily; it is + thrown once per object a user may not see. Do not let it acquire a stack + trace, do not wrap it in an exception that fills one in, and do not build its + message eagerly to log it. +- **`policyIdent` carries a version token.** `RowFilterSpec` and `DataMaskSpec` + are values with real equality because the SQL result cache compares them to + detect a policy change. An ident that does not move when the policy is edited + in place makes a stale plan look current; specs that do not compare equal when + identical evict the cache on every lookup. +- **Security-relevant flags parse strictly.** A property that switches an + exemption on or off must reject anything that is not exactly its allowed + values. Reading a typo leniently silently changes who may reach what — see + `RangerAccessController.DEFER_TO_GLOBAL_SCOPE_AUTHORITY`. +- **One service descriptor per plugin directory.** The loader admits exactly one + factory per directory; a second descriptor is a load failure, not a second + source. That is why `fe-authorization-plugin-ranger-common` publishes none and + ships only into both plugins' `lib/`. +- **`checkAction` is not an engine entry point.** The engine always asks about a + whole requirement, so `checkAction` is only ever reached through the default + `checkPrivilege`. A source implements one or the other; do not add a caller + that reaches `checkAction` directly, or a source that answers whole + requirements suddenly has a method it never meant to provide on its path. +- **The two `build.sh` module lists stay identical.** The build list (`_authz_mod`) + and the deploy list (`AUTHZ_PLUGIN_DIR`). The deploy step unzips whatever + archive is left in a module's `target/`, so a divergence ships a stale plugin + without failing anything. + +## Task Recipes + +- **Change the SPI or the api surface**: edit → run fe-authorization-spi's own + suite → refresh `authorization-plugin-surface.txt` and bump the major in the + same commit → adjust the in-tree sources (`ranger-doris`, `ranger-hive`, + `InternalAuthorizationPlugin`, `LegacyAccessControllerPlugin`, the example in + fe-core's test tree) → run fe-core's authorization tests, including the + behaviour baseline. +- **Add an authorization plugin**: follow README "Adding a New Authorization + Plugin" top to bottom; the obligations and invariants above apply from the + first commit. +- **Fix a plugin bug**: module-scoped `package` with the cache off → fe-core's + behaviour baseline if the fix changes a decision → a regression suite under + `regression-test/suites/ranger_p2/` for anything only visible against a live + Ranger. +- **Change routing or the manager**: fe-core's tests are the gate here — + `AccessControlBehaviorBaselineTest` for what is decided, + `AuthorizationPluginFromDirectoryTest` for what an installed plugin can + actually do, `AuthorizationSourceSelectorCompatibilityTest` for what an + upgraded deployment still resolves. + +## Commit Conventions + +See the repository-root `AGENTS.md` for commit-message format, the PR template +and code-review checkpoints. Nothing here overrides it. diff --git a/fe/fe-authorization/README.md b/fe/fe-authorization/README.md new file mode 100644 index 00000000000000..ec59c461ad4680 --- /dev/null +++ b/fe/fe-authorization/README.md @@ -0,0 +1,383 @@ +# fe-authorization Developer Guide + +This directory holds the authorization plugin framework of the Doris FE: the +contract an authorization source implements, the vocabulary it decides with, +and the sources shipped with the release. An *authorization source* decides, +for the resources it governs, what a user may do with them — without fe-core +knowing anything about how it decides. + +Three companion documents divide the work with this one: + +- `AGENTS.md` (next to this file) — build/test recipes, the machine-checked + obligations, and the invariants that are not expressible as a gate. Read it + before changing code here. +- `fe-authorization-spi/README.md` — the plugin author's quickstart: a minimal + plugin end to end, the `META-INF/services` line, the installed directory + layout, and the API-version manifest entry. Copy from there; this file + explains the framework the copy lands in. +- Generic plugin machinery is NOT defined here: contracts live in + `fe/fe-extension-spi` and directory loading in `fe/fe-extension-loader`. Each + has its own README. + +## What This Is + +Four design rules shape everything in this directory: + +1. **One source answers, and its answer is the whole answer.** Which source is + asked follows from the resource alone: the plugin a catalog is bound to + answers for everything inside that catalog, the plugin installed for the + instance answers for everything else. Nothing grants access before a plugin + is asked and no second plugin is consulted after it. `AccessControllerManager` + (fe-core) is pure routing — it establishes no privilege of its own and never + combines two verdicts — which is what makes the policies in force on an + object readable from the configuration. +2. **Refusing is throwing, and silence refuses.** A check that returns has + allowed the access; a check that refuses throws `AccessDeniedException`. + There is no third outcome and no boolean for a caller to ignore. Every check + method defaults to refusing, so an omission costs you access control you did + not think about, never a hole. The two data-policy methods are the exception: + their empty default means "this source defines no policy", which is not the + same as allowing anything. +3. **A plugin never imports fe-core.** It compiles against + `fe-authorization-api` and `fe-authorization-spi` and nothing else of Doris — + the plugin modules simply do not have fe-core on their classpath, which is a + stronger guarantee than a gate. What a source cannot decide alone it asks the + engine for through `AuthorizationContext` (Doris roles, instance-scope + authority, ownership). +4. **The engine adds no caching and grants no exemptions.** It cannot know when + an external policy changed, so whatever caching a source needs belongs inside + the plugin, where it can be invalidated on that source's own terms. And + exemptions that used to be the engine's — "an administrator may go anywhere" — + are each plugin's own to grant or refuse. + +## Module Map + +Roles only — one line each. For anything deeper, read the javadoc; it is kept +authoritative. + +**Contracts** (both loaded from the FE, never from a plugin jar — see +"Classloading") + +| Module | Role | +|---|---| +| `fe-authorization-api` | The decision vocabulary shared by the engine and every source: `AuthorizedSubject`, `AuthorizedResource` (a closed hierarchy) with `ResourceKind`, `AccessAction`, `AccessRequirement` / `ActionMatch` and the named `AccessRequirements`, `AccessContext`, `AccessDeniedException`, and the data-policy payloads `RowFilterSpec` / `DataMaskSpec` / `RowFilterMergeType`. No dependencies at all. | +| `fe-authorization-spi` | The contract itself: `AuthorizationPluginFactory` (what a jar publishes), `AuthorizationPlugin` (the decisions), `AuthorizationContext` (what the engine answers when a source asks it something). Depends on the api and on `fe-extension-spi`. Its javadoc **is** the API reference. | + +Two modules where fe-connector has one, because here the split is acyclic: a +plugin decides with api types and implements spi interfaces, and the api never +mentions the spi. (fe-connector's boundary is bidirectional, which is why +splitting it by "who implements" would be circular there.) + +**Plugins** — one module per source, each installed from its own plugin +directory. + +| Module | Role | +|---|---| +| `fe-authorization-plugin-ranger-common` | What the Ranger sources share: asking a Ranger policy engine and reading its answer, plus the row-filter/data-mask translation. A library, not a plugin — it publishes no service descriptor and ships no zip, so it can sit in both plugins' `lib/` without either directory appearing to publish two sources. | +| `fe-authorization-plugin-ranger-doris` | A Ranger service of type `doris`. Answers about every resource kind, which is why it is the one Ranger source installable for a whole instance (`access_controller_type = ranger-doris`). | +| `fe-authorization-plugin-ranger-hive` | A Ranger service of type `hive`, for one external catalog (`"access_controller.class" = "ranger-hive"`). Also carries the audit handler. | + +One source per module is not a style choice: `DirectoryPluginRuntimeManager` +admits exactly one factory per plugin directory, so two service descriptors +under one directory is a load failure rather than a pair of sources. + +**Not in this directory, but part of the picture:** the built-in privilege model +is `InternalAuthorizationPlugin` (fe-core, name `default`), an authorization +source like any other and what `access_controller_type` defaults to. It lives +with `Auth` because it *is* `Auth`'s front door. Read it as the reference for +how a source that governs everything answers the whole contract. + +## How an Authorization Source Runs + +**Startup.** `Env` builds `AccessControllerManager` (fe-core, +`org.apache.doris.mysql.privilege`), which discovers factories in two rounds: +first `ServiceLoader` on the classpath (built-ins and tests), then +`DirectoryPluginRuntimeManager` over the roots named by +`Config.authorization_plugins_dir` (production). A classpath factory keeps its +name against a directory one, so dropping a jar into the plugin directory can +never displace a source shipped with the FE. Directory plugins pass an API +version gate; classpath ones deliberately do not — what is on the classpath was +built from this same source tree, so the version there would be a number +compared against itself. A directory that fails is logged and skipped, because +one unusable plugin must not stop an FE from starting; if the failed one is the +source the configuration names, the manager's constructor refuses right +afterwards with the rejection reason appended to the error. + +There is also a deprecated channel: `AccessControllerFactory` (fe-core), read +from loose jars at the *root* of the same directory. It is still loaded and +wrapped in `LegacyAccessControllerPlugin`, and a name published both ways +resolves to the newer publication — but it carries no declared API version, so a +plugin built against an older Doris is admitted with no diagnosis. Do not write +new sources against it. + +**Selection.** Two channels, both naming the source by the string +`AuthorizationPluginFactory.name()` returns: + +| Where | Key | Meaning | +|---|---|---| +| `fe.conf` | `access_controller_type` | The source governing the instance. Defaults to `default`, the built-in privilege model. | +| `fe.conf` | `authorization_plugins_dir` | Plugin roots, comma-separated. Default `${DORIS_HOME}/plugins/authorization`. | +| `fe.conf` | `authorization_config_file_path` | Properties handed to the instance-wide source, as a flat properties file. Default `/conf/authorization.conf`, resolved under `DORIS_HOME`. | +| catalog property | `access_controller.class` | The source governing that one external catalog. | +| catalog property | `access_controller.properties.` | Properties handed to that source, with the prefix stripped. | + +`access_controller.class` is persisted with the catalog and read back verbatim +by every later FE, so it also accepts the *class name* of the publishing +factory, and `AccessControllerManager.SOURCES_THAT_LEFT_THE_KERNEL` keeps +working the class names of sources that have since moved out of fe-core. This +is why renaming a factory class is a compatibility event; see AGENTS.md. + +**Routing a check.** Every access decision in fe-core funnels through +`AccessControllerManager.decide` (yes/no) or `decideColumns` (which reports the +column that was refused), and both route through `controllerOf(resource)`: +global, resource, workload group, storage vault, the cloud kinds — and +*catalog-level* grants, which only the instance-wide source ever stores — go to +the source installed for the instance; database, table and columns go to the +source their catalog is bound to. Row filters and column masks +(`evalRowFilterPolicies`, `evalDataMaskPolicy`) route the same way. + +**Lifecycle.** A source is created once and kept: unlike an authentication +attempt, an authorization decision happens many times within a single statement, +so a source that caches policies has to be the same instance throughout. The +engine builds a new one only when what configures it changes, and calls +`close()` on the one it replaces (also on catalog DROP or reset). Note that +`Plugin.initialize(PluginContext)` is **never called** for this family: +everything a source needs arrives in `create(properties, context)`, which is +also the only moment the `AuthorizationContext` can be handed over — the context +has to name the source it belongs to, and the source does not exist until its +factory has run. + +**Cost.** These methods are on the path of every statement, several times over: +planning one query checks each table it reads, and listing what a user may see +checks every object that exists. That shapes the contract in two visible places — +requirements are asked about as a whole rather than one action at a time, and +`AccessDeniedException` records no stack trace and composes its message only if +somebody reads it. + +**Classloading.** Each plugin directory gets its own child-first classloader. +`org.apache.doris.authorization.` is parent-first for this family, so the api +and spi types crossing the boundary exist exactly once — a plugin carrying its +own copy would hand back objects the engine refuses to recognise as the types it +asked for. Keep both jars `provided` and out of your zip. The +loader's mandatory parent-first prefixes apply on top and are additive: +`java.`, `javax.`, `sun.`, `com.sun.`, `org.slf4j.`, `org.apache.logging.`, +`org.apache.doris.extension.spi.`, `org.apache.doris.connector.spi.`. Two +consequences worth knowing before debugging a `ClassCastException` here: + +- `org.apache.hadoop.` is **child-first** for this family (unlike the filesystem + and connector families), so a plugin bundling Hadoop gets its own copy. +- `com.sun.` being parent-first splits a bundled Jersey: the host's + `com.sun.jersey.core` wins, which is why fe-core declares `jersey-client` + itself *and* ranger-common bundles it as a `findClass` fallback. Neither side + alone is enough. + +The factory call itself runs with the thread context classloader pinned to the +plugin's own loader, because a bundled library that resolves class names through +the TCCL (Hadoop's `Configuration` is the recurring case) would otherwise load +half its classes from the engine's copy. Worker threads such a library starts +inherit that loader, which is what keeps them working after the swap is undone. + +**Packaging.** Each plugin module assembles +`target/doris-fe-authorization-.zip` via +`src/main/assembly/plugin-zip.xml`: the module jar at the zip root — the only +place scanned for the service descriptor — and everything else under `lib/`. +`build.sh` unzips each into its own subdirectory of +`output/fe/plugins/authorization/`, which is also where an administrator drops a +third-party source. That subdirectory's name is free: the loader takes the +source's name from the factory, never from the directory, so the two need not +match (they do for the sources shipped here, and keeping them equal is kind to +whoever reads the deployment). + +## Reading the API + +This document does not list SPI methods. The truth lives in code, behind four +mechanisms: + +1. **Javadoc is the API reference.** Start at `AuthorizationPlugin` and + `AuthorizationPluginFactory`, then `AuthorizationContext`. Every method that + decides has a default that refuses, so each one's javadoc states when a + source is expected to override it and what the default does instead. +2. **The recorded surface.** + `fe-authorization-spi/src/test/resources/authorization-plugin-surface.txt` + freezes everything a plugin can see — not just the SPI interfaces but the + whole `fe-authorization-api` vocabulary a plugin *decides* with, computed as + a closure rather than hand-listed. `AuthorizationPluginSurfaceTest` fails on + any drift. That file is the API inventory; do not maintain one in prose. +3. **Named requirements, but decisions from the action set.** + `AccessRequirements` names the questions the engine asks — `VISIBILITY`, + `SELECT`, `ADMINISTRATION`, `ANY_PRIVILEGE`, … — as values, so a source can + recognise which one it is being asked. Recognising them is optional and + matching on them exclusively is a bug: requirements are also composed at run + time (granting a privilege requires holding it *and* the right to grant it), + and a source that only answers the named ones looks, from outside, like a + source with a mysteriously incomplete policy. Decide from + `requirement.getActions()` and `requirement.isSatisfiedBy(granted)`. +4. **Unknown is refusal.** `ResourceKind` is closed and known when a plugin is + compiled, and the API version gate admits only matching majors, so a kind a + source does not recognise means the plugin was built against a different + Doris — refuse it, never guess. A kind that *exists* but this source does not + govern is likewise a refusal, not an error. + +The worked example is +`fe/fe-core/src/test/java/org/apache/doris/authorizationexample/`: a factory, a +plugin that grants by Doris role with one row filter, and a test that installs +it from a plugin directory and puts SQL through it. It is the shortest thing +that answers the questions a first plugin runs into. + +## Adding a New Authorization Plugin + +Copy from the reference implementations: **`fe-authorization-plugin-ranger-doris`** +exercises the whole framework (instance-wide scope, every resource kind, row +filters and masks, a shared library module, a plugin zip); +`ExampleAuthorizationPlugin` in fe-core's test tree is the minimal contrast; +`InternalAuthorizationPlugin` (fe-core) shows how a source that governs +everything answers the whole contract. + +Steps 2, 12 and 13 are for a source shipped in this repository. A third-party +source needs none of them — only the two jars, the service descriptor and the +manifest entry from `fe-authorization-spi/README.md`. + +1. **Decide the scope first**, because it decides what you must answer. A + catalog-bound source only ever sees `DATABASE`, `TABLE`, `COLUMNS` and the + data policies on tables. An instance-wide source is asked about every kind, + including `GLOBAL`, `RESOURCE`, `WORKLOAD_GROUP`, `STORAGE_VAULT` and the + `CLOUD_*` kinds — **and it answers for administration itself.** + `AuthorizationContext.grantedByGlobalScopeAuthority` returns false when the + source asking *is* the instance-scope authority, so an instance-wide source + with no administration rule of its own locks out every account, including the + one that would fix the configuration. +2. **Module.** Create + `fe-authorization-plugins/fe-authorization-plugin-/` and register it in + `fe-authorization-plugins/pom.xml` ``. Start the pom from + ranger-doris: keep `fe-authorization-api` and `fe-authorization-spi` at + `provided`, set `doris-fe-authorization-`, + and bind `maven-assembly-plugin` to the `package` phase. +3. **Name.** Whatever `AuthorizationPluginFactory.name()` returns is the whole + selector — the value of `access_controller_type` and of + `access_controller.class`. Make it globally unique across sources, and note + that it is matched **case-sensitively** (`default` is the one exception, + compared case-insensitively because it predates all of this). It is persisted + in catalog properties, so renaming it later is a compatibility event; pin it + and the factory's class name as literals in a test (copy + `RangerDorisAccessControllerFactoryTest#testTheSelectorsThisSourceIsNamedBy`). +4. **Factory.** Implement `AuthorizationPluginFactory` with a no-arg + constructor, and register it in + `src/main/resources/META-INF/services/org.apache.doris.authorization.spi.AuthorizationPluginFactory` — + exactly one class, and exactly one such descriptor per plugin directory. + Do not implement the no-arg `create()`; the SPI default already refuses it + with the right message. If your source starts background threads (a policy + refresher, say), decide explicitly whether a second binding gets a second + instance — ranger-doris returns a singleton and warns that the later + properties are ignored. +5. **Plugin.** Implement `AuthorizationPlugin`. Override `checkPrivilege` when + your source can answer about a set of actions in one pass — a bit set, or a + walk down a resource hierarchy that remembers what an outer level already + granted. Override `checkAction` alone when it cannot, and let the SPI default + take the requirement apart; leaving the other at its default is not a hole, + because the engine only ever asks about a whole requirement. +6. **Refuse by throwing, and name yourself.** `AccessDeniedException.of(subject, + resource, requirement, name())` for the ordinary case; + `AccessDeniedException.withMessage(...)` where the wording *is* the answer, as + for a column check that must say which column failed. Naming the source is + what later lets an operator tell which of the configured sources said no. +7. **Deference to instance scope.** If the source is catalog-bound, decide + whether an administrator of the instance may reach what you govern, and + express it by calling `AuthorizationContext.grantedByGlobalScopeAuthority` + rather than by testing for a built-in privilege — the answer comes from + whoever actually governs instance scope, which may itself be a plugin. Make + it configurable if a deployment might want it off (see + `RangerAccessController.DEFER_TO_GLOBAL_SCOPE_AUTHORITY`), and parse that flag + strictly: reading a typo as `false` silently takes away every administrator's + access to every object you govern. +8. **Roles.** `AuthorizationContext.rolesOf` is the bridge for policies written + against Doris roles — the engine, not your source, knows who holds them. Ask + for them inside the check that needs them; they are deliberately not carried + on the subject, because listing what an account may see walks thousands of + objects per statement. +9. **Row filters and column masks.** Both are SQL text in Doris dialect; the + engine parses, type-checks and plans them. Give every spec a `policyIdent` + that **changes when the policy changes** (`:` is the + shape): the SQL result cache decides "did the policies move?" by comparing + specs, so a constant ident makes an edited policy look unchanged, while a + spec that does not compare equal to an identical one evicts the cache on + every lookup. Restrictive filters are ANDed and permissive ones ORed — the + engine owns the merge. Returning nothing means "no policy here", never a + refusal. +10. **Caching.** Inside the plugin, invalidated on your source's terms. The + engine adds none and cannot. Your instance is long-lived, so instance state + is the place for it. +11. **Properties.** Instance-wide sources are configured from + `conf/authorization.conf`, catalog-bound ones from + `access_controller.properties.*` — both arrive as a flat + `Map` with no binder. Validate in the factory or the + constructor and fail loudly; a source built with an unparseable setting is + worse than one that refused to be built. +12. **Packaging.** Add `src/main/assembly/plugin-zip.xml` (copy from + ranger-doris): the module jar at the zip root, everything else in `lib/`, + log4j and slf4j excluded because logging is the host's. Verify through + `package`, not `test` — the zip only materialises then. Unzip it once and + look: the jar at the root, no `fe-authorization-api`/`-spi`, no logging + implementation. +13. **Ship it.** Add the module to **both** lists in `build.sh` — the build list + (search `_authz_mod`) and the deploy list (search `AUTHZ_PLUGIN_DIR`). + Missing from one is not a no-op: the deploy step unzips whatever archive is + left in the module's `target/`, so a plugin built but not deployed, or + deployed but not rebuilt, ships stale without failing anything. +14. **API version.** Nothing to do in tree: `fe/fe-authorization/pom.xml` stamps + `Doris-Authorization-Plugin-Api-Version` into every jar built under it. Out + of tree, add the `maven-jar-plugin` `` block from + `fe-authorization-spi/README.md` — a jar that declares nothing is refused, + so a plugin written with no awareness of this contract cannot slip through. +15. **Third-party dependencies.** Bundle what you need rather than depending on + what the host happens to carry; a plugin whose dependencies come half from + the host is a plugin whose behaviour changes when the host is upgraded. Two + CI gates apply: the ASF header check on every new file, and the dependency + license review on every changed pom (see AGENTS.md). +16. **Tests.** Module-level unit tests with Mockito, as the Ranger plugins do. + If the new source changes what the engine decides, it belongs in fe-core's + behaviour baseline too — see "Testing and Verification". + +## Testing and Verification + +- **Unit tests** live in each module; recipes are in `AGENTS.md`. +- **The frozen contract** is guarded by fe-authorization-spi's own suite: + `AuthorizationPluginSurfaceTest` (the surface baseline) and + `AuthorizationPluginContractTest` (silence refuses; `ANY` vs `ALL` are taken + apart the way the requirement says). Run that module's tests after any change + to the api or the spi — a consumer-only run will not catch a stale baseline. +- **The behaviour baseline.** `AccessControlBehaviorBaselineTest` (fe-core) + records every decision over the matrix (resource kind × action × source × + privilege level of the caller) into + `fe/fe-core/src/test/resources/access-control-behavior-baseline.txt`. The + built-in half runs against a real FE with real `GRANT` statements; the Ranger + half runs the production controller over a deterministic stub policy engine. + After a change meant to be structural, `git diff` on that file must be empty. +- **Installed-plugin end to end.** `AuthorizationPluginFromDirectoryTest` + (fe-core) writes a plugin jar into a temporary `authorization_plugins_dir`, + starts an FE on it and checks from SQL that the plugin really decides — that + an account the built-in model granted nothing can read, that an account it + granted `SELECT` cannot, and that the row filter is planned. Copy it whenever + a new mechanism can only be proved from SQL. +- **Version wiring** is `PluginApiVersionWiringTest` (fe-core), which proves + each family's gate is built from its own kernel resource and moves + independently of the others. +- **Selector compatibility** is `AuthorizationSourceSelectorCompatibilityTest` + (fe-core): every string an older release let an operator select a source by + still selects it. +- **Against a live Ranger**: suites under `regression-test/suites/ranger_p2/`, + environment under `docker/thirdparties/docker-compose/ranger/`. +- **At runtime**: `SELECT * FROM information_schema.extensions` lists what was + actually admitted, per family — a plugin refused on its API version is absent + there and explained in `fe.log`. + +## When to Update This Document + +Update this file ONLY when a framework-level fact changes: + +- a module is added, removed or renamed under `fe/fe-authorization/`; +- the loading, selection, routing or lifecycle model changes; +- a durable invariant appears that no gate or test can express. + +Do NOT update it for SPI method changes (javadoc is the API reference, +`authorization-plugin-surface.txt` is the recorded surface), for a new +`AccessAction` or `ResourceKind` constant, or for bug fixes. diff --git a/fe/fe-authorization/fe-authorization-spi/README.md b/fe/fe-authorization/fe-authorization-spi/README.md index 07c4c985137319..fd8ea52d2640b3 100644 --- a/fe/fe-authorization/fe-authorization-spi/README.md +++ b/fe/fe-authorization/fe-authorization-spi/README.md @@ -1,5 +1,11 @@ # Doris FE Authorization SPI +This is the plugin author's quickstart: what to implement, how to package it and +how to install it. For the framework around it — the module map, how the engine +discovers and routes to a source, and a step-by-step walkthrough for a new +source — read `../README.md`; for build recipes and the obligations a change +here carries, `../AGENTS.md`. + ## Overview `fe-authorization-spi` defines the plugin contract for authorization in Doris FE: an *authorization source*