[feat](authorization) introduce an authorization plugin SPI and move the Ranger sources out of fe-core - #66770
Open
morningman wants to merge 14 commits into
Open
[feat](authorization) introduce an authorization plugin SPI and move the Ranger sources out of fe-core#66770morningman wants to merge 14 commits into
morningman wants to merge 14 commits into
Conversation
…cy payloads First artifact of the FE authorization plugin work: a dependency-free API module that will hold the types an authorization plugin exchanges with the engine, starting with the two that have a consumer in this series - the row filter and column mask payloads. Both carry a SQL expression in Doris dialect rather than a parsed expression tree. That is the lossless form for every producer that exists: an internal row policy comes from CREATE ROW POLICY text, a Ranger filter is text a human typed into the Ranger UI, and a Ranger mask is a transformer template from the service definition - which is always a function call, a shape the neutral connector expression layer's reverse converter does not accept at all. Trino splits the same way: structured expressions engine-to-plugin (pushdown), SQL text plugin-to-engine (ViewExpression, masks). Both types are immutable values with real equality, which is load bearing rather than cosmetic: the SQL result cache decides "did the policies change?" by comparing the specs recorded at plan time with the specs evaluated now, so identity equality there evicts the cache on every lookup. The tests assert both directions - equal while the policy is untouched, unequal the moment its version or text changes. Verified: mvn -pl fe-authorization/fe-authorization-api -am test (7/7, build cache disabled) and checkstyle:check, both green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ger data policies Pins the contract NereidsSqlCacheManager relies on before serving a cached result: it re-evaluates a query's row-filter and data-mask policies and compares them by value with what was recorded at plan time, so an untouched policy must re-evaluate to an equal object. The Ranger policy types do not hold up their end - neither implements equals, and RangerAccessController builds a fresh instance on every evaluation - so the comparison degrades to identity and always reports a change. A user with any row filter or masked column configured therefore never gets a SQL cache hit; the entry is evicted on every single lookup. Confirmed here rather than inferred: the two "untouched policy" cases fail and the five change-detection cases pass, which is the exact signature of missing value equality. This commit is deliberately RED - the payload rework in the next commit turns it green. The change-detection cases are pinned alongside so that fix cannot buy cache hits by weakening the comparison: an edited or revoked policy must still invalidate, or the cache keeps serving rows the user may no longer see. Verified: mvn -pl fe-core -am test -Dtest=NereidsSqlCacheDataPolicyTest (build cache disabled) - 7 run, 2 failures, both the untouched-policy cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… planner as SQL text The two data-policy payloads an access controller returns were fe-core types that leaked the planner into the authorization surface: RowFilterPolicy handed back a Nereids Expression, so any third-party controller had to compile against fe-core and build an expression tree to express "region = 'cn'". They are now the neutral value types from fe-authorization-api, carrying the predicate and the mask as SQL text in Doris dialect, and the planner does the parsing - which is what the Ranger path already did. Fixes the SQL cache never hitting for anyone with a row filter or a masked column: the cache asks "did the policies change?" by comparing what it recorded at plan time with what evaluates now, the Ranger payloads had no equals, and each evaluation built a new instance, so the answer was always "changed". The new payloads are values with real equality, and their identity folds in the policy version so an edited policy still compares unequal. The repro test from the previous commit is green. Two things this turned up, both now pinned by tests: - The payload has to carry the merge type. The design sketch had the engine always AND row filters, but built-in policies can be permissive, which ORs them; without the field, two permissive policies would silently become an AND and the user would see nothing. CheckRowPolicyTest now covers a permissive pair and a restrictive pair - a single policy cannot tell the two merge modes apart, which is why no existing test caught this. - A built-in policy's predicate cannot be produced by rendering its parsed expression: CompoundPredicate.toSql() emits the diagnostic form AND[a,b], which does not parse back, so every policy combining two conditions would have broken. The text is instead recovered from the CREATE statement the policy already stores, captured at parse time - the text the administrator actually wrote. RowPolicyFilterSqlTest runs 25 representative predicates through create -> store -> hand to planner and asserts the predicate comes back identical; 7 of them failed against the rendering approach. An unparseable stored policy still fails the query with the same message as before rather than disappearing, since a vanished row filter exposes the whole table. Verified: 175 tests across every suite touching the changed surface (cache, policy, privilege, parser, Ranger), 0 failures, 0 skipped, build cache disabled. Mutation check: forcing permissive to merge as restrictive turns CheckRowPolicyTest red, and the assertion that fires is the new one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lden matrix Two reworks are queued behind this: dropping the hasGlobal argument from the controller interface, and collapsing AccessControllerManager into pure routing with no cross-plugin OR. Both are meant to change structure and nothing else, and "nothing else" is only checkable against a recording made beforehand. The matrix crosses the default controller (built-in / Ranger) with the catalog kind (internal, unbound external, Ranger-governed external), five callers whose privilege shapes each pin one invariant, every PrivPredicate constant found by reflection, and every check the manager exposes. The built-in half runs against a real FE with real grants and a real row policy; only Ranger's policy engine is stubbed, and it denies the built-in admin everywhere so that wherever that user still passes on a Ranger-governed resource, the verdict provably came from the engine's OR rather than from Ranger. Each row lists the actions that were allowed, so a flipped decision shows up as one action name appearing or disappearing on one line. Recording it surfaced three things worth writing down: - checkCloudPriv cannot be probed at all on a non-cloud FE: the built-in path casts the system info service to its cloud subclass unconditionally, so every caller that does not already hold a global privilege gets a ClassCastException. That probe is excluded rather than frozen. - "the default workload group is always allowed" is implemented twice, in Auth and again in Role, so removing either one alone changes nothing. - a column check rejects any privilege carrying neither SELECT nor LOAD with an IllegalStateException, and whether a caller hits it depends on whether the manager's global short circuit answered first - which is exactly the ordering the rework moves, so the baseline records that outcome as its own column. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l exemption The manager used to establish a global verdict on every controller's behalf and hand it down as the hasGlobal argument, which the interface turned into "if (hasGlobal) return true" for catalog, database, table and column checks. Two things were tangled in that one line. For the built-in controller it was its own privilege model talking to itself. For a catalog bound to Ranger it was something else entirely: a second controller's verdict granting access inside a range Ranger governs, decided by the engine, invisible from the plugin. Which policies apply to a resource was therefore not readable from which controller the catalog is bound to. The argument is gone and the manager only routes now. Each controller says for itself what a global privilege buys: - the built-in one checks global privileges ahead of the fine grained ones. Not redundant with the per-role checks underneath: Auth refuses NODE privileges below global level, so a caller holding only global NODE_PRIV is granted by that line and by nothing else. - the Ranger controllers ask whichever controller owns global scope - the one access_controller_type installs - and honour its grant. That is the same question the engine used to ask, so the answer is the same in every deployment: with the built-in controller installed globally an administrator still reaches a Ranger-governed catalog, and with Ranger installed globally Ranger keeps deciding its own exemptions. Asking the built-in model directly instead would have handed built-in ADMIN_PRIV a way into a Ranger-only deployment, which is a policy change and not this commit's business. A controller that is itself the global-scope authority skips the question: its own global check answers it one line later, and asking through the manager would evaluate the same Ranger policies twice per database, table and column check. The behaviour baseline is unchanged, byte for byte. The interface no longer hands a third-party controller the exemption for free, which is the point - it is a policy decision, and a plugin that refuses it is a legal plugin. The tests that covered the deleted default methods move to the built-in implementation, where that policy now lives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ision The manager offered nine ways to ask the same question, each naming its own resource in loose strings and each picking a controller on its own. Which controller answers for what was therefore spread across nine method bodies, and the resource was described differently in every one of them. There is one routing point now. decide(subject, resource, requirement) picks the controller from the resource alone and returns its answer; the nine entry points build a resource and call it. Adding a chain of sources later, or an audit record of what was asked, is a change to that one function. The resource and the requirement are neutral types, in fe-authorization-api, so they can later cross into a plugin that must not see fe-core. Three things about their shape were decided by what the code actually does: - an action stands one-to-one with a privilege Doris grants. Folding cluster usage and stage usage into plain usage would leave the built-in model - one of the sources being asked - unable to tell which of its three privilege bits the caller meant. - a requirement carries a set of actions and whether one or all are needed, rather than a single action. PrivPredicate is not a closed set of constants: a GRANT statement builds one on the spot out of the privileges it names, and those are the "all of" ones. Asking one action at a time would also undo the Ranger controller's habit of remembering which privileges an outer resource level already granted, turning one walk of the hierarchy into several. - translating back returns the very constant the requirement came from. The engine tells questions apart by comparing predicates against those constants with ==, in Role and in both Ranger controllers, so an equal-but-different object silently drops "may see this catalog" for every user who had no explicit grant on it. The behaviour baseline caught exactly that, twelve cells of it, before this was rewritten to canonicalize. Columns keep an entry point of their own. Their answer has a different shape - which column was refused, carried by an exception - and merging it into a yes-or-no now would mean swallowing that. Verified: behaviour baseline unchanged; 67 tests in 10 classes, no failures; checkstyle clean. Mutation: making a GRANT statement's "all of these privileges" read as "any of them" leaves the baseline green - it only enumerates the OR-shaped constants - and turns the round trip tests red, which is why they exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…plement Until now "who decides access" and "how the decision is made" were the same code. The manager routed to a controller, and that controller was an fe-core interface with a method per kind of object, each answering with a boolean. Nothing outside this repository could be that thing without also being part of it. There is a contract now, in fe-authorization-spi, and the built-in privilege model is the first source implementing it. The manager keeps only the routing - a table from resource kind to the source that governs it, and one conversion of a refusal back into the boolean its callers still expect. The switch over nine controller methods now lives inside each source, which is where the knowledge of how that source answers belongs. The contract carries only what the engine asks today: check a requirement, check one action, ask for row filters, ask for column masks, and a lifecycle. Filtering a list of objects, "has any privilege below here", enumerating grants and declaring capabilities are designed and deliberately absent - each has its consumer in a later phase, and the filtering signature in particular should follow from the 45 call sites that will use it rather than from guessing at them. An interface freezes when a release ships it, not when it is written, so growing it before then costs nothing. A refusal is thrown rather than returned. There is no third answer for a source to express and no boolean for a caller to forget, and the reason - which every caller currently invents for itself - has somewhere to live for the first time. The exception records no stack trace and composes its message only if something reads it: listing what a user may see refuses most of the objects that exist, so refusing is the common path here, not the exceptional one. Three things were settled by what the code does rather than by design: - the neutral subject carries the account and nothing else, and that is lossless. The built-in model has to turn it back into a UserIdentity to look up grants and row policies, and UserIdentity's own equality is exactly user, host, and whether the host is a domain; the certificate fields take part in authentication, never in a decision about what an account may do. It reads the account with getUser() rather than getQualifiedUser() - the two return the same field, the second additionally insists the identity went through analysis, and callers exist that check access with one that has not. - roles are not on the subject. Ranger needs them, the built-in model never does, and filtering what a user may see asks thousands of questions per statement, so they are a lazy lookup on AuthorizationContext instead. - controllers written against the older interface keep working, wrapped. That interface is what a catalog's access_controller.class names, so implementations of it exist outside this repository. Wrapping breaks one thing that had to be repaired. The Ranger controllers ask "am I myself the one governing global scope?" by object identity, so as not to ask themselves a question they are about to answer. Reached through a wrapper, that comparison silently stops matching - the behaviour is identical, the same Ranger policies are simply evaluated a second time on every database, table and column check. The manager answers that question now, and sees through the wrapper to do it. Two defects this introduced were caught by the tests written for it, and neither of them is visible to the behaviour baseline: - a refused column check answers by naming the column that failed, and that message leaves the privilege model as an AuthorizationException and reaches the caller as one again. Carrying the rendered form across the middle instead of the bare wording puts that class's error-code prefix in twice, changing what every denied column query prints. - checking access with an identity that never went through analysis used to work. Asking it for its qualified name on every check turned those callers into IllegalStateException. Verified: behaviour baseline unchanged; 428 tests across the 79 classes that touch the decision path, no failures, the four skips pre-existing; 34 tests in the two new modules; checkstyle clean. Mutations: reading "all of these actions" as "any of them" turns three contract tests red; dropping "the host is a domain" from the subject turns the round trip red; routing a storage vault through the resource check turns the adapter's routing test red and moves one baseline line; and making the global-scope identity comparison stop seeing through the wrapper leaves the baseline green while turning red only the test that counts policy evaluations - which is precisely why that test counts them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… contract themselves The two Ranger sources shipped in the tree were reached through the adapter for the older per-scope controller interface, so the contract had exactly one native implementation - the built-in privilege model, which answers out of its own tables. A contract that has only been implemented by the thing it was extracted from has not been tried. They now implement it directly: one question about a typed resource, answered by refusing or not, dispatched inside the source. What each of them decides is unchanged, down to walking the resource hierarchy one privilege per request while remembering which ones an outer level already granted - that walk is why the engine asks about a whole requirement rather than one action at a time. Two things they used to reach into the engine for now arrive through the context the engine hands them: the roles a Doris account holds, which a Ranger policy may be written against and which only the engine can resolve, and whether whoever governs instance scope already grants the privilege. The latter is also configurable now, so a deployment can decide that inside what Ranger governs only Ranger's policies grant anything - not even to an administrator of the instance. Recognising which question is being asked used to be object identity against the engine's own predicate constants. Those constants cannot follow the sources out of the engine, so the questions the engine asks by name are now values in the neutral module, compared by equality and pinned on both sides: a source that stopped recognising "may this subject see the object" would silently answer a different question rather than fail. Behaviour matrix unchanged. The refusal message for a column now names the requirement in neutral terms.
… directory Deciding access became a contract in the previous change, but the only way to publish an implementation of it was to be on the FE's class path - which means being built into the FE. A source that ships separately had nowhere to be. It has one now: plugins/authorization/<name>/, the layout the connector, filesystem, authentication and lineage families already use, read through the same loader. That loader is what makes the difference from the older channel this one sits beside: it gives each plugin its own classloader, and it takes a version gate as a mandatory argument rather than as something a family remembers to add. So a jar dropped in there has to declare, in its manifest, the authorization plugin API it was built against, and one that declares nothing - or another major - is refused instead of inheriting the kernel's own answer. Both channels read the same directory and cannot collide: the older one lists jars lying loose at the root, this one lists subdirectories. Three things follow from what the loader does rather than from the design: - org.apache.doris.authorization. is loaded parent-first, so the vocabulary crossing the boundary exists exactly once. A plugin bundling its own copy of ResourceKind would otherwise hand back a value the engine refuses to recognise as the type it asked for, which reads as a plugin bug and is not one. - a name already answered by a source shipped with the FE keeps it. Otherwise a jar named ranger-doris, dropped into a directory, would displace the real one - and an allow-everything plugin under a trusted source's name is the whole point of refusing that. - the class path channel is deliberately NOT gated. What is on the class path was built from this tree in this build, so its version would be compared against itself; and the in-tree Ranger sources live in fe-core.jar, which carries no authorization stamp at all, so gating it would refuse them at startup. No unit test can see this - a test loads its classes from target/classes, a directory, where there is no manifest to read. A refusal used to be invisible where it hurt. "No authorization plugin factory found for X" read identically whether X was never installed or was installed and refused on its version, so the reason is now appended to it, naming both versions. The older AccessControllerFactory and CatalogAccessController interfaces keep working and are now deprecated. Registering one logs what to implement instead, where to put it, and which manifest attribute to declare. AuthorizationPluginSurfaceTest freezes the contract - the SPI interfaces and the api types they speak in - so that adding a method or an enum constant fails until the major version moves in the same commit. Copying the authentication family's baseline would not have worked: erasure hides RowFilterSpec inside List<RowFilterSpec> unless generic signatures are walked, members inherited from the JDK have to stay out or a JDK upgrade turns it red for nothing, and the self-referential Enum<E extends Enum<E>> overflows a recursive closure. Verified: behaviour baseline unchanged; 480 tests across the 88 classes that touch the decision path, no failures, the four skips pre-existing; 28 api plus 8 spi tests; checkstyle clean in all three modules. The two version numbers were checked in the built jars by hand, there being no test that can read a pom's <manifestEntries> against the attribute name the gate derives. Mutations, five of five: adding a default method to the contract turns the frozen surface red; not recording why the gate refused a plugin turns the two "the message names both versions" tests red while leaving the admitted case green; changing the parent-first prefix turns the shared-vocabulary test red; admitting every version turns six tests red across three families. The fifth - deleting the guard on a name already taken - passed at first, and the escape was the finding: the assertion compared Class objects across classloaders, where they can never be equal, so it could not fail. Comparing class names instead turns it red. A second assertion in the same test, on the inventory table, could not fail either and now checks that a refused plugin is released along with its classloader. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y decides An authorization source can be shipped as a jar and dropped into plugins/authorization/ since the previous change, and the tests written for it reach as far as "the source named in fe.conf is the one the manager reports". That is one step short of what the channel exists for. Nothing put a statement through the engine, so nothing would have noticed a plugin that was installed and then never asked, or asked and then overruled. There is a worked example now, and a test that installs it the way a third party installs one - a jar written into the plugin directory, discovered at startup, admitted on the API version it declares, loaded through its own classloader, built by its factory - and then checks three things that are only observable from SQL: - an account the built-in model granted nothing reads a table, because the example allows it; - an account the built-in model granted SELECT on that same table cannot, because the example does not. This is the least intuitive consequence of one source answering for a resource, and the easiest for an implementation to quietly lose; - the row filter the example returns is planned over the table for the account it applies to, and not for the account it does not. The example grants by Doris role rather than allowing everything, which is what the plan had called for. An allow-everything plugin cannot tell "the plugin said yes" apart from "something else said yes first" - both look identical from a passing test. Only a refusal that really refuses pins the channel down, and the second check above is that refusal. Three things this turned up, all written into the example's own comments because a plugin author meets them immediately: - a source installed for the whole instance has to answer for administration itself. grantedByGlobalScopeAuthority answers false when the source asking IS that authority, which an instance-wide source always is, so a source with no admin rule of its own locks out every account - including the one that would put the configuration back. - the properties an instance-wide source is configured with cannot be fed from a unit test: their path is built from the DORIS_HOME environment variable, which the harness never exports. The example runs on its defaults and says so. The per-catalog property channel is a catalog property and is reachable. - the example cannot live under org.apache.doris.authorization, which is loaded parent-first. Placed there it would come off the FE's own class path, with no jar and so no declared version, and every assertion would hold for the wrong reason. A plugin written outside this repository never meets this. The jar-writing helper moves out of the version-gate test into utframe.PluginJarWriter. Both tests depend on the same directory layout, and a layout contract kept in two copies is one that drifts. Deliberately not here: the example is not packaged into a release. It would put a jar on every FE that turns into an allow-everything authorization source one config line later, and no other plugin family ships an example module either. That also rules out a regression-level test, which would need the jar to be lying in a real cluster's plugin directory. Verified: 87 test classes, 480 tests, no failures, the four skips pre-existing; the behaviour baseline unchanged; the legacy channel's own regression still green. Mutations, four of four: never refusing turns the built-in-grant test red; returning no row filter turns the row-filter test red; filtering everyone turns the same test red through its negative control; and a jar declaring no API version stops the FE from starting at all, refused with both versions and the missing attribute named. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l and into plugins of their own The Ranger sources answer the authorization plugin contract already, but they still lived inside fe-core, which meant every FE compiled and shipped ranger-plugins-common and its hadoop closure whether or not anyone authorized against Ranger. They move out into their own modules under fe-authorization/fe-authorization-plugins/. Three modules rather than one: a plugin directory admits exactly one factory (DirectoryPluginRuntimeManager refuses a directory whose jars declare more than one), and Ranger publishes two sources - one for a whole instance, one bound to a catalog - so each needs its own directory and its own jar carrying its own service descriptor. What they share goes in a third module that deliberately declares no descriptor, so it can sit in both plugins' lib/ without either directory looking like it publishes two sources. Same shape as fe-filesystem-s3-base. Package names are unchanged, so a catalog naming its source by class name keeps working. The one exception is the doris-service factory, which used to live in fe-core's privilege package; leaving it there would have split that package across two jars. Two references to fe-core had to go, both unrelated to authorizing anything: the shared audit-log flush timer is now built in the plugin (which costs its entry in the FE thread-pool metrics - no plugin loaded from a directory can register there), and the name of the workload group everyone may use is a constant in the plugin (the behaviour baseline catches it drifting from the engine's). The lookup behind row filters and data masks used to ask with the Doris service's spelling of the read access type for both services. That is what tied the shared code to one of them, so each source now names its own; both spell it SELECT, so nothing changes on the wire. Left alone, and now recorded: the Hive privilege checks lower case the access type while these lookups do not. fe-core keeps a test-scope dependency on the doris-service plugin, as it already does for the connector and filesystem plugin modules. The access-control behaviour baseline records what a Ranger-governed instance decides for every user and every privilege by running the production controller, and that is the only thing that can show this move changed no decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…installs The previous commit took Ranger out of fe-core. Nothing put it back into a release, so a build from that commit alone produces an FE where access_controller_type = ranger-doris refuses to start, without anything in the build having failed. Each of the two sources now assembles the plugin zip DirectoryPluginRuntimeManager reads - its own jar at the root, everything it needs at runtime in lib/ - and build.sh unpacks them into plugins/authorization/ranger-doris/ and plugins/authorization/ranger-hive/. The build list and the deploy list are the same list, written next to each other, because a module in one and not the other does not fail: the deploy step unpacks whatever archive an earlier build left behind, and ships stale. plugins/authorization/ is created whether or not either module built, since it is also where an administrator drops a third-party source. The api and spi jars are provided and so absent from the zips; they are loaded parent-first from the FE, which is what makes the types a plugin hands back the ones the engine asked for. Ranger and its hadoop closure are bundled instead of borrowed from fe/lib, which costs about 69MB per plugin and buys a plugin whose behaviour does not change when the host's dependencies do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lder release did A catalog names the source governing it in access_controller.class, and that value is persisted with the catalog: an upgraded FE reads back whatever was written when that catalog was created. Moving the Ranger sources into plugins kept every package name except one - the doris-service factory, which lived in fe-core's privilege package and could not stay there without splitting that package across two jars. A catalog naming that class became unreachable at the first statement touching it. A table of the factory classes fe-core used to publish, and the names publishing them now, answers after the runtime table built from the classes that still exist. It lives in the engine rather than in the plugin because the class it names was the engine's own: whoever owned an identifier owns remembering it. Trino, renaming its Hive connector, put the superseded name in the Hive plugin for that same reason - there the identifier that went stale was the plugin's. Both "no plugin factory found" failures now name the directory an authorization plugin is installed in. That is the message an upgrade produces when the plugin directory was not carried across, and without the path it sends an operator looking in fe/lib, where the source used to be. The selectors are pinned as literals, never derived from the classes publishing them: derived, an expectation travels with the code it is meant to pin, and the next package move would leave it green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…guration files A plugin's configuration lives in $DORIS_HOME/conf, which is on the FE's class path and therefore the parent of every plugin classloader, and a library inside a plugin looks its configuration up through the classloader that loaded it - its plugin's. The Ranger sources are the live example: RangerConfiguration asks its own classloader for ranger-<service>-security.xml, and now that those classes are a plugin, that call reaches ChildFirstClassLoader. It works because only the plural getResources is overridden; the singular one keeps the JDK's parent-first search. Nothing said so, and overriding that one too for symmetry with the class-loading policy is an easy change to make. Ranger does not treat a missing configuration as an error - it carries on with an empty one, authorizing against no policies at all - so this is pinned rather than left to be noticed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
morningman
requested review from
924060929,
CalvinKirs,
englefly,
gavinchou,
morrySnow and
starocean999
as code owners
August 14, 2026 06:50
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Deciding what a user may access is wired into
fe-core.CatalogAccessControlleris anfe-coreinterface with one method per kind of object, the two Ranger integrations implement it from inside
the kernel, and the row-filter / data-mask payloads handed to the planner are kernel classes.
Anything outside this repository that wants to decide access has to be compiled against
fe-coreinternals, and the Ranger sources ship inside
fe-core.jarwhether a deployment uses them or not.This PR turns "who decides access" into a plugin contract, and makes both the built-in
GRANTmodeland the Ranger sources implementations of that contract.
New modules
fe-authorization-api— a neutral vocabulary with nofe-coretypes: actions, requirements("any of these" / "all of these"), the resources that can be asked about, subjects, and the
row-filter / data-mask payloads.
fe-authorization-spi— the contract itself: check a requirement, check one action, ask for rowfilters, ask for column masks, plus a lifecycle and an
AuthorizationContextcarrying thequestions a decision needs (the roles of a subject; whether the instance-wide source has already
granted it).
fe-authorization-plugins/{ranger-common,ranger-doris,ranger-hive}— the Ranger sources, nowoutside the kernel and shipped as installable plugins.
What changed in the engine
AccessControllerManagerkeeps only therouting — a table from resource kind to the source governing it — and converts a refusal back into
the boolean its existing callers expect.
together. Each source now grants or refuses its own exemptions, which is what makes the policy in
force on an object readable from the configuration rather than from two places at once.
version gate, a child-first class loader, a per-plugin directory under
plugins/authorization/,and registration in
information_schema.extensions.Compatibility
access_controller.classkeeps accepting the class names it accepted before. The one name thatmoved out of the kernel is mapped by an alias table inside the kernel, and a third-party
controller still implementing the old interface keeps working through an adapter (the old
interface is deprecated, not removed).
access_controller_typeis unchanged; its accepted values are the same.the four payload classes it used were replaced by neutral ones. Erasure hides this at class-load
time, so it would otherwise surface as
NoClassDefFoundErrorduring execution.Release note
Authorization sources are now plugins.
ranger-doris,ranger-hive) are no longer part offe-core.jar. Therelease installs them under
plugins/authorization/, and an upgraded deployment must carry thatdirectory over — a deployment that does not will start normally and fail on the first statement
that reaches the affected catalog.
information_schema.extensionsnow reports both withSOURCE = EXTERNALinstead ofBUILTIN.ranger-hiveaudit flush timer no longer appears in thedoris_fe_thread_poolmetrics; aplugin cannot register into the FE's thread-pool registry.
recompiled against the new payload types.
Check List (For Author)
Test
The change is meant to keep every access decision identical, so the main evidence is a golden
matrix rather than new assertions:
AccessControlBehaviorBaselineTestrecords the answer forevery combination of default source (built-in / Ranger) × catalog (internal / plain external /
Ranger-governed) × user (global admin, global node, table-level grantee, Ranger-only grantee, no
grants) × all 17
PrivPredicateconstants × probe point (catalog, database, table, columns, rowfilters, column masks, resources, workload groups, storage vaults). It stayed byte-for-byte
unchanged across all of the commits here. Alongside it: an end-to-end test that starts an FE on an
authorization plugin installed from a directory and shows the plugin's refusal is what the user
gets, a frozen interface surface for the api and spi modules, and the tests of the classes
touched (~480 tests) run as a set at each step.
Ranger's own regression suite (
regression-test/suites/ranger_p2) needs a live Ranger server andhas not been run for this PR.
Behavior changed:
SOURCEininformation_schema.extensions, one thread-pool metric, and the recompilation athird-party controller with data policies needs.
Does this need documentation?
documentation PR will follow.
fe-authorization-spi/README.mdcarries a worked example in themeantime.
Check List (For Reviewer who merge this PR)