fix(data-products): guard ports against non-data-asset entities and harden portsView#30186
Conversation
…arden portsView
A data product port must reference a data asset (a dataset such as a table,
topic, or dashboard). Two gaps allowed and then surfaced invalid state:
- getPaginatedPorts fetched each port relationship by grouping on the stored
entity type and calling Entity.getEntities, which throws
EntityNotFoundException for a type that has no repository (e.g. the
search-only pseudo-type "tableColumn"). A single such relationship row made
the whole GET /dataProducts/{..}/portsView endpoint return 500.
- The bulk add-ports operation did not validate that a reference is a data
asset, so resolvable non-asset entities (user, team, ...) could be attached
as ports.
Fix:
- portsView now skips relationship rows whose entity type has no repository
(logged at WARN), so a legacy/invalid row no longer breaks the response.
- Add-ports now rejects any reference that is not a real entity supporting the
dataProducts field, reported as a per-row failure. Eligibility is derived
from the entity's own capabilities (hasEntityRepository + dataProducts
field), so newly added asset types are covered automatically with no list to
maintain.
Adds an integration test asserting a non-data-asset entity is rejected as a
port and that the ports view still loads.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
backend
safe to test
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| result.setNumberOfRowsProcessed(result.getNumberOfRowsProcessed() + 1); | ||
|
|
||
| if (isAdd) { | ||
| if (!isPortEligibleType(ref.getType())) { |
There was a problem hiding this comment.
populateEntityReferences(assets) removes unresolved references before this loop. A tableColumn or unknown-type request therefore never reaches this guard, and the bulk response can omit the requested row instead of reporting the promised per-row failure.
Context Used: CLAUDE.md (source)
| if (!Entity.hasEntityRepository(entityType)) { | ||
| LOG.warn( | ||
| "Skipping {} port(s) of unresolvable type '{}' for data product {}", | ||
| entry.getValue().size(), | ||
| entityType, | ||
| dataProductId); | ||
| continue; |
There was a problem hiding this comment.
When a page contains a historical unresolvable relationship, the DAO has already applied offset and limit before this branch drops the row. The response can return a short or empty non-final page while paging.total still counts the dropped row, so clients that stop on a short page can miss valid ports at later offsets.
Context Used: CLAUDE.md (source)
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
| // registered entity (excludes search-only pseudo-types like tableColumn) that can itself | ||
| // belong to a data product (i.e. its schema declares a `dataProducts` field). This is derived | ||
| // from the entity's own capabilities, so it stays correct as new asset types are added. | ||
| private static boolean isPortEligibleType(String entityType) { |
There was a problem hiding this comment.
💡 Edge Case: isPortEligibleType can throw for time-series entity types
hasEntityRepository returns true for time-series types (it also checks ENTITY_TS_REPOSITORY_MAP), but entityHasField resolves via Entity.getEntityRepository, which only consults ENTITY_REPOSITORY_MAP and throws EntityNotFoundException for TS-only types. So if a time-series entity reference is added as a port, isPortEligibleType throws instead of returning a clean per-row failure, re-introducing the 500 this PR aims to eliminate. The read-path guard in getPaginatedPorts has the same gap: for a TS type hasEntityRepository is true, so it proceeds to Entity.getEntities, which can throw. The primary tableColumn case is unaffected (it is in neither map). Guard the eligibility/read check against types not in the standard entity repository map, or catch the lookup failure.
Exclude time-series entity types so entityHasField (which throws for TS-only types) is never called on them.:
private static boolean isPortEligibleType(String entityType) {
if (!Entity.hasEntityRepository(entityType) || Entity.isTimeSeriesEntity(entityType)) {
return false;
}
return Entity.entityHasField(entityType, Entity.FIELD_DATA_PRODUCTS);
}
- Apply fix
Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎
Code Review 👍 Approved with suggestions 0 resolved / 1 findingsHardens the data product ports read path and adds validation to prevent non-data asset entities from being assigned as ports. Ensure 💡 Edge Case: isPortEligibleType can throw for time-series entity types📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DataProductRepository.java:629-643
Exclude time-series entity types so entityHasField (which throws for TS-only types) is never called on them.🤖 Prompt for agentsOptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
|
🟡 Playwright Results — all passed (36 flaky)✅ 4532 passed · ❌ 0 failed · 🟡 36 flaky · ⏭️ 95 skipped
🟡 36 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Problem
Opening a Data Product detail page can fail with a 500 and the toast:
The failing request is
GET /api/v1/dataProducts/name/{fqn}/portsView.Root cause
DataProductRepository.getPaginatedPortsloads each input/output port by grouping theport relationship rows on their stored entity type and calling
Entity.getEntities(...)per type.
Entity.getEntitiesresolves a repository viaEntity.getEntityRepository,which throws
EntityNotFoundExceptionfor any type that has no registered repository —including
tableColumn, a search-index-only pseudo type (Entity.TABLE_COLUMN) that isdeliberately not a first-class entity.
So a single port relationship row pointing at a non-entity type makes the entire
portsViewresponse 500, breaking the page for every port on that data product.A port is meant to reference a data asset (a dataset such as a table, topic, or
dashboard). The bulk add-ports operation also did not validate this, so a resolvable but
non-asset entity (user, team, …) could be attached as a port.
Fix
Harden the read path.
getPaginatedPortsnow skips relationship rows whose entitytype has no repository (logged at WARN) instead of letting one row throw and fail the
whole response. This is the same graceful-skip behaviour
EntityUtil.populateEntityReferencesalready uses for orphaned references.
Validate on add. Adding a port now rejects any reference that is not a real entity
supporting the
dataProductsfield, reported as a per-row failure (consistent with theexisting opposite-port and ownership guards). Eligibility is derived from the entity's
own capabilities:
There is no hardcoded type list, so new asset types are covered automatically as their
schemas gain the
dataProductsfield.Tests
Adds
DataProductResourceIT#test_addPort_rejectsNonDataAssetEntity: a user (a real entitythat is not a data asset) is rejected when added as an input port, and the ports view still
loads with zero ports. Fails without change (2).
Note
Change (2) prevents new invalid rows via the public API. Change (1) is what recovers data
products that already have an invalid port relationship (created by non-API/historical
paths); such existing rows should also be cleaned up out of band.
Greptile Summary
This PR validates data-product ports and makes the ports view tolerate unregistered entity types. The main changes are:
Confidence Score: 4/5
The unresolved-input and mixed-row pagination paths need fixes before merging.
openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DataProductRepository.java
Important Files Changed
Reviews (1): Last reviewed commit: "fix(data-products): guard ports against ..." | Re-trigger Greptile
Context used (3)