Proposal: Cherry-pick pxf-iceberg feature from `GreengageDB/pxf` to Apache Cloudberry PXF

[tuhaihe]

Background

Currently, Cloudberry offers integration with Apache Iceberg through the FDW (Foreign Data Wrapper) solution, and related PRs are being actively pushed forward. However, users who are already familiar with PXF may prefer a PXF-native approach for reading/writing Iceberg tables.

The GreengageDB/pxf project has implemented an pxf-iceberg extension that enables PXF to intergate Apache Iceberg tables. Since GreengageDB/pxf is licensed under Apache 2.0, and apache/cloudberry-pxf is also licensed under Apache 2.0, there is no license conflict or incompatibility in cherry-picking this feature.

Proposal

Cherry-pick the pxf-iceberg feature from GreengageDB/pxf into apache/cloudberry-pxf, so that users can choose between the FDW approach and the PXF approach when integrating Apache Iceberg with Cloudberry.

Benefits

• Lower barrier for PXF users — Users already familiar with PXF can leverage their existing knowledge and workflows to query Iceberg tables.
• Broader ecosystem support — Offering both FDW and PXF pathways gives Cloudberry users more flexibility.
• License compatibility — Both repositories use Apache 2.0, so there are no legal obstacles.
• Proven implementation — The feature already exists and works in GreengageDB/pxf, reducing the development effort.

Considerations / Open Questions

• What is the scope of code changes required for the cherry-pick?
• Are there any dependencies that need to be ported alongside the core feature?

Related

• Existing FDW-based Iceberg integration PRs GreengageDB/pxf — pxf-iceberg

[leborchuk]

+1, why not? If we can make the projects more closely related and the license allows us to do so, let's do it. Users will be grateful.

There is one consideration. I would not take the code as-is. It is necessary to make a code review. Unfortunately, Greengage code belongs to one company, so the release is tied to the requirements of commercial customers. The developer needs time to freeze the code by a certain date and release. Sometimes this leads to the need to agree to "cut corners" and "do it here and now".

Also, we may need to adopt it to our testing system.

[tuhaihe]

+1, why not? If we can make the projects more closely related and the license allows us to do so, let's do it. Users will be grateful.

There is one consideration. I would not take the code as-is. It is necessary to make a code review. Unfortunately, Greengage code belongs to one company, so the release is tied to the requirements of commercial customers. The developer needs time to freeze the code by a certain date and release. Sometimes this leads to the need to agree to "cut corners" and "do it here and now".

Also, we may need to adopt it to our testing system.

Thanks for the feedback. I agree that we should not simply cherry-pick the code as-is. The goal is to reuse the existing implementation while ensuring it meets Apache Cloudberry PXF quality standards, including code review, dependency checks, and CI/test adaptation.

[ostinru]

I think that we can evaluate this patches (cherry-pick AS IS and run load tests). Then we will be able to compare with alternative approaches.
PS: Probably, this will require some patches to clodberry core.