[Do not merge yet]Use hive-exec:core for compilation and and shaded for tests#37780
[Do not merge yet]Use hive-exec:core for compilation and and shaded for tests#37780stankiewicz wants to merge 1 commit intoapache:masterfrom
Conversation
…d adding necessary test dependencies alongside the full jar for runtime, addressing issue with test failures related to Hive Metastore missing classes.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request refines the dependency management for the HCatalog module, specifically addressing how Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the build.gradle file for the HCatalog IO module to use the hive-exec:core classifier for compilation and manages joda-time and other dependencies more explicitly, especially for testing. It also adds necessary JVM arguments for Java module system compatibility to test tasks. The documentation for HCatalog IO has been updated to reflect the hive-exec:core classifier usage. The changes improve dependency management and ensure compatibility with newer Java versions.
| provided library.java.jackson_databind | ||
| // Calcite (a dependency of Hive) bundles without repackaging Guava which is why we redeclare it | ||
| // here so that it appears on the compile/test/runtime classpath before Calcite. | ||
| provided library.java.joda_time |
There was a problem hiding this comment.
The joda_time library is already declared as an implementation dependency on line 48. Adding it again as a provided dependency on line 56 might be redundant unless there's a specific reason related to dependency resolution order or classpath management for the provided scope that is not immediately apparent. If the implementation declaration already covers the need for joda_time during compilation, this line can be removed to avoid redundancy.
| exclude group: "org.apache.hive", module: "hive-exec" | ||
| exclude group: "com.google.protobuf", module: "protobuf-java" | ||
| } | ||
| testImplementation library.java.joda_time |
There was a problem hiding this comment.
Similar to the provided declaration, joda_time is already an implementation dependency on line 48. If the implementation dependency is sufficient for test compilation and runtime, this testImplementation declaration might be redundant. Please confirm if this explicit testImplementation is necessary, perhaps due to specific test classpath requirements or to override a transitive dependency.
| description = "Runs HCatalog tests with Hadoop version $kv.value" | ||
| classpath = configurations."hadoopVersion$kv.key" + sourceSets.test.runtimeClasspath | ||
| include '**/*Test.class' | ||
| jvmArgs "--add-opens=java.base/java.net=ALL-UNNAMED" |
There was a problem hiding this comment.
The jvmArgs for --add-opens is already applied globally to all Test tasks via the tasks.withType(Test).configureEach block starting on line 130. This specific declaration on line 126 for hadoopVersion${kv.key}Test tasks becomes redundant and can be removed.
|
Assigning reviewers: R: @chamikaramj for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
1 participant
Removing hive-exec which brings joda-time and other dependencies shaded.
for tests, shaded is used, but ensuring order of dependencies - full hive-exec fat jar is strictly needed for embedded cluster setups due to many dependencies.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.