Native SSH Tunnel support for database connections #6573#6776
Open
pyjams wants to merge 3 commits intoapache:mainfrom
Open
Native SSH Tunnel support for database connections #6573#6776pyjams wants to merge 3 commits intoapache:mainfrom
pyjams wants to merge 3 commits intoapache:mainfrom
Conversation
…ers to connect to databases in private networks through an SSH bastion host - Supports password, keyboard-interactive, and private key authentication - New "SSH Tunnel" tab in the database connection editor with enable/disable logic - SSH keepalive (30s interval) to prevent VPN/firewall from dropping idle connections - Tunnel lifecycle properly managed even for pipeline connection groups - **SshTunnelManager** (new): manages JSch sessions with local port forwarding - **IDatabase/BaseDatabaseMeta**: SSH tunnel fields persisted via `@HopMetadataProperty` - **Database.java**: opens tunnel before JDBC connect, closes in `closeConnectionOnly()` (not `disconnect()`) to prevent tunnel leaks with grouped connections - **DatabaseMetaEditor**: new SSH Tunnel tab with field enable/disable based on config - i18n: English and Italian labels - [x] Unit tests for SshTunnelManager (5 tests) - [x] Unit tests for Database SSH tunnel integration (5 tests) - [x] All 10 tests pass - [x] Manual testing with MySQL over SSH tunnel (verified working)
Contributor
|
Thanks for this @pyjams! |
Contributor
|
We removed the vintage Junit engine and migrated all our tests to Junit 5+. I can take a look at that if you want |
Author
Thank you, I’ve just updated the files. There shouldn’t be any errors now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Native SSH Tunnel support for database connections
Addresses #6573
Problem
Currently, Apache Hop has no built-in way to connect to databases that sit inside private networks behind an SSH bastion host. Users are forced to set up external SSH tunnels manually (e.g. via
ssh -L) before running pipelines, which complicates deployment and is error-prone in automated/containerised environments.Solution
This PR adds native SSH tunnel support directly in the database connection metadata, so that Hop can transparently open an SSH tunnel before establishing the JDBC connection — no external tooling required.
What changed
SshTunnelManager(new class): manages JSch SSH sessions with local port forwarding. Supports three authentication methods: password, keyboard-interactive, and private key (with optional passphrase). Includes a 30-second keepalive interval to prevent VPN/firewall idle-connection drops.IDatabase/BaseDatabaseMeta: added SSH tunnel configuration fields (sshEnabled,sshHost,sshPort,sshUsername,sshPassword,sshPrivateKey,sshPassphrase) persisted via@HopMetadataProperty.DatabaseMeta: delegate getters/setters for the new SSH fields.Database: opens the SSH tunnel before the JDBC connect and rewrites the JDBC URL to point tolocalhost:<localPort>. The tunnel is closed incloseConnectionOnly()(notdisconnect()) to avoid premature teardown when connections are shared across a pipeline connection group.DatabaseMetaEditor(UI): new "SSH Tunnel" tab in the database connection dialog, with fields that enable/disable dynamically based on the selected authentication method.Testing
SshTunnelManager(session creation, port forwarding, authentication modes, error handling)DatabaseSSH tunnel integration (tunnel open/close lifecycle, URL rewriting, grouped connections)Checklist
mvn clean install apache-rat:checkpassesDependencies
Added
com.jcraft:jsch:0.1.55tocore/pom.xml(widely used, Apache-compatible BSD license).Note: pre-existing LDAP build failure
The
hop-transform-ldapmodule fails to compile on currentmaindue to commit859d0dbc(PR #6665 — LDAP XML cleanup) which changed several methods inLdapOutputMetaandLdapInputMetatoprivatethat are still called by external converter classes (LdapOutputOperationTypeConverter,LdapOutputDerefAliasesTypeConverter,LdapOutputReferralTypeConverter,LdapInputSearchScopeConverter). This is unrelated to the changes in this PR.