[feature](paimon) Add Paimon 2.0 transactional writes

[damokelis]

What problem does this PR solve?

Issue Number: close #66784

Related PR: #65868

Problem Summary:

The Connector SPI path on master can read Paimon tables, but it still uses Paimon 1.3.1 and does not provide a complete external-table write path. The implementation from branch-4.1 cannot be applied directly because catalog, transaction, planning, scan, and write contracts have moved to the connector modules on master.

This draft:

• upgrades the FE connector, BE JNI scanner/writer, and shaded Hive runtime to Apache Paimon 2.0.0;
• adds Connector SPI write planning and a Paimon table sink backed by the Paimon Java SDK;
• aggregates BE commit messages in the FE transaction and provides acknowledgement, retry reconciliation, abort cleanup, and statement-level commit;
• supports external catalog CREATE TABLE, DROP TABLE, INSERT INTO, INSERT OVERWRITE, and CTAS paths;
• maps primitive and complex values through Arrow, including Paimon 2.0 BLOB, VARIANT, and VECTOR representations;
• supports Parquet and Vortex reads, with automatic JNI fallback for schemas that the native reader cannot decode;
• rejects invalid Vortex schemas before the remote catalog create call.

Release note

Support transactional writes to Apache Paimon 2.0 external catalogs, including CREATE/DROP, INSERT, INSERT OVERWRITE, and CTAS.

Check List (For Author)

• Test
  • Regression test
  • Unit Test
  • Manual test
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

Current validation:

• FE Paimon connector: 552 tests, 0 failures, 1 skipped.
• C++ focused unit tests: 13/13 passed.
• Linux release BE incremental build: passed.
• Changed-line clang-tidy for all 5 modified C++ files: passed.
• clang-format, checkstyle, and git diff --check: passed.
• TKE live deployment:
  • backend ID 1786733167633 remained stable and is alive on 172.21.48.181;
  • deployed image digest: sha256:8f82d5cf69147101a15e6c46f90523391c1a07a0ea28226901fd36bf677f40b6;
  • doris_be SHA256: 055313b210249c0f595f97211855590d0e95336655bbdd03de93b5bc9d9f4887;
  • Paimon scanner SHA256: 0a9b83bb92f376275ef9a428fdc333d414e60c0813a3cad4c175c4b9e44b4b8e;
  • COS Parquet+VARIANT query returned the expected rows, and EXPLAIN VERBOSE reported SplitStat [type=JNI] with force_jni_scanner=false;
  • Vortex+VARIANT create was rejected before the REST catalog call, with no residual table;
  • 28/31 continuously tiered COS tables completed exact COUNT(*) scans; the remaining three were blocked by external COS getFileStatus API timeouts, including after raising the S3A request timeout from 15s to 120s. Their Paimon snapshot/file metadata remained readable.
• Continuous CDC remained healthy during validation:
  • both DTS-to-Fluss tasks kept advancing with queues draining to zero;
  • Fluss-to-Paimon tiering checkpoints advanced through checkpoint 2347 with no failed table epochs.

The PR remains draft because repository automated code review requires a MEMBER/OWNER/COLLABORATOR to trigger /review, and the three full-table COS scans are still externally blocked.

• Behavior changed:

  • No.
  • Yes. Paimon external catalogs gain transactional DDL/DML writes and use Paimon 2.0.0.

• Does this need documentation?

  • No.
  • Yes. A follow-up documentation PR will describe supported catalog properties, statements, and type mappings.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?