Decouple core library from libcudf

[mbrobbel]

Summary

Closes #142.

cuCascade is meant to be a generic, tiered GPU memory/data manager, but several pieces were tightly
coupled to libcudf — forcing every consumer to depend on the whole cuDF stack. This PR removes that
coupling: the core library now depends only on RMM + CUDA (plus libnuma, and kvikIO/cuFile for disk),
and the cuDF-specific representations, converters, and bandwidth profiler move out to the domain layer
(downstream consumers that link libcudf).

cuCascade's abstractions — idata_representation, the converter registry, data_batch, repositories,
disk I/O backends, the memory subsystem — were already generic; this PR makes the dependency boundary
match the architecture.

42 files changed, +259 −10,444

What changed

Public API / library

• idata_representation no longer exposes any cuDF type. The cuDF-backed concrete representations are
  gone from the library — disk_data_representation is now the only in-library concrete representation.
• record_writer_event() / get_writer_event() moved onto the idata_representation base as virtuals
  (no-op / nullptr defaults). read_only_data_batch::get_writer_event() now dispatches polymorphically
  instead of dynamic_cast-ing to the GPU type.
• Extracted a generic, domain-agnostic memory::column_metadata (opaque int32_t type tag — the numeric
  value of a consumer's column-type enum, which cuCascade never interprets) into
  include/cucascade/memory/column_metadata.hpp, used by the disk tier.
• The converter registry now ships empty — register_builtin_converters() is removed; consumers
  register tier converters via register_converter<Source, Target>().

Build

• Remove find_package(cudf) and cudf::cudf from the link interface; remove find_dependency(cudf)
  from the installed CMake config — consumers no longer transitively pull cuDF.
• pixi: depend on librmm directly (was transitive via libcudf); rename cudf-* features → rmm-*.

Removed (now provided by the domain layer)

• gpu_data_representation, cpu_data_representation (host_data_representation,
  host_data_packed_representation), host_table.hpp, host_table_packed.hpp.
• The ~1,800-line built-in converters in representation_converter.cpp + register_builtin_converters().
• The bandwidth profiler (bandwidth_profiler.{hpp,cpp}).
• The cuDF-coupled tests + test/utils/cudf_test_utils.*, and the converter/disk benchmarks.

Docs / CI

• Updated README, docs/*, and CLAUDE.md to the decoupled architecture; removed docs/bandwidth-profiler.md.
• Disabled the CI benchmark job (its converter benchmarks moved out with the cuDF code).

Acceptance criteria (#142)

• find_package(cudf) absent from CMakeLists.txt
• cudf::cudf absent from every link target
• No #include <cudf/...> in any cuCascade source/header (remaining cuDF mentions are explanatory comments)
• cmake-release + build-release succeed; the build graph has zero cuDF references
• cuCascade tests pass using only synthetic/mock representations
• Disk I/O backends remain functional with raw GPU-buffer round-trips (test_disk_io_backend)

Testing

Built and ran the full suite locally on an NVIDIA RTX PRO 6000 (sm_120):

cmake --preset release -DCUCASCADE_BUILD_TESTS=ON
cmake --build build/release -j && (cd build/release && ctest --output-on-failure)
# 100% tests passed, 0 failed  (cucascade_tests, cucascade_topology_discovery_tests)

Verified the build references no cuDF via find_package / link / #include. (A fully cuDF-absent
environment will confirm at the package level once the lockfile is re-solved against the new rmm-*
features.)

Reviewer notes

• column_metadata kept (not removed): the issue lists host_table.hpp's column_metadata among
  the types to move out, but disk_data_representation stays and needs it — so it's retained as a
  generic, cuDF-free struct rather than deleted.
• On-disk format clarified: a disk file is raw 4 KB-aligned column buffers with no header;
  per-column metadata lives in-memory in disk_table_allocation (a file is only meaningful alongside its
  allocation). Corrected a stale CLAUDE.md claim of a disk_file_header.
• Follow-up: add cuDF-free benchmarks (e.g. raw-buffer disk I/O) and re-enable the CI benchmark job.

---

🤖 This PR was prepared with the assistance of an AI agent (Claude Code).

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[mbrobbel]

/ok to test b24413f

[mbrobbel]

/ok to test 79fb183