Expose recent committed ledger prefixes

[Amaury Chamayou (achamayou)]

Summary

• Add opt-in include_committed_prefix=true discovery for committed ledger entries not yet available in canonical .committed files.
• Serve immutable .committed_prefix resources with 307 Temporary Redirect, Cache-Control: no-store, range requests, ETags, and representation digests.
• Preserve physical ledger boundaries and keep prefix files excluded from committed-only discovery and recovery.
• Add host and Python tests, endpoint e2e coverage, a 64 KiB-50 MiB benchmark matrix, OpenAPI updates, and operator documentation.

Design discussion: #8207

Testing

• ledger_test and files_cleanup_test
• Python SDK test suite
• ledger_bench
• Node OpenAPI validation
• Full logging sample build

[Copilot]

Pull request overview

Adds opt-in access to recent committed ledger prefixes while preserving canonical ledger and recovery behavior.

Changes:

• Adds prefix discovery, immutable downloads, ranges, digests, and metadata.
• Extends host and Python ledger parsing.
• Adds tests, benchmarks, OpenAPI definitions, and operator documentation.

Custom instructions used:

• .github/copilot-instructions.md
• .github/instructions/changelog.instructions.md
• .github/instructions/reviewing.instructions.md

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
CHANGELOG.md	Records the new capability.
include/ccf/http_consts.h	Adds the chunk-kind header.
src/host/ledger.h	Materializes committed prefixes.
src/host/ledger_filenames.h	Recognizes and excludes prefix files.
src/host/test/ledger.cpp	Tests prefix creation and recovery exclusion.
src/host/test/ledger_bench.cpp	Benchmarks prefix materialization.
src/host/test/files_cleanup_test.cpp	Tests strict prefix filenames.
src/node/rpc/file_serving_handlers.h	Implements discovery and download endpoints.
src/node/rpc/ledger_interface.h	Extends the ledger subsystem interface.
src/node/rpc/ledger_subsystem.h	Connects RPC and host ledger operations.
src/node/rpc/node_frontend.h	Bumps the API version.
python/src/ccf/ledger.py	Parses committed-prefix filenames.
python/tests/test_ledger.py	Tests Python filename handling.
tests/e2e_operations.py	Adds endpoint integration coverage.
tests/schema.py	Registers the new e2e scenario.
doc/operations/configuration.rst	Documents operator feature gating.
doc/operations/ledger_snapshot.rst	Documents prefix semantics and usage.
doc/schemas/node_openapi.json	Updates the generated API schema.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.