[Mixed-source orchestration] Phase 4: Support cross-format Compare with native source engines

[paullizer]

Parent: #1055

Phase 4 - Cross-Format Compare

Planning baseline version: 0.250.061

Status: Proposed

Depends on: Phase 1; Phase 3 evidence contracts are strongly recommended

Objective

Support one-left-to-many-right comparison across any supported source combination:

• table to table
• narrative to table
• table to narrative
• narrative to narrative

Every source must be summarized with its native engine before pairwise comparison.

Current Defects Addressed

• Mixed Compare abandons tabular tooling when any selected source is non-tabular.
• The tabular comparison helper only succeeds when every comparison source is tabular.
• The generic comparison path summarizes CSV/XLSX through ordinary document chunks.
• There is no engine-neutral comparison evidence contract.

Comparison Strategy

Narrative to narrative

Preserve current document analysis summaries and pairwise reduction.

Tabular to tabular

For each source:

• schema and sheet coverage
• relevant counts, totals, distributions, keys, dates, statuses, anomalies
• query-specific computed evidence

When the request requires row-level alignment, joins, overlap, or deltas, invoke validated multi-file tabular operations rather than comparing prose summaries alone.

Narrative to tabular

Produce:

• narrative evidence summary from document analysis
• computed tabular evidence from tabular tools
• one comparison prompt that labels evidence type and source role

Do not imply row-level support from a narrative document unless citations provide it.

Scope

1. Comparison manifest

Resolve Source and Targets through the Phase 1 manifest. Preserve role labels and request order.

2. Native summary pass

Replace the current universal run_document_analysis(...) summary pass with per-source engine dispatch.

Each result becomes a comparison-ready evidence envelope containing:

• source role
• concise summary
• structured facts
• citations
• coverage/failure state
• source version

3. Pairwise comparison

Reuse the existing left-to-many sequence, but build each pairwise prompt from engine-neutral envelopes.

For table-to-table pairs, allow a tabular comparison plan to add structured operations before synthesis.

4. Reduction

Preserve the current multi-target final reduction. It must state:

• which targets were compared
• which engines supplied evidence
• failed/partial targets
• whether conclusions are row-level, aggregate, or narrative

5. Coverage and artifacts

Merge native coverage and generated artifacts without treating an export as comparison prose.

Acceptance Criteria

• CSV Source vs XLSX Target invokes tabular tooling for both.
• PDF Source vs CSV Target invokes narrative analysis and tabular tooling.
• XLSX Source vs DOCX Target works in the reverse direction.
• One failed Target remains visible while successful Targets are compared.
• File order and duplicate filenames across scopes do not alter source identity.
• Table-to-table overlap/delta questions execute validated cross-table operations.
• Pairwise and final responses cite the correct source documents.
• The response distinguishes aggregate tabular conclusions from narrative claims.

Tests

• CSV vs XLSX
• PDF vs CSV
• XLSX vs DOCX
• one Source with mixed Target types
• duplicate filenames in different scopes
• cross-table shared-key overlap
• schema mismatch and missing join key
• partial target failure
• source authorization revoked before execution
• source version change
• agent/model and streaming/non-streaming parity
• generated artifacts and token aggregation

Rollout

Use enable_cross_format_compare. Start with one Source and one Target, then enable one-to-many mixed Targets after pairwise coverage is stable.

Rollback

Disable cross-format support. Same-type comparisons can continue on their established paths; mixed requests should return a clear validation message rather than silently using the wrong engine.

Exit Gate

Proceed when every source in a mixed comparison is demonstrably processed by its native engine and pairwise/final coverage is complete or explicitly partial.

Not in This Phase

• Many-to-many comparisons
• Exhaustive All Documents comparison
• Automatic comparison target discovery
• Conversation follow-up source reuse

[paullizer]

Implemented Phase 4 foundation in PR #1069 from branch feature/1059-phase-4-cross-format-compare (commit 1f93bc50).

Delivered:

• Default-off enable_cross_format_compare plus staged default-off one-to-many flag.
• Fresh Phase 1 authorization-safe Source/Target manifest; preserves requested target order and role labels.
• Native narrative document-window and tabular runner dispatch with bounded evidence envelopes, source-version coverage, engine labels, artifacts/citations, and partial target visibility.
• Flag-off rollback returns a clear mixed-format limitation rather than treating tables as narrative chunks.
• Version bumped to 0.250.067, with feature documentation and release notes.

Validated: focused Phase 1/3/4 and tabular workflow functional tests, Python compilation, editor diagnostics, scoped BAC/XSS scans, and CRLF-aware diff validation all passed.

Remaining rollout gates: validated structured table overlap/delta operations; expanded personal/group/public/chat authorization and version-change runtime cases; Foundry/streaming parity; and frontend Playwright coverage before enabling either rollout flag.

Refs #1055 #1056 #1057 #1058