Parent: #1055
Phase 2 - Chat and Search Consistency
Planning baseline version: 0.250.061
Status: Proposed
Depends on: Phase 1
Objective
Make explicit document selections authoritative in ordinary Chat and Search, and combine narrative retrieval with tabular computation in one grounded response.
Chat and Search remain relevance-bounded. They do not exhaustively process every document in All Documents mode.
Current Defects Addressed
chat-messages.js::isWorkspaceDocumentSearchEnabled() derives hybrid_search only from the Search Documents button's active class.- Explicit
selected_document_ids can therefore be sent with hybrid_search: false.
- Backend workspace tabular analysis is gated by
hybrid_search_enabled or history_grounded_search_used.
- Workflow
DOCUMENT_ACTION_TYPE_SEARCH performs narrative search only and has no tabular cohort execution.
- All Documents discovers tabular files only among top-N search chunks.
Product Semantics
Explicit selection
If one or more document IDs are explicitly selected:
- document context is enabled regardless of panel visibility
- narrative sources use bounded hybrid/semantic retrieval
- tabular sources use tabular tools when the question requires source data
- final synthesis receives both evidence envelopes
The backend, not CSS state, is authoritative.
No explicit selection
Chat/Search may use:
- conversation history
- previously grounded authorized sources
- relevance-selected catalog candidates
No mode should enumerate and process the entire catalog merely because All Documents is displayed.
Search action
DOCUMENT_ACTION_TYPE_SEARCH should use the same mixed-source orchestrator:
- narrative: relevant chunks
- tabular: relevant computed facts or schema/query results
- final response: one answer with citations and explicit source coverage
Scope
1. Decouple selection from panel state
Change the request contract to send:
selection_modeselected_document_idsdocument_context_requested
For explicit IDs, document_context_requested is true even if the panel is closed.
Retain hybrid_search as a retrieval preference/compatibility field, not the source-of-truth for whether selected documents matter.
2. Backend context resolution
Derive effective context as follows:
- Explicit selected manifest, if present
- Authorized conversation grounding, if the history assessor says fresh evidence is required
- Relevance-bounded catalog search
Explicit selection always supersedes stale conversation grounding.
3. Mixed Search execution
Run two bounded branches:
- narrative search over narrative IDs
- tabular planning/execution over tabular contexts
Tabular execution must validate coverage for every explicitly selected tabular source. For catalog/relevance mode, coverage is required only for sources selected by the planner.
4. Shared synthesis
Build one system/evidence handoff that clearly labels:
- narrative excerpts and citations
- computed tabular facts and tool citations
- missing/failed sources
- whether evidence was selected, historical, or relevance-derived
5. All Documents relevance
Do not infer all tabular candidates only from the first 12 returned chunks. Use a bounded authorized metadata/schema candidate stage so tables can compete for relevance without processing every table.
Possible candidate inputs:
- filename/title
- tags/classification
- cached schema summary
- workbook/sheet names
Only selected candidates receive analytical calls.
Acceptance Criteria
- Selecting a PDF and XLSX then closing the panel still grounds Chat in both files.
- A plain "summarize" prompt over explicit mixed files includes narrative and computed tabular evidence.
- A question requiring calculation invokes tabular tools for the selected table.
- A narrative-only question does not force unnecessary row-level table processing when history/schema context is sufficient.
- Search Documents active or inactive produces equivalent behavior for the same explicit selection.
- Workflow Search can return evidence from both narrative and tabular sources.
- Search All Documents remains relevance-bounded.
- The answer states partial coverage when a selected source fails.
Tests
- frontend payload with selected IDs and panel closed
- standard and streaming Chat parity
- explicit PDF + XLSX selection
- explicit DOCX + CSV selection in both orders
- question requiring a table calculation
- question answerable from history without tool rerun
- workflow Search mixed sources
- All Documents with relevant table outside initial chunk hits
- tabular planner coverage validation
- personal/group/public/chat-upload mixed scopes
- model and agent runners
Observability
Track:
- explicit-selection context activations
- narrative result count
- tabular candidate and completed-call counts
- mixed synthesis count
- history-sufficient versus evidence-rerun decisions
- sources omitted due to authorization or failure
Rollout
Use a setting such as enable_mixed_source_chat_search. Start with explicit selections, then enable relevance-mode table candidates after telemetry confirms stable coverage and latency.
Rollback
Disable the setting to restore legacy toggle-gated Chat/Search while preserving Phase 1 contracts.
Exit Gate
Proceed when explicit mixed selections work in standard and streaming Chat/Search without requiring the Search panel to remain active, with visible per-source coverage.
Not in This Phase
- Exhaustive Analyze All Documents
- Cross-format Compare
- Per-document workflow mode changes
- Full route-to-service extraction
Parent: #1055
Phase 2 - Chat and Search Consistency
Planning baseline version: 0.250.061
Status: Proposed
Depends on: Phase 1
Objective
Make explicit document selections authoritative in ordinary Chat and Search, and combine narrative retrieval with tabular computation in one grounded response.
Chat and Search remain relevance-bounded. They do not exhaustively process every document in All Documents mode.
Current Defects Addressed
chat-messages.js::isWorkspaceDocumentSearchEnabled()deriveshybrid_searchonly from the Search Documents button's active class.selected_document_idscan therefore be sent withhybrid_search: false.hybrid_search_enabled or history_grounded_search_used.DOCUMENT_ACTION_TYPE_SEARCHperforms narrative search only and has no tabular cohort execution.Product Semantics
Explicit selection
If one or more document IDs are explicitly selected:
The backend, not CSS state, is authoritative.
No explicit selection
Chat/Search may use:
No mode should enumerate and process the entire catalog merely because All Documents is displayed.
Search action
DOCUMENT_ACTION_TYPE_SEARCHshould use the same mixed-source orchestrator:Scope
1. Decouple selection from panel state
Change the request contract to send:
selection_modeselected_document_idsdocument_context_requestedFor explicit IDs,
document_context_requestedis true even if the panel is closed.Retain
hybrid_searchas a retrieval preference/compatibility field, not the source-of-truth for whether selected documents matter.2. Backend context resolution
Derive effective context as follows:
Explicit selection always supersedes stale conversation grounding.
3. Mixed Search execution
Run two bounded branches:
Tabular execution must validate coverage for every explicitly selected tabular source. For catalog/relevance mode, coverage is required only for sources selected by the planner.
4. Shared synthesis
Build one system/evidence handoff that clearly labels:
5. All Documents relevance
Do not infer all tabular candidates only from the first 12 returned chunks. Use a bounded authorized metadata/schema candidate stage so tables can compete for relevance without processing every table.
Possible candidate inputs:
Only selected candidates receive analytical calls.
Acceptance Criteria
Tests
Observability
Track:
Rollout
Use a setting such as
enable_mixed_source_chat_search. Start with explicit selections, then enable relevance-mode table candidates after telemetry confirms stable coverage and latency.Rollback
Disable the setting to restore legacy toggle-gated Chat/Search while preserving Phase 1 contracts.
Exit Gate
Proceed when explicit mixed selections work in standard and streaming Chat/Search without requiring the Search panel to remain active, with visible per-source coverage.
Not in This Phase