Skip to content

Support retrieved_from: :doc_values for direct leaf fields#1110

Draft
jwils wants to merge 2 commits intojoshauw/fetchable-false-prototypefrom
joshuaw/docvalue-leaf-fetching
Draft

Support retrieved_from: :doc_values for direct leaf fields#1110
jwils wants to merge 2 commits intojoshauw/fetchable-false-prototypefrom
joshuaw/docvalue-leaf-fetching

Conversation

@jwils
Copy link
Copy Markdown
Collaborator

@jwils jwils commented Apr 6, 2026

Summary

This PR adds a narrow secondary retrieval path for fields that should stay returnable in GraphQL while being excluded from stored _source.

t.field "internal_code", "String", retrieved_from: :doc_values

When a field is marked retrieved_from: :doc_values, ElasticGraph:

  • keeps the field in GraphQL output types
  • excludes the field from datastore _source via _source.excludes
  • requests the field through datastore docvalue_fields
  • resolves the field from fields in datastore hits when _source omits it

Implementation

The change is intentionally narrow:

  • only direct fields on indexed root document types are supported
  • only GraphQL leaf fields are supported
  • dotted name_in_index paths are rejected
  • text fields are rejected because they do not support datastore doc values
  • query planning only uses docvalue_fields when every participating index definition agrees on the retrieval method

The runtime metadata now records how an index field is retrieved, the GraphQL datastore query can split requested fields between _source and docvalue_fields, and datastore response helpers can read direct leaf values back from hit fields when needed.

Tests

Updated unit specs cover:

  • keeping retrieved_from: :doc_values fields in GraphQL output types
  • emitting _source.excludes and runtime metadata for those fields
  • requesting docvalue_fields and falling back to _source when index definitions disagree
  • resolving direct scalar and list field values from hit fields
  • rejecting unsupported retrieved_from usages

I was not able to run the RSpec files in this local environment because Bundler is missing required gems/native extensions.

Stack Info: Stacked on #1108

@jwils jwils changed the title Support for direct leaf fields Support retrieved_from: :doc_values for direct leaf fields Apr 6, 2026
@jwils jwils force-pushed the joshuaw/docvalue-leaf-fetching branch 3 times, most recently from ad363a5 to a2b2407 Compare April 7, 2026 19:38
@jwils jwils force-pushed the joshauw/fetchable-false-prototype branch from 8e9b12f to bde522d Compare April 7, 2026 19:42
@jwils jwils force-pushed the joshuaw/docvalue-leaf-fetching branch from a2b2407 to fdf2582 Compare April 7, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant