Add self.upstream property for pre-restricted ancestor access in make()

[dimitri-yatsenko]

Summary

Inside make(), provide self.upstream — a pre-constructed Diagram.trace(self & key) — as a property on auto-populated tables. This gives make() convenient, provenance-safe access to pre-restricted ancestor table expressions.

Context

Discussion: #1232
Depends on: #1423 (Diagram.trace())
Related: #242 (farfetch)

Design

The make(self, key) signature stays unchanged. self.upstream is simply a new property available during make() execution:

def make(self, key):
    # Read from upstream — pre-restricted by key
    session_date = self.upstream[Session].fetch1("session_date")
    traces = self.upstream[ExtractTraces].to_arrays("trace")

    # All query operators work — upstream[Table] returns a QueryExpression
    self.upstream[Session].aggr(self.upstream[Scan], n='count(scan_id)')

    # Write to self
    self.insert1({**key, "result": compute(traces)})

Read from self.upstream, write to self. The read/write boundary is the provenance boundary.

• self.upstream only exposes declared ancestors and their part tables — requesting anything else raises an error
• key is passed as before — no signature changes, no redundancy
• make_kwargs work unchanged
• Existing code that doesn't use self.upstream is completely unaffected

Implementation

• The framework sets self.upstream = Diagram.trace(self & key) before calling make()
• self.upstream[TableClass] returns pre-restricted QueryExpression objects with full fetch capabilities
• Requesting a table outside the declared ancestor graph raises DataJointError

[github-actions]

This issue is stale because it has been open for 45 days with no activity.

[dimitri-yatsenko]

Status update — targeting datajoint-python 2.3 by 2026-06-01 (2026-06-03 at the latest)

Picking this up personally. The stale label is misleading and will be removed.

Depends on #1423. The pair (self.upstream here + Diagram.trace() in #1423) gives make() provenance-safe access to pre-restricted ancestor table expressions, which then becomes the foundation for runtime enforcement in #1425.

Part of the canonical provenance trinity (#1423 / #1424 / #1425) shipping in 2.3 — see the cross-track downstream consumer list on #1423.