Skip to content

grass.temporal: support fully qualified names on read (#7641)#7670

Open
gkneighb wants to merge 2 commits into
OSGeo:mainfrom
gkneighb:issue-7641-temporal-fqn
Open

grass.temporal: support fully qualified names on read (#7641)#7670
gkneighb wants to merge 2 commits into
OSGeo:mainfrom
gkneighb:issue-7641-temporal-fqn

Conversation

@gkneighb

@gkneighb gkneighb commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Temporal read tools could only access a space time dataset from mapsets on the current search path, even when given a fully qualified name (name@mapset). This makes open_old_stds() resolve a fully qualified name from any accessible mapset, matching how other GRASS data types behave.

Part of #7641.

Approach

  1. Add an idempotent SQLDatabaseInterfaceConnection.add_mapset() that pulls an accessible mapset's temporal database into an existing connection (building on the mapsets support from t.list: Add mapset options for t.list #7631 / temporal: add parameter to skip creation of TGIS DB in current mapset on tgis.init #7645; a no-op when the mapset is already present).
  2. Call it in open_old_stds(), after the connection is initialized, so the dataset's own mapset is part of the connection before the dataset is queried.

Scope

Limited to tools that open datasets through open_old_stds. Tools that build their own database interface and bypass open_old_stds (e.g. t.info, which calls is_in_db(dbif) on an interface it created itself) must ensure the required mapset is added to that interface; that is left as follow-up. The write/create path is unchanged (datasets are still created only in the current mapset).

Test plan

New pytest suite python/grass/temporal/tests/grass_temporal_fully_qualified_names_test.py:

  • add_mapset unit: adds an off-search-path mapset to a connection; idempotent.
  • open_old_stds resolves name@mapset for a mapset off the search path.
pytest python/grass/temporal/tests/grass_temporal_fully_qualified_names_test.py

AI assistance

Implemented with the assistance of an AI coding assistant (Claude), reviewed and verified by the author.

@github-actions github-actions Bot added Python Related code is in Python libraries tests Related to Test Suite labels Jul 12, 2026
@gkneighb

Copy link
Copy Markdown
Contributor Author

Heads-up on the red macOS build check here: it's a pre-existing, unrelated test-isolation flake, not something in this PR. grass_temporal_core_init_skip_db_test.py::test_init_succeeds_without_db_creation depends on a sibling test running first to set the global tgis_backend; it fails standalone and under pytest-xdist even on main (pytest python/grass/temporal/tests/grass_temporal_core_init_skip_db_test.py -n 2). Adding test files in this PR just reshuffled the xdist worker distribution enough to expose it. Root cause is that init(skip_db_init=True) returns before setting tgis_backend (contrary to its docstring). Fix is up in #7671; once that merges I'll rebase this PR and CI should go green.

@ninsbl ninsbl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @gkneighb and thanks for your PR. Extending the SQLDatabaseConnection class with an add_mapset method is one option to solve the issue that I have not thought about before. Could be a good approach.

However, injecting a call to that function to every attempt to fetch data from the TGIS DB is probably not what I would do, as the sql functions may get executed quite frequently, and there we would not want to have avoidable overhead.

I can imagine two slightly alternative approaches.

  1. One approach may be to modify the open_stds functions to identify is the mapset in the ident is in the SQLDatabaseConnection and use your add_mapset function if not.
    I have not checked if all tools use that function set or rather e.g. dataset factory, or even the space_time_datasets We would have to research a bit. The advantade with such a solution on the library level is that we do not have to adjust all the tools.
  2. Another approach would be that all tools that take STDS as input, do the following:
    a) get (all) required mapsets from user input (fully qualified dataset names)
    b) initialize SQLDatabaseConnection with those mapsets (in addition to the current mapset if write access is required).

return
new_mapsets = get_available_temporal_mapsets(mapset)
if mapset not in new_mapsets:
return

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If one actively tries to add a mapset to the SQLDatabaseConnection object, that either does not exist or does not have a temporal database, at least a warning should be given.
However, that of course depends on how the function is supposed to be used. With the pattern below (with possibly frequent calls) a warning would be annoying...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the warning: the existing guard already surfaces this — if the mapset has no accessible temporal DB, add_mapset no-ops and the if mapset not in self.tgis_mapsets: fatal(...) in the query methods raises the usual "no permission ... or no temporal database" error; a genuinely missing dataset falls through to the normal not-found path. A warning inside add_mapset would also fire in the benign "already present" case. If we move the call up to the per-dataset entry points (see the main comment), a targeted warning there is cleaner and I can add it.

@gkneighb

Copy link
Copy Markdown
Contributor Author

On the overhead: agreed the connection's SQL methods (execute/fetchone/fetchall) are hot and shouldn't carry per-call work.

To scope the alternatives I checked which read paths actually reach an off-search-path mapset, and it's broader than open_stds:

  • t.info doesn't use open_stds at all — it builds the dataset via dataset_factory() and calls is_in_db() / select() directly.
  • Some reads issue their own dbif.execute(mapset=self.base.mapset) without going through is_in_db / select either — STRDSMetadata.get_semantic_labels() (metadata.py) and get_registered_maps() plus two other reads in abstract_space_time_dataset.py. t.info -g on a fully-qualified off-path STDS fails specifically at get_semantic_labels().

So Alt 1 (open_stds only) would leave t.info and the metadata reads unfixed.

Proposed middle ground that avoids the per-statement overhead: call add_mapset at the per-dataset read entry points instead of in the connection's SQL methods — i.e. is_in_db() / select() (base.py) plus the direct metadata reads (get_semantic_labels, get_registered_maps, and the two other execute(mapset=self.base.mapset) sites). That runs once per dataset access rather than per SQL statement, stays at the library level (no per-tool changes), and covers the bypass paths.

Does that direction work for you? If so I'll rework it that way.

For reference, I verified coverage on a two-mapset setup (STRDS in an off-search-path mapset, queried from PERMANENT): t.info, t.rast.list, t.rast.univar, t.topology all resolve name@mapset, and off-path writes stay blocked (t.remove still refuses). Full temporal suite passes under pytest-xdist.

@ninsbl

ninsbl commented Jul 13, 2026

Copy link
Copy Markdown
Member

OK, I think it would be reasonable to limit the scope of this PR to:

  1. add the add_mapset method to SQLDatabaseConnection and
  2. use it for now only in open_stds (dataset_factory is also used for e.g. registering single raster maps in STRDS, so I would try to avoid adding mapsets there).

Tools like t.info create the database interface separately, so they would have to make sure that the interface contains connections for the required mapsets. t.info then call database related function with that interface, e.g. is_in_db(dbif). So there the mapset should have been added earlier.

For such cases like it may make sense to create a helper function to check if an ID references a dataset on the search path mapset. But I am not sure where or how such a function should be implemented in detail. Would have to think about it a bit more...

@gkneighb gkneighb force-pushed the issue-7641-temporal-fqn branch from 1122073 to 4079381 Compare July 14, 2026 00:44
@gkneighb

Copy link
Copy Markdown
Contributor Author

Reworked to that scope: add_mapset is now used only in open_old_stds (removed from the connection's SQL methods, and not touching dataset_factory). Fully-qualified names resolve for open_old_stds-based tools; t.info and other tools that build their own interface are left as follow-up (noted in the commit and PR description). Updated the tests accordingly and repointed the PR to "Part of #7641" so it won't auto-close the issue while that follow-up remains. Force-pushed.

gkneighb added 2 commits July 13, 2026 22:49
Add an idempotent method to include a mapset's temporal database in an
existing connection, enabling access to space time datasets outside the
current search path. Part of OSGeo#7641. Authored with the help of an AI
assistant.
open_old_stds() now ensures the dataset's own mapset is part of the database
connection (using SQLDatabaseInterfaceConnection.add_mapset), so a fully
qualified name (name@mapset) can be opened even when the mapset is not on
the current search path.

Scope is limited to tools that go through open_old_stds. Tools that build
their own database interface (e.g. t.info) need to add the required mapset
to that interface themselves and are left as follow-up.

Part of OSGeo#7641. Authored with the help of an AI assistant.
@gkneighb gkneighb force-pushed the issue-7641-temporal-fqn branch from 4079381 to 8c8761e Compare July 14, 2026 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

libraries Python Related code is in Python tests Related to Test Suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants