Relative imports from Cython files in cuda_core cause duplicate imports

[mdboom]

If you have code like

from . import typing

in a .pyx file, and another part of the system does:

from cuda.core.system import typing

it causes the typing module to be imported twice.

The first import places the module in sys.modules as cuda.core.cu13.system.typing, and the second import places the module in cuda.core.system.typing. It means that any isinstance checks on classes in this module will fail.

This is clearly a side effect of the "megapackage" approach handled by _import_versioned_module. It is not clear whether we can address it at that level.

At a minimum, maybe we can write a lint or an AGENT.md rule that relative imports should be avoided from Cython modules.

[acosmicflamingo]

Fascinating; I was trying to reproduce this locally but was unsuccessful. After some debugging, I found that the logic in the try block was actually throwing an exception, and noticed the comment that stated # This is not a wheel build, but a conda or local build, do nothing. I should've read that first!

Looking further, I compared the directory structures and noticed that they actually differ depending on which package manager was used to install cuda.core:

# pip package install path for typing.py
lib/python3.13/site-packages/cuda/core/cu13/system/typing.py

# conda package install path for typing.py
lib/python3.13/site-packages/cuda/core/system/typing.py

If my understanding of that function is correct, where it's a way of importing the content within cuda.core.cu<X> namespace in a way where users don't have to actually worry about the versioning when importing, then perhaps the fix for this is to just remove subdir usage from the module and then update globals() with it?

If so, then this could be all that's needed:

diff --git a/cuda_core/cuda/core/__init__.py b/cuda_core/cuda/core/__init__.py
index f2d7c85b62..02fc33ceae 100644
--- a/cuda_core/cuda/core/__init__.py
+++ b/cuda_core/cuda/core/__init__.py
@@ -16,7 +16,14 @@ def _import_versioned_module():
 
     subdir = f"cu{cuda_major}"
     try:
-        versioned_mod = importlib.import_module(f".{subdir}", __package__)
+        relative_mod = f".{subdir}"
+        versioned_mod = importlib.import_module(relative_mod, __package__)
+        # TODO: just set "__name__" and "__package__" instead of iterate?
+        # Clear any mentions of '.cu12' or '.cu13' before updating globals()
+        for k, v in versioned_mod.__dict__.items():
+            if isinstance(v, str) and relative_mod in v:
+                setattr(versioned_mod, k, v.replace(relative_mod, ""))
+
         # Import all symbols from the module
         globals().update(versioned_mod.__dict__)
     except ImportError:

I'd be happy to create a PR with this change (along with tests that assert that sys.modules never contain any modules whose name has a prefix of cuda.core.cu12 or cuda.core.cu13). Unless of course, there is something else I am missing, in which case I can still try and find a fix for this issue :)

[mdboom]

Thanks for the investigation @acosmicflamingo. I think that solution is a little brittle -- updating every string in a module seems ripe for unintended consequences. I think I'd prefer to just update all of the relative imports in cuda_core to absolute ones.

[acosmicflamingo]

@mdboom no problem! Ha, yes it does seem a little brittle. Would you prefer I take this off your hands and continue to investigate and work on it? No pressure of course 😉

[mdboom]

Sorry to miss your note, @acosmicflamingo. I've unassigned myself. If you want to submit a PR to handle this, that would be most welcome (if your offer still stands).

[acosmicflamingo]

No worries :) Hah, yes it does still stand! Here's the PR I created that will hopefully address the issue: #2386