mutate_only_covered_lines crashes on C-extension deps (numpy, libyaml): cannot load module more than once per process

[ejfine]

Summary

When mutate_only_covered_lines = true, the coverage-gathering phase unloads
every module imported during its test run (_unload_modules_not_in), including
third-party C extensions. The subsequent stats run then re-imports them in the same
process. C extensions that forbid re-initialization crash:

• numpy → ImportError: cannot load module more than once per process
• PyYAML C loader (_yaml, used by VCR/vcrpy cassettes) → corrupted parse:
  yaml.constructor.ConstructorError: could not determine a constructor for the tag None

This is not test-specific. The crash depends only on when the C extension first
loads, not on who imports it. It is triggered by any single-init C extension that is
first imported during the coverage run (after mutmut's sys.modules snapshot) and then
re-imported during the stats run — whether that import originates in a test module or in
the source code under mutation. A source_paths module that imports numpy crashes the
same way as a test that imports it.

Setting mutate_only_covered_lines = false avoids it entirely (no coverage pre-run →
the deps import exactly once in the stats run; per-mutant runs os.fork() and inherit
via copy-on-write), which is what points at the unload as the cause.

Environment

• mutmut 3.6.0
• Python 3.13.9 (Linux)
• pytest 9.x, numpy, vcrpy + PyYAML built with libyaml

Reproduction

pyproject.toml:

[tool.mutmut]
source_paths = ["src/mypkg/"]
pytest_add_cli_args_test_selection = ["tests/unit/"]
mutate_only_covered_lines = true

A single unit test importing numpy is enough:

  from numpy.testing import assert_almost_equal

  def test_close():
      assert_almost_equal(1.0, 1.0)

Equivalently, importing numpy from a src/mypkg/ module under mutation (with any test
that exercises it) reproduces the identical crash — confirming the issue is not confined
to the test suite.

mutmut run fails in the stats phase:
ImportError: cannot load module more than once per process
(For libyaml, any vcrpy cassette test produces the tag None ConstructorError instead.)

Root cause

mutmut/code_coverage.py gather_coverage():
modules = dict(sys.modules) # snapshot before the coverage run
cov = coverage.Coverage(data_file=None)
runner.collect_main_test_coverage(cov) # pytest #1 -> imports numpy, _yaml, ...
...
_unload_modules_not_in(modules) # pops EVERYTHING imported during the run
_unload_modules_not_in removes any sys.modules entry not present in the
pre-snapshot — i.e. the entire delta, not just the modules under mutation. The stats
run (run_stats, also in the parent process) then re-imports those C extensions, which
fail or corrupt on second init. (The snapshot is taken before pytest #1, so any
dependency first imported during the run — by a test, a conftest, or a source module —
is in the delta and gets unloaded.)

The unload is intended so the mutated source modules reload fresh in later phases.
It does not need to reload stdlib/third-party deps — and must not, for C extensions
that single-init-guard.

Suggested fix

Scope the unload to modules under source_paths (the only ones that get mutated),
leaving everything else loaded. Roughly:

  def _unload_modules_not_in(modules, *, mutation_paths):
      for name in list(sys.modules):
          if name == "mutmut.code_coverage":
              continue
          if name in modules:
              continue
          mod = sys.modules.get(name)
          file = getattr(mod, "__file__", None)
          if file and _is_under(file, mutation_paths):  # only reload what we mutate
              sys.modules.pop(name, None)
      importlib.invalidate_caches()

This keeps the fresh-reload behavior for the code being mutated while never touching
numpy/libyaml/other C extensions, fixing the crash without disabling
mutate_only_covered_lines.

[Otto-AA]

Hi, thanks for raising and explaining the issue!

Scope the unload to modules under source_paths (the only ones that get mutated),
leaving everything else loaded.

I like the idea, but I think we would get a problem if the top-level code of a source module changes some (global) state in a 3rd party library.

Some examples: (I only verified the logging example, SQLAlchemy output by the AI but makes sense at least conceptually)

Logging has a global logger. Following code would register the logging handler twice:

# src/main.py
import logging
logging.getLogger().addHandler(logging.StreamHandler())

logging.getLogger().setLevel(logging.DEBUG)
logging.info("Hello")

Libraries sometimes provide global event listening. Here's a (not double checked example by the AI) of SQLAlchemy:

# src/main.py
from sqlalchemy import event
@event.listens_for(Engine, "before_cursor_execute")
def log_query(query): print(query)

So in general, when we load the 3rd party library only once but the source code twice, we may register functions too often. We would need to reset the 3rd party library as well (which is what we currently do).

I guess a proper fix would be to refactor the coverage to run in a separate process, such that the modules are not shared imported into the main mutmut process. Then we do not need to unload them, which makes our life easier in this regard. But this likely needs some more communication between the coverage utility and the main mutmut process. I did not look much into this yet, so maybe I'm missing something, but I think isolation would be the direction we need to go to, to prevent further issues.