Skip to content

Add tee-fd capture mode and capteefd fixtures#14432

Open
FazeelUsmani wants to merge 12 commits into
pytest-dev:mainfrom
FazeelUsmani:tee-fd-capture
Open

Add tee-fd capture mode and capteefd fixtures#14432
FazeelUsmani wants to merge 12 commits into
pytest-dev:mainfrom
FazeelUsmani:tee-fd-capture

Conversation

@FazeelUsmani
Copy link
Copy Markdown
Contributor

Summary

This PR implements --capture=tee-fd mode and capteefd/capteefdbinary fixtures, addressing issue #14159.

The new capture mode enables FD-level capture (including subprocess output) while simultaneously passing output through to the terminal in real-time.

closes #14159

Changes

  • Add --capture=tee-fd CLI option
  • Add capteefd and capteefdbinary fixtures
  • Implement FDCaptureTee and FDCaptureBinaryTee classes using pipe+thread architecture
  • Platform-specific implementations for Unix (using select) and Windows (using queue-based approach)
  • Add comprehensive tests for the new capture mode
  • Update documentation

Test Plan

  • Run pytest testing/test_capture.py::TestFDCaptureTee - 11 tests pass
  • Run pytest testing/test_capture.py::TestCaptureFixture::test_capteefd - fixture tests pass
  • Verified no regressions in existing capture tests

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label May 1, 2026
Copy link
Copy Markdown
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me

This will also need mitigation in xdist

The details that is a pain is the introduction of threads
threads
This is the first active use of threads in pytest

I do also wonder if modern syscalls like sendfile could be used for leaving most of the work to the kernel as i do have a number of unfortunate interactions with select loops

@FazeelUsmani
Copy link
Copy Markdown
Contributor Author

Thanks!

I poked at sendfile/splice. The terminal-passthrough side could use splice on Linux to save a memcpy, but the capture side has to land in a Python bytes object for readouterr(), and the kernel calls can't do that - they only move pages between fds. So a userspace reader stays no matter what. Net effect would be a Linux-only optimisation on one half, not removing the thread.

I can consolidate the two reader threads (one per fd) into a single one for the whole MultiCapture session if that helps. Let me know.

Will open an xdist follow-up after this merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tee-fd/capteefd - file descriptor level capture with pass-through

2 participants