Skip to content

audio: reap the pipeline's children when nobody is watching - #211

Merged
pcarrier merged 1 commit into
mainfrom
eng/audio-reap-children
Aug 6, 2026
Merged

audio: reap the pipeline's children when nobody is watching#211
pcarrier merged 1 commit into
mainfrom
eng/audio-reap-children

Conversation

@pcarrier

@pcarrier pcarrier commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #204, which I flagged there while verifying it.

#204 narrowed reap_zombies from draining waitpid(-1) to sweeping only PTY-owned pids. That was the right call — the global drain was reaping other subsystems' children and discarding their statuses, which is what broke the audio pipeline's own try_wait. But it also stopped it covering for two places where the audio pipeline does not collect its own children, and both are live on main now that #204 has merged.

A missed wait. The dbus-daemon exited without printing an address bail-out kills the child and returns without waiting it. Every other bail-out in spawn waits what it kills; this one was missed.

A duty-cycle gap — the same one #204 fixed for terminals. is_alive is what collects dead sub-processes, as a side effect of the try_wait calls it makes to decide whether to heal, and it is only reached from the delivery tick. The tick does not run while no client is attached, so an idle server whose PipeWire died kept the corpse until somebody connected. reap_children collects and does nothing else — none of is_alive's restart behaviour, which is not something to run on a timer nobody asked for — and the supervisor calls it, that loop existing precisely because it runs when the tick does not.

Verification

The dbus path is A/B'd with a fake dbus-daemon on PATH that exits without printing anything, which drives exactly that branch:

  • before: log shows [audio] failed to start pipeline: dbus-daemon exited without printing an address, and the server is left with a dbus-daemon <defunct> child
  • after: same log line, no children at all

The reap_children half is not verified end-to-end. PipeWire does not start on the machine I have (pipewire exited before creating its socket), so there is no live pipeline whose child I can kill and watch get collected. What it does is a try_wait on each stored Child with no other state change, called from a loop that already runs every 5s — but that is an argument, not a test, and someone with working audio should confirm it.

nix run .#lint exit 0, cargo test -p blit-server 287 passing.

Note on scope

There is a third option I did not take: registering the audio children with the same owned-pid mechanism PTYs use, so one reaper covers everything. That is the tidier end state, but it means moving ownership out of AudioPipeline, and this fix is small enough to stand alone and stop the bleeding first.

#204 narrowed `reap_zombies` from draining `waitpid(-1)` to sweeping only
PTY-owned pids, which stopped it stealing exit statuses from the audio
pipeline's own `try_wait`. It also stopped it covering for two places
where the audio pipeline does not collect its own children.

The first is a plain bug: the "dbus-daemon exited without printing an
address" bail-out kills the child and returns without waiting it. Every
other bail-out in `spawn` waits what it kills; this one was missed, so a
dbus-daemon that started and then said nothing stayed a zombie for the
life of the server.

The second is structural, and the same duty-cycle problem #204 fixed for
terminals. `is_alive` is what collects dead sub-processes, as a side
effect of the `try_wait` calls it makes to decide whether to heal — and
it is only called from the delivery tick, which does not run while no
client is attached. So an idle server whose PipeWire died kept the corpse
until somebody connected. Give the pipeline a `reap_children` that only
collects, with none of `is_alive`'s restart behaviour, and call it from
the supervisor, which is the loop that exists precisely because it runs
when the tick does not.

Verified the dbus path with a fake `dbus-daemon` on PATH that exits
without printing: before, the server is left with a `dbus-daemon
<defunct>` child; after, the same log line and no children at all.

The `reap_children` half is not verified end-to-end here — PipeWire will
not start on this machine ("pipewire exited before creating its socket"),
so there is no live pipeline whose child I can kill and watch get
collected. It is a `try_wait` on each stored `Child` with no other state
change, called from a loop that already runs every 5s.
@indent

indent Bot commented Aug 6, 2026

Copy link
Copy Markdown

Issues

Review closed.

View session

@pcarrier
pcarrier merged commit 98bce88 into main Aug 6, 2026
11 checks passed
@pcarrier
pcarrier deleted the eng/audio-reap-children branch August 6, 2026 02:41
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Coverage

Crate Lines Functions Regions
alacritty-driver 72.7% (806/1109) 75.0% (63/84) 76.2% (1310/1719)
browser 0.0% (0/822) 0.0% (0/68) 0.0% (0/1401)
cli 24.9% (2436/9801) 36.4% (300/825) 27.5% (4128/15020)
compositor 12.2% (1221/9999) 22.8% (111/487) 12.3% (1651/13411)
fonts 81.4% (721/886) 88.6% (70/79) 83.0% (1427/1719)
fssync 92.5% (4880/5274) 94.1% (445/473) 92.5% (8926/9647)
gateway 25.5% (375/1469) 29.9% (38/127) 19.2% (470/2448)
git 87.4% (4188/4793) 88.3% (331/375) 87.0% (6626/7614)
lsp 76.0% (2503/3295) 78.2% (248/317) 73.8% (3886/5266)
proxy 19.2% (172/898) 20.5% (26/127) 21.0% (293/1392)
remote 90.3% (9306/10302) 93.2% (681/731) 88.5% (15677/17723)
sd-notify 73.9% (68/92) 100.0% (6/6) 83.2% (109/131)
server 40.4% (7838/19424) 54.9% (775/1411) 42.9% (13261/30885)
ssh 32.2% (165/512) 48.2% (27/56) 31.4% (261/830)
upsidedown 31.4% (391/1247) 27.8% (55/198) 34.8% (797/2287)
webrtc-forwarder 2.7% (72/2624) 2.1% (4/187) 1.2% (50/4335)
webserver 62.1% (1133/1825) 65.9% (164/249) 64.5% (1912/2964)
Total 48.8% (36275/74372) 57.7% (3344/5800) 51.2% (60784/118792)

@pcarrier

pcarrier commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Both caveats from this PR are now closed. Neither needed a code change, but the first one deserved better than the shrug I gave it.

1. reap_children is verified end-to-end

I said PipeWire would not start here. It does — my shell had XDG_RUNTIME_DIR=/tmp (the inherited-from-the-system-server value on this box), and the pipeline needs a real runtime dir. With XDG_RUNTIME_DIR=/run/user/1000 it comes up with all four children. That was my environment, not the machine, and I should have chased it instead of writing the verification off.

A/B, killing the server's own wireplumber with no client attached, so the delivery tick is asleep and only the supervisor can collect it:

build at t+10s
5f26eb65 (before this PR) wireplumber <defunct> — still there
98bce889 (after) reaped, no children in Z

That is exactly the path the PR claims to fix, and it is the one I could not exercise before.

2. Unified child ownership — audited, not needed

The open question was whether to move the audio children onto the same owned-pid registry PTYs use, so one reaper covers everything. I went through every Command::spawn in the tree instead:

So nothing is uncovered, and the refactor would move ownership out of AudioPipeline for no behavioural gain. Dropping it rather than carrying it as a TODO.

One piece of drift found while auditing, not worth its own PR but worth knowing: crates/lsp/src/backend.rs:718 still says "The daemon's global reaper may win the race and steal the status", which stopped being true when #204 narrowed the reaper to PTY-owned pids. The reap_backstop_status it pairs with is already an empty stub (:1994). Happy to delete both in a passing commit if you want the file to stop describing a race that no longer exists.

pcarrier added a commit that referenced this pull request Aug 6, 2026
…217)

Follow-up to #204/#211, found while auditing which subsystems still reap their own children.

#204 narrowed the daemon's backstop from draining `waitpid(-1)` to sweeping PTY-owned pids only. It no longer touches LSP children at all, and three things in this crate still describe the world before that:

- **`reap_backstop_status`** was already an empty stub (`fn reap_backstop_status(_pid: u32) {}`) whose doc comment explained a pid-recycling collision it no longer prevents. It and its two call sites go, along with the `pid` bindings that existed only to feed it.
- **The `try_wait` comment** still warned that *"the daemon's global reaper may win the race and steal the status"*. It cannot. What's left is the ordinary case — an `Err` means the child is already gone — so the comment says that.
- **`docs/design/lsp.md` § Reaping** described the backstop as reaping every child while parking statuses selectively. It reaps selectively now too.

No behaviour change; the stub did nothing.

## On the doc

I rewrote that bullet rather than deleting it, because the consequence is worth keeping and is not obvious: now that the backstop only sweeps what it owns, a subsystem that doesn't wait its own children **leaks zombies** instead of being quietly mopped up. That makes the engine's every-path `wait()` load-bearing rather than defensive.

That's not hypothetical — it's exactly what #211 had to fix for the audio pipeline, where `is_alive` was the only thing collecting children and it only ran from the delivery tick. The LSP engine was already correct here; the note is so the next person adding a subprocess knows the floor moved.

`nix run .#lint` exit 0, `cargo test -p blit-lsp` 39 passing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant