systemd: serialize initrd root fsck with a drop-in

[Pawel Winogrodzki (PawelWMS)]

Temporary workaround

Important

This change is temporary until the upstream e2fsck: take flock(LOCK_EX) on whole-disk device during filesystem check patch is available and integrated into Azure Linux.

Remove this workaround as part of that integration. The outer udevadm lock and the proposed native e2fsck lock must not run together: each opens the whole disk independently, so e2fsck can block waiting for the lock held by its waiting parent and deadlock boot.

What

• Add one drop-in that replaces systemd-fsck-root.service execution with a root-only lock helper.
• Add the helper and a dracut configuration that copies the two custom files into every rebuilt initramfs.
• Add source signatures and prevent the systemd package from claiming dracut-owned parent directories.

Why

The root filesystem check can update ext4 metadata while systemd-udevd probes the same disk. If udev observes metadata during that update, its filesystem probe can temporarily lose the filesystem UUID and remove the corresponding /dev/disk/by-uuid link. The generated root mount then cannot resolve its device, and boot stops in the initramfs.

systemd-udevd already takes a shared whole-disk lock while probing. Running the root filesystem check under an exclusive udevadm lock prevents those reads from overlapping the metadata update.

How it works

Inside the initramfs, systemd-fstab-generator creates systemd-fsck-root.service for the exact root device. The generated unit includes a dependency such as:

BindsTo=dev-disk-by\x2duuid-....device

The drop-in replaces the generated ExecStart with systemd-fsck-root-device-lock. That single-purpose helper:

1. Reads the BindsTo= property of systemd-fsck-root.service.
2. Requires exactly one dev-*.device dependency.
3. Converts the escaped device-unit name back to its /dev/... path.
4. Acquires the exclusive whole-disk lock with udevadm lock --device=<path>.
5. Runs systemd-fsck <path> while the lock is held.

The same unit name also has a static non-initrd form. When no generated device dependency exists, the helper preserves that path by resolving the mounted root through udevadm lock --backing=/ and invoking argument-free systemd-fsck.

Initramfs integration

The systemd RPM installs 50-systemd-fsck-root-device-lock.conf under /usr/lib/dracut/dracut.conf.d. Azure Linux's initramfs package already watches systemd updates and runs dracut --regenerate-all.

During regeneration, dracut reads this configuration from the host filesystem and copies only these custom files into each initramfs:

/usr/lib/systemd/systemd-fsck-root-device-lock
/usr/lib/systemd/system/systemd-fsck-root.service.d/systemd-fsck-root-device-lock.conf

Standard dracut modules already provide systemctl, systemd-escape, udevadm, and systemd-fsck.

split-files.py generates file lists for systemd's RPM subpackages. Its one-line change excludes only the dracut-owned parent directory entries. The configuration file itself remains in systemd-udev and is installed on the final image.

Scope

This change targets only the generated root filesystem check involved in the boot failure. It does not change separate /usr checks or instantiated checks for other filesystems. Draft #18640 demonstrates the alternative generator-layer implementation.

Risk

• Early boot now depends on the helper, systemctl show, systemd-escape, and dracut file inclusion.
• The generated root unit must retain exactly one block-device BindsTo= dependency.
• Lock acquisition has a 180-second ceiling. A timeout prevents fsck from starting and fails the root check.
• This drop-in approach has more packaging and runtime indirection than systemd: serialize generated initrd fsck device probes #18640.

Verification

• Completed a 20-attempt, 10-VM resize campaign with the root check under the whole-disk lock: 390 measured boot transitions, zero UUID removals, and zero root lookup failures.
• Verified device-unit names round-trip through systemd-escape --unescape --path.
• Validated helper shell syntax after Git LF normalization.
• Validated dracut install_items entries and exact staged source hashes.
• Confirmed no custom %check changes remain.
• Passed repository commit and push hooks.
• Completed adversarial Advocate, Skeptic, Architect, and focused helper reviews.
• Full package build was not available locally; draft CI and review are required.