Implement virtio-input for keyboard and mouse#122
Conversation
|
The virtio-input feature added in this branch slightly increases boot time: SDL initialization is attempted on every run, and two additional virtio device objects are initialized even in the headless environment used by the netdev test. As a result, the netdev CI test now times out. The netdev test script runs two network-device tests back to back: on macOS, it exercises both user-mode networking and vmnet, each with a 900-second per-statement Although I previously made optimizations for the headless-mode case, they still do not seem to be sufficient. Therefore, for now, I have added a temporary commit to extend the timeout to 30 minutes, so that we can stay focused on the vinput implementation first. |
3784aec to
de65ee8
Compare
|
The macOS netdev CI test timed out again. From the log, it appears to be stuck at |
f0d5e7f to
fbeb8ee
Compare
|
While testing #123 on my own remote, I found that the sound test also experienced timeouts. At the same time, this PR has also shown timeout behavior. These timeouts have occurred in both the netdev and sound tests, but they appear to be random and are difficult to reproduce. After extending the timeout to two hours, I found that the jobs were not consistently hanging, but rather sometimes just running much more slowly. The difference usually starts to appear only after the init process has been launched, and the gap between the fast and slow cases can be quite large. For example, the virtio-input test just took 31 minutes, whereas it normally only takes about 5 minutes, such as in this run. The codebase did not change at all between these two runs. Taking the Based on this, I believe the problem is most likely with the GitHub runner. The fact that #123 also experienced timeouts is a key reason for this conclusion, because #123 does not change the overall architecture at all, it only fixes a value inside virtio-fs, which should not have any impact on execution time. |
This comment was marked as resolved.
This comment was marked as resolved.
Update guest kernel configuration to enable virtio-input. Co-authored-by: Shengwen Cheng <shengwen1997.tw@gmail.com>
71a4803 to
c725884
Compare
jserv
left a comment
There was a problem hiding this comment.
Drop "DO NOT MERGE" commits. Then, I will merge this PR.
Add virtio-input keyboard and mouse devices and connect them to the SDL frontend so host input can reach the guest. Introducing window-driven input requires more than just another virtio device. Once SDL owns the frontend, the emulator needs a threading model that keeps video handling on the process main thread, lets the emulator continue running while the window loop is active, and still allows a window-close request to wake a blocked SMP 'poll(-1)' path. The same shutdown path also needs to work in both normal execution and 'gdbstub'-driven execution. There are a few ways to structure this. One option is to keep the older execution model and bolt the SDL loop onto it. In practice that is a poor fit. The official SDL documentation [1] notes that most graphics backends are not thread-safe and that SDL video functions should only be called from the application's main thread. That matters especially on macOS, and a plain shared close flag is still not enough to unblock an idle emulator. The more workable structure is to reserve the original main thread for the window loop and move 'semu_run()' / 'semu_run_debug()' to a worker thread, with an explicit wakeup path from the frontend back into the emulator. This commit takes that approach. The main thread now stays in the window backend when one is present, while the emulator runs on a worker thread. Shutdown is completed with a wake pipe so closing the window can break a blocked SMP poll and let both sides unwind cleanly. At the device level, this adds virtio-input keyboard and mouse support, including MMIO state, configuration handling, event injection, and integration with the existing virtio-mmio framework. The advertised capabilities are tightened to match what the frontend can actually generate. The mouse is modeled as a relative pointing device with 'REL_X'/'REL_Y' and wheel axes rather than an absolute pointer. On the frontend side, SDL input is translated into Linux input events for keyboard, mouse buttons, relative mouse motion, and wheel scrolling. Repeated SDL keydown events are dropped once 'EV_REP' is advertised so the guest remains responsible for key-repeat timing. Mouse clicks also enter SDL relative mouse mode, and 'Ctrl+Alt+G' releases that grab again so continuous relative motion can be delivered to the guest. At this stage the SDL thread still calls the virtio-input update helpers directly, so shared device state remains protected by 'virtio_input_mutex'. A later commit moves that ownership boundary onto the emulator thread. The guest-side enumeration path is covered by a new virtio-input smoke test in CI, and manual testing still confirms end-to-end event delivery through the guest evdev interfaces. [1]: https://wiki.libsdl.org/SDL3/FAQDevelopment Co-authored-by: Shengwen Cheng <shengwen1997.tw@gmail.com>
Exclude the SMP stamp file, full root filesystem image, and the buildroot, linux, and rootfs working directories so they do not appear as untracked noise.
Set SDL_VIDEODRIVER=offscreen in common.sh so that SDL initialises without a display server, which is required for headless CI runs.
Manual testing of virtio-input currently relies on guest-side tools such as 'hexdump /dev/input/eventX'. That confirms that input reaches the guest, but it is not very convenient when checking how host input is turned into virtio-input events in real time. Relative mouse motion and wheel input are especially tedious to inspect from the raw evdev stream alone. Add a build-time input debug switch for the virtio-input event injection path. When 'ENABLE_INPUT_DEBUG=1' is set in the Makefile, the build defines 'SEMU_INPUT_DEBUG' and enables verbose logging in the virtio-input update helpers. With the flag enabled, the virtio-input update path prints the events that are about to be injected into the guest. The output includes: - keyboard events, including the Linux input key code and evdev value (release, press, or repeat) - mouse button events, including the Linux input button code and pressed state - relative mouse motion events, including dx/dy deltas - mouse wheel events, including dx/dy deltas - drop messages when an input event cannot be delivered because the guest has not supplied buffers to 'EVENTQ'
The current virtio-input path gives the SDL thread direct access to guest-visible virtio-input state. SDL input is translated on the main thread and then immediately forwarded into the virtio-input update helpers, which write EVENTQ, advance the used ring, and set 'InterruptStatus'. At the same time, the emulator thread polls 'virtio_input_irq_pending()' from 'emu_tick_peripherals()', which puts this shared state on a hot path. That ownership model creates an awkward tradeoff. Without synchronization, the SDL thread and the emulator thread can race on the same 'virtio_input_state_t'. With 'virtio_input_mutex', the hot path has to pay for cross-thread locking just to observe whether an interrupt is pending. This commit moves the ownership boundary earlier. SDL input is still collected and translated on the main thread, but it is now stored as bounded host event records instead of updating guest memory, virtqueues, or virtio registers directly. 'emu_tick_peripherals()' drains those records on the emulator thread and dispatches them into the existing 'virtio_input_update_*()' helpers. Guest-visible virtio-input state is therefore owned by the emulator thread, while the SDL thread only owns host event collection. This change also simplifies the interrupt pending path. 'virtio_input_irq_pending()' now only checks 'InterruptStatus', and the vinput-internal mutex is removed because descriptor processing, queue updates, status queue draining, MMIO accesses, and other guest-visible state transitions all run on the emulator thread. At the interface level, the new event queue carries keyboard, mouse button, mouse motion, and mouse wheel input in a host-centric form. This keeps the split between the window side and the emulator side explicit while preserving the existing internal virtio-input event injection helpers.
|
The current CI failure is expected because the Image on the main blob branch is still the old one, and CI does not rebuild the kernel Image. I've manually rebuilt the kernel locally with the current config and tested it by hand. The virtio-input functionality works correctly with the rebuilt Image. |
|
Thank @Mes0903 for contributing! |
This pull request is derived from #34 and #121, implements VirtIO-Input keyboard and mouse devices for semu, enabling host keyboard and mouse input to reach the guest through an SDL-backed window path, in conformance with the VirtIO specification v1.3.
VirtIO-Input
VIRTIO_INPUT_CFG_ID_NAMEVIRTIO_INPUT_CFG_ID_SERIALVIRTIO_INPUT_CFG_ID_DEVIDSVIRTIO_INPUT_CFG_PROP_BITSVIRTIO_INPUT_CFG_EV_BITSVIRTIO_INPUT_CFG_ABS_INFOEVENTQandSTATUSQEVENTQis used for keyboard and mouse event deliverySTATUSQis drained and acknowledged so guest LED updates can make forward progress without stalling the queuePrerequisites
Linux
macOS
Running semu
If SDL is available, semu should open a window for input testing.
Basic guest verification
After the guest has booted, verify that the VirtIO-Input devices are present:
ls /dev/input/event* cat /proc/bus/input/devicesThe guest should expose
/dev/input/event*nodes, and/proc/bus/input/devicesshould report VirtIO input devices.Manual end-to-end verification
Launch semu:
If SDL is available, semu should open a window for input testing. With
ENABLE_INPUT_DEBUG=1, semu also prints the host input events translated by the SDL backend, including:In the guest, verify that the VirtIO-Input devices are present:
ls /dev/input/event* cat /proc/bus/input/devicesOptionally inspect the guest event stream directly:
Focus the SDL window and:
You should see the translated host-side events in semu's debug output, and new records should appear in the guest input event stream.
Automated testing
.ci/test-vinput.shas a smoke test/dev/input/event*exists/proc/bus/input/devicesreports VirtIO input devicesSDL_VIDEODRIVER=offscreenin CI so the guest-visible deviceenumeration path can be validated without requiring a visible desktop
window
Summary by cubic
Implements
virtio-inputkeyboard and mouse over VirtIO‑MMIO with an SDL‑backed window so host input reaches guest evdev. The window runs on the main thread; the emulator runs in a worker and uses a coalesced wake pipe plus bounded per‑device SPSC queues for fast IRQs and clean shutdown (including gdbstub and window close).New Features
EV_REPadvertised, host autorepeat dropped), mouse buttons, relative motion (REL_X/REL_Y), and wheel (REL_WHEEL/REL_HWHEEL). Click window to grab; Ctrl+Alt+G releases.ID_NAME,ID_SERIAL,ID_DEVIDS,PROP_BITS,EV_BITS,ABS_INFO); unknown selectors returnsize=0. DrainsSTATUSQfor LED updates (no host LED control).EV_KEYbitmap from the SDL key map,INPUT_PROP_POINTER, and relative axes only.ENABLE_INPUT_DEBUG=1prints injected events and queue‑drop messages.Dependencies
SDL2; ifsdl2-configis missing,virtio-inputis disabled. CI installsSDL2, exportsSDL_VIDEODRIVER=offscreen, adds.ci/test-vinput.shon Linux/macOS, and extends macOS timeouts.CONFIG_INPUT_EVDEVandCONFIG_VIRTIO_INPUT; the device tree adds two VirtIO‑MMIO input nodes with separate IRQs. CI forces freshImage/rootfs.cpiodownloads to avoid stale artifacts.Written for commit 5c24de0. Summary will update on new commits.