Cross platform scaffolding

[LikeTheSalad]

These changes attempt to abstract Windows-specific functionality into interfaces and create implementations for Windows. The functionality should remain the same, however, it sets up a scaffolding for adding new platform support in the future.

Since I'm not using Windows, the validations I ran are the existing unit tests, so it's probably wise to try these changes out in a Windows machine to make sure nothing's broken (in case some use cases aren't covered by tests).

[petoncle]

Thanks @LikeTheSalad! I'll look into it. There are little unit tests unfortunately, the hard parts are difficult to test (keyboard/mouse OS interaction and overlay UI).
Are you interested in a Linux version or macOS? Did you ever use mousemaster on Windows?

[LikeTheSalad]

Thanks @LikeTheSalad! I'll look into it. There are little unit tests unfortunately, the hard parts are difficult to test (keyboard/mouse OS interaction and overlay UI). Are you interested in a Linux version or macOS? Did you ever use mousemaster on Windows?

Thanks 🙏 I'm interested in a macOS version. I still haven't tried to make it work locally, so I'm frankly not sure if all the Windows apis have a 1-1 alternative here, but still, fully isolating the existing platform should make it easier to test it later. I still haven't tried mousemaster before, but from the docs, it seems the most feature-rich tool available out there, which is why it'd be nice to use it on other platforms.

[petoncle]

Forgot about your PR, sorry. I just reviewed and it's looking good, thanks a lot for this. I just tested quickly and it's working fine.
I just have some notes I took while reviewing:

• WindowsKeyboardAdapter implements Keyboard: Instead of WindowsKeyboardAdapter, maybe it could be WindowsKeyboard implements Keyboard. And we'd remove static methods from WindowsKeyboard. And I think it needs to be called KeyboardController instead of Keyboard, because for Mouse there is a conflict (there's a Mouse class that already exists).
• Same for WindowsMouseAdapter implements PlatformMouse: could it be WindowsMouseController implements MouseController?
• Same for WindowsOverlayAdapter: WindowsOverlay implements Overlay
• Same for WindowsUiAutomationAdapter: WindowsUiAutomation implement UiAutomation
• WindowsKeyRegurgitator implements KeyRegurgitator: I'm not sure if it needs to be platform-aware (for now at least?), i.e. it could stay as one single KeyRegurgitator class.
• Question I asked myself regarding the enum WindowsVirtualKey. Maybe it's possible to make this enum cross-platform, i.e. the enum would be VirtualKey. This would avoid the need to carry around the enum name, we could just carry around the enum instance instead. We'd keep keyFromVirtualKey() instead of keyFromVirtualKeyName(); we wouldn't need KeyboardLayoutKeyDeserializer.

[LikeTheSalad]

Thanks for the feedback, @petoncle

• WindowsKeyboardAdapter implements Keyboard: Instead of WindowsKeyboardAdapter, maybe it could be WindowsKeyboard implements Keyboard. And we'd remove static methods from WindowsKeyboard. And I think it needs to be called KeyboardController instead of Keyboard, because for Mouse there is a conflict (there's a Mouse class that already exists).
• Same for WindowsMouseAdapter implements PlatformMouse: could it be WindowsMouseController implements MouseController?
• Same for WindowsOverlayAdapter: WindowsOverlay implements Overlay
• Same for WindowsUiAutomationAdapter: WindowsUiAutomation implement UiAutomation

I removed the adapters and, regarding the names, my understanding is that you'd like to keep some consistency while avoiding clashings. So regarding MouseController, I noticed that there's an existing class named like that, so to try and keep the new types consistently named and avoid conflicts, I went with the Platform* prefix. I'm open to suggestions in case that doesn't work for you.

WindowsKeyRegurgitator implements KeyRegurgitator: I'm not sure if it needs to be platform-aware (for now at least?), i.e. it could stay as one single KeyRegurgitator class.

Done

Question I asked myself regarding the enum WindowsVirtualKey. Maybe it's possible to make this enum cross-platform, i.e. the enum would be VirtualKey. This would avoid the need to carry around the enum name, we could just carry around the enum instance instead. We'd keep keyFromVirtualKey() instead of keyFromVirtualKeyName(); we wouldn't need KeyboardLayoutKeyDeserializer.

This one's tricky because it's not possible to reuse the same codes across platforms, and keyboardLayout wasn't easy to make platform-agnostic. So what I did was to abstract it and create some Window-specific implementation. This is a big change (added in my latest commit) so I can revert it in case it's too much for this PR.

Btw, the refactorings were substantial so those were AI assisted. Let me know what you think 👍

[petoncle]

Great work! Many thanks. I have one comment about the keyboard layout stuff. Let me start by trying to explain how things are architected.

I scraped keyboard layout data (KbdlayoutinfoParser) from https://kbdlayout.info/ because there was no reliable way to get it from the Windows API. It outputs KeyboardLayout objects holding a List<KeyboardLayoutKey>:

record KeyboardLayoutKey(int scanCode, WindowsVirtualKey virtualKey, Key key,
                         String text, String name) {}

scanCode identifies the physical key, independent of layout. virtualKey identifies the key after a layout is applied: scan code 0x10 reports as VK_Q on US QWERTY but VK_A on AZERTY.

I only briefly looked into it, but I believe this record could be extended with the physical-key identifiers macOS/Linux use (W3C KeyboardEvent.code "KeyQ", macOS keycode, X11/XKB code, Linux evdev code, USB HID usage), all derivable from the scan code by joining Chromium's dom_code_data.inc, whose rows are DOM_CODE(usb, evdev, xkb, win, mac, code, id); win is our scan code.

The Key record is the OS-independent and layout-independent key representation:

record Key(String staticName, String staticSingleCharacterName, String character) {}
// Two examples:
static final Key esc = new Key("esc", "⎋", null);
static final Key plus = new Key("plus", null, "+");

It is identified by a static name or by a produced character because no single identifier fits every key, and what a user writes in a config to refer to a key must be typeable and stable across layouts. Layout-independent keys (esc, f1, arrows, modifiers) sit in the same place on all layouts and mostly produce no character, so they get a staticName. Layout-dependent keys (letters, digits, punctuation) produce different characters per layout, so they are identified by character: +a means "wherever a lives on the active layout". Key holds no scanCode/virtual key so combo matching, macros, and config parsing stay platform-agnostic; KeyboardLayoutKey is the only place that maps a Key to and from the OS-level scanCode/virtualKey.

All of that to say instead of WindowsKeyboardLayout, I think we could have a KeyboardLayout that holds a List<KeyboardLayoutKey>, and a KeyboardLayoutKey holds not only Windows-only identifiers (scanCode + virtualKey), but also macOS and Linux identifiers. But extending the fields of KeyboardLayoutKey is probably some separate work, not in this PR. And if we do that (no WindowsKeyboardLayout, just one KeyboardLayout platform-agnostic class), then we can also get rid of KeyboardLayoutProvider. Let me know if that makes sense.

[petoncle]

Now about naming, this is what I would suggest:

• PlatformClock -> Clock
• PlatformOverlay -> Overlay
• PlatformUiAutomation -> UiAutomation
• PlatformKeyboard -> KeyboardController
• WindowsKeyboard -> WindowsKeyboardController
• PlatformMouse -> MouseController
• WindowsMouse -> WindowsMouseController

[LikeTheSalad]

Thanks for the quick response, @petoncle. I addressed all your comments in the latest commit (again, it's a lot of changes, so the work is AI-assisted). Let me know if some further changes are needed.

[petoncle]

Thank you! I just realized there was an existing MouseController class which I forgot about. I renamed it to MouseManager. I also updated the GraalVM native-image build files with the new class names and packages.