[Req]: Improve Windows guest Retina/HiDPI support and VMSVGA throughput on macOS ARM

[moreaki]

Problem

Windows 11 ARM guests on macOS/Arm hosts are currently awkward to use with Retina/HiDPI displays and VMSVGA 3D enabled.

The local test system showed these specific issues:

• Setting VMSVGA VRAM to 256 MiB simply did not work with the current ARMv8 PCI MMIO layout. The 256 MiB VRAM BAR left insufficient low 32-bit PCI MMIO space for the VMSVGA3 MMIO BAR, so the Windows guest display stack did not initialize normally.
• A 5760x3240 Retina guest mode needs the additional VRAM. With the smaller practical VRAM configuration, resizing beyond a certain maximum screen size could produce gray or colorful display artifacts, consistent with insufficient framebuffer/surface memory for the requested mode.
• macOS fullscreen/autoresize requested logical host sizes instead of the Retina backing-pixel guest mode, so resizing did not scale correctly and the guest did not reliably enter true Retina resolutions.
• The Windows WDDM Guest Additions mode dropdown exposed the native Retina mode and then mostly legacy 4:3/5:4 fallback modes, with no useful 16:9 intermediate Retina modes.
• The macOS/Arm VMSVGA DX11 output path spent too much CPU on repeated readbacks and frontend repaint work.

Patch Series

I have split the work into seven pull requests so each subsystem can be reviewed independently:

1. Expand ARMv8 PCI 32-bit MMIO to support 256 MiB VMSVGA VRAM.
2. Reduce VMSVGA DX11 readback work and skip unchanged output updates.
3. Reduce macOS Qt repaint and Dock preview overhead.
4. Request Retina backing-size guest modes in macOS fullscreen.
5. Add supporting Darwin/Qt build compatibility fixes.
6. Add supporting DXMT/LLVM 17 build compatibility fixes.
7. Add WDDM aspect-preserving Retina intermediate video modes.

Validation

Local validation was done on a Mac Studio with an Apple Studio Display, running macOS ARM64 with a Windows 11 ARM guest:

• vram=256
• 3D acceleration enabled
• VMSVGA VRAM BAR at 0xe0000000..0xefffffff
• VMSVGA3 MMIO BAR at 0xf0400000..0xf07fffff
• Guest Additions run level 3
• guest mode 5760x3240x32

Measured active CPU in a repeated Windows Explorer open/close workload:

Run	Resolution	Active CPU avg
Official VirtualBox 7.2.12 baseline	3840x2160	257.3%
Local test build with PRs #743-#746 and #749, repeat 1	3840x2160	95.1%
Local test build with PRs #743-#746 and #749, repeat 2	3840x2160	97.6%
Local test build with PRs #743-#746 and #749 at Retina backing resolution, repeat 1	5760x3240	77.5%
Local test build with PRs #743-#746 and #749 at Retina backing resolution, repeat 2	5760x3240	68.9%

For the WDDM mode-list change, I also validated the same intermediate modes through the existing driver's custom-mode registry path. Windows Display Settings exposed:

• 5120x2880
• 3840x2160
• 3200x1800
• 2880x1620
• 2560x1440
• 1920x1080

Notes

The Windows Guest Additions WDDM driver patch still needs final compile/install validation in a Windows WDK environment. The local macOS ARM64 build tree cannot build VBoxWddm because PATH_SDK_WINSDK10 is not configured.

[moreaki]

Relevant pull requests for this macOS/Arm Windows guest HiDPI and graphics-performance issue:

1. Expand ARMv8 MMIO32 for 256 MiB VMSVGA VRAM #743 Expand ARMv8 PCI 32-bit MMIO to support 256 MiB VMSVGA VRAM.
2. VMSVGA: coalesce DX11 readbacks and skip unchanged updates #744 Reduce VMSVGA DX11 readback work and skip unchanged output updates.
3. GUI: reduce macOS repaint and Dock preview overhead #745 Reduce macOS Qt repaint and Dock preview overhead.
4. GUI: request Retina backing size in macOS fullscreen #746 Request Retina backing-size guest modes in macOS fullscreen.
5. Additions/WDDM: add Retina-scaled video modes #749 Add WDDM aspect-preserving Retina intermediate video modes.

These are the patches that directly address the behavior reported in #742: high-resolution/Retina operation, VRAM limits, repaint overhead, and excessive graphics update work.

Additional build, documentation, and DXMT/winemetal correctness fixes found during the same investigation are tracked separately in their own issues and pull requests. They are intentionally not listed here so #742 stays focused on the user-visible HiDPI/performance problem.

[moreaki]

Important: the comment at #742 (comment) is not from me and is not part of this patch series.

I did not upload, review, endorse, or ask anyone to download the attached vmsvga_fix_arm.zip. Please treat that attachment as untrusted and ignore it. I have reported the user/comment through GitHub's reporting flow. Maintainers, please remove or flag that comment/attachment if possible.

The only branches and PRs associated with this issue are the pull requests listed in my previous comment from the moreaki/virtualbox fork.

[moreaki]

Fullscreen Retina proof for the macOS fullscreen/autoresize part of the series (#746):

This screenshot was captured from the running Windows 11 ARM guest with VBoxManage controlvm Win11 screenshotpng after entering macOS fullscreen Retina mode.

Validation details from the same run:

vram=256
accelerate3d="on"
VideoMode="5760,3240,32"@0,0 1
GuestAdditionsRunLevel=3

The PNG itself is exactly 5760 x 3240 pixels, matching the guest mode shown in Windows Display Settings.

[moreaki]

Current fullscreen HiDPI dropdown proof for the WDDM mode-list change (#749):

This screenshot was captured from the currently running Windows 11 ARM guest with VBoxManage controlvm Win11 screenshotpng while the VM was in fullscreen and Windows Display Settings had the resolution dropdown open.

Validation details from the same current run:

vram=256
accelerate3d="on"
VideoMode="2880,1620,32"@0,0 1
GuestAdditionsRunLevel=3

The dropdown shows the useful HiDPI/aspect-correct modes, including 5760x3240, 5120x2880, 3840x2160, 3200x1800, 2880x1620, 2560x1440, and 1920x1080.