fix(graphics): apply YCbCr->RGB fixed-point descale in progressive decoder/encoder#1400
Open
Rocco De Angelis (rdeangel) wants to merge 2 commits into
Open
Conversation
…coder/encoder The progressive (RemoteFX) decoder produced posterised output against real EGFX servers (xrdp, GNOME Remote Desktop): smooth gradients collapsed to saturated primaries and luma lost its mid-tones. reconstruct_to_rgba() fed the dequantized Y/Cb/Cr into the BT.601 YCbCr->RGB conversion without removing the +5 bits of fixed-point headroom that dequantize_component_ccq intentionally retains (the DWT itself is unity-gain; see tests/dwt_gain.rs). Every channel then exceeded the legal +/-128 range and clamp_u8 pinned it to 0/255. Add the `>> 5` descale plus the 128<<5 luma offset at the colour stage, matching FreeRDP's yCbCrToRGB_16s8u_P3AC4R. Apply the matching `<< 5` on the encoder's RGB->YCbCr path so the crate's encode/decode round-trip stays an identity. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The upstream CI runs clippy with -D warnings, which denies both clippy::print_stderr and clippy::print_stdout. The diagnostic prints were only useful with --nocapture and are redundant with the assert! messages, so remove them entirely.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Against some real-world EGFX servers (xrdp, GNOME Remote Desktop), the progressive (RemoteFX) decoder produces posterised output. Smooth colour gradients collapse to a handful of saturated primaries and luma loses its mid-tones. Spatial detail (text, icons) stays crisp, which points away from a scaling issue and toward colour conversion.
Root cause
reconstruct_to_rgba()passes the dequantized Y/Cb/Cr values into the BT.601 YCbCr->RGB conversion, but the fixed-point headroom thatdequantize_component_ccqintentionally retains isn't removed before conversion.dequantize_component_ccqapplies the spec-defined<< (quant_value - 1)shift. For the wire quant table this works out to +5 bits of headroom, which matches FreeRDP's reference decoder (see the/* -6 + 5 = -1 */comment inlibfreerdp/primitives/prim_colors.c). The colour-conversion stage is the natural place to remove that headroom (R = ((CrR + Y) >> divisor) >> 5). This PR adds the missing>> 5and the corresponding128 << 5 = 4096luma offset. Without them, each channel exceeds the legal ±128 range andclamp_u8pins it to 0 or 255.Worth noting: this isn't the integer inverse DWT. The DWT is unity-gain, verified separately below.
Verification
×2^5scale doesn't originate there. Addedtests/dwt_gain.rsto lock this in.<< (q-1)factors land at +5 bits of headroom (max coefficient scaled ~32×). The +5 bits come from the dequant table.libfreerdp/codec/progressive.candlibfreerdp/primitives/prim_colors.capply the same<< (q-1)shift and the matching>> 5removal at the colour stage, so this patch's math lines up with the reference.>> 5descale applied, xrdp and GNOME RD streams render with correct colour gradients; without it, the same streams are posterised.Scope of this PR
Kept symmetric so the crate's internal encode<->decode round-trip test stays identity:
>> 5plus+128luma offset inreconstruct_to_rgba(matching FreeRDP).<< 5plus offset on the RGB->YCbCr path inrgba_to_ycbcr, so the round-trip doesn't regress.Files
crates/ironrdp-graphics/src/progressive.rs— decoder descale and encoder scale-upcrates/ironrdp-graphics/tests/dwt_gain.rs— regression test confirming the DWT is unity-gain