Fix rounded-rectangle rendering artifacts with high-quality inset stroke

[KlausLoeffelmann]

Summary

Fixes the anti-aliasing artifacts on rounded-rectangle corners rendered by
Graphics.DrawRoundedRectangle and Graphics.FillRoundedRectangle
(System.Drawing.Common, NET9_0_OR_GREATER).

Both methods now render under SmoothingMode.AntiAlias +
PixelOffsetMode.HighQuality (saved and restored so the caller's Graphics
state is not permanently changed). DrawRoundedRectangle additionally insets
the stroke by half the pen width so the whole border sits inside the requested
bounds on a consistent sub-pixel grid. This realigns the corner arcs with the
straight edges and removes the artifact. Public API signatures are unchanged.

Changes

• Graphics.DrawRoundedRectangle(Pen, RectangleF, SizeF) — high-quality inset stroke.
• Graphics.FillRoundedRectangle(Brush, RectangleF, SizeF) — anti-aliased, high-quality fill.
• Added tests: thick-pen rendering and rendering-mode restoration for both Draw and Fill.

Test

System.Drawing.Common.Tests — all *RoundedRectangle* tests pass (9/9).

Fixes #14804

Microsoft Reviewers: Open in CodeFlow

[KlausLoeffelmann]

@Olina-Zhang: Can you take a look?
When drawn with a black border onn bright or worse white background, the arcs did not render correctly.

This should fix the anti-alias artifacts and make it look really clean.
Is kind of urgent, because a lot of things are based off of it.

[LeafShi1]

"this" can be removed

[Olina-Zhang]

Hi @KlausLoeffelmann, the rendering quality looks much better overall after this change, especially the rounded corners on larger rectangles. While testing the PR, we found two possible regressions:

1. Exception when the adjusted radius becomes 0

using Pen pen = new(Color.Black, 4);
g.DrawRoundedRectangle(
    pen,
    new RectangleF(20, 20, 120, 60),
    new SizeF(2, 2));

This throws System.ArgumentException from GraphicsPath.AddArc(). The same also happens with Pen = 16 and Radius = 4. It seems to occur when the adjusted radius becomes (0, 0) after subtracting the inset.

2. Rendering regression for very small rectangles

With Pen.Width = 10 and Radius = 8, the 12×12 rectangle is rendered as a star-like shape after this change, whereas it rendered correctly before. It looks like the inset reduces the drawing area to a very small rectangle while the adjusted radius is still relatively large.

Could you please take a look? Both seem to be edge cases related to the new inset logic.
By the way, I also send these issues in Teams Group for you.

[LeafShi1]

Maybe we can drop the inset logic entirely and keep just the  AntiAlias  +  PixelOffsetMode.HighQuality  change.

1. It crashes on valid input. When the pen.Width = 8 and radius = 4, the strokeRadius will be 0, which causes an exception: System.ArgumentException: 'Parameter is not valid.'
2. It deforms small shapes. The  rect > pen.Width  guard is a hard threshold, so geometry jumps discontinuously as size crosses it. With a 10px pen, a 12×12 rect insets down to a 2×2 path (radius 3) → visibly malformed next to the 8×8/10×10 ones that aren't inset at all.
3. It breaks alignment with  DrawRectangle . Every other  Draw*  centers the pen on the bounds, insetting makes  DrawRoundedRectangle  the odd one out, so mixing it with  DrawRectangle  at the same bounds no longer lines up.