Read l10n .properties files as UTF-8 with ISO-8859-1 fallback (#4883)#4885
Merged
Conversation
java.util.Properties.load(InputStream) reads files as ISO-8859-1, so UTF-8-encoded bundles like Bundle_it.properties get mis-decoded -- "à" (0xC3 0xA0) shows up as "Ã ". Users had to run native2ascii to work around this. New PropertiesUtil.loadUtf8WithFallback mirrors JDK 9+ PropertyResourceBundle (JEP 226): try strict UTF-8 first, fall back to ISO-8859-1 if the bytes are not valid UTF-8 so legacy native2ascii / Latin-1 files keep working. \\uXXXX escapes are honored in either mode. Wired into the three places that load locale .properties files: - CN1CSSCLI (Maven build / CSS pipeline) - L10NTask (Ant resource-builder task) - L10nEditor (Designer GUI properties import) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bundle_*.propertiesfiles were displayed incorrectly (e.g. "à" rendered as "Ã ") becausejava.util.Properties.load(InputStream)reads bytes as ISO-8859-1.PropertiesUtil.loadUtf8WithFallbackreads.propertiesfiles as UTF-8 first and falls back to ISO-8859-1 only when the bytes are not valid UTF-8 — mirrors the JDK 9+PropertyResourceBundlebehavior from JEP 226. Existingnative2ascii(pure ASCII +\uXXXX) and legacy Latin-1 files keep working..propertiesfiles:CN1CSSCLI(Maven build / CSS pipeline),L10NTask(Ant resource builder), andL10nEditor(Designer GUI import).Test plan
CSSLocalizationTestextended with three new sub-tests (run via itsmain):caféwritten as Latin-1 bytes) — verifies fallback\uXXXX-escapednative2ascii-style file — verifies escape decoding still worksmvn -pl designer compile -Plocal-dev-javaseclean🤖 Generated with Claude Code