Localization: stage translations into the String Catalog (manual)

.gitignore

@@ -140,3 +140,7 @@ Tests/AgentTests/results
 # Generated during localization, not committed: the build-free String Catalog (parity-verified by CI),
 # shelved until the cutover commits it as the runtime backing store at a build-member path.
 WordPress/Resources/Localizable.xcstrings
+
+# Scratch dir for the GlotPress `.strings` the catalog fold reads as input (download_catalog_strings ->
+# localize_catalog). Kept out of the tracked *.lproj tree so the catalog flow never modifies the shipping files.
+/.catalog-translations/

docs/localization-pipeline.md

@@ -45,13 +45,18 @@ The plural reverse-fold (`PluralStrings.fold_translations!`) fills each `(key, l
 
 > **This does not ship machine translations yet.** `Plurals.xcstrings` is built into the app but **not consumed at runtime** — nothing reads the catalog, and nothing references its keys yet; the app still renders plurals the legacy way. The fold *pre-populates* the catalog so it's ready when plurals cut over to it. Until that cutover, the AI plural translations sit in the catalog unused.
 
-## What's deferred: regular strings
+## What's staged, not shipped: regular strings
 
-Regular (non-plural) strings are **not** machine-translated, by design. The app still ships the legacy `WordPress/Resources/<locale>.lproj/Localizable.strings` for them — `Localizable.xcstrings` (`generate_strings_catalog`) is the designated future backing store, but today it's only generated transiently in CI as a coverage check — the file is gitignored, not committed, and nothing ships from it. A machine translation written into the legacy `.strings` would be **live immediately**, and we don't want machine-translated regular strings shipping before the catalog cutover.
+Regular (non-plural) strings still ship the legacy way — from `WordPress/Resources/<locale>.lproj/Localizable.strings`, with no machine translation. A machine translation written there would be **live immediately**, and we don't want machine-translated regular strings shipping before the catalog cutover. `Localizable.xcstrings` (`generate_strings_catalog`) is the designated future backing store; it's gitignored and not a build member, so nothing ships from it.
 
-So regular-string MT waits for the same shape as plurals: once `Localizable.xcstrings` becomes the runtime store, a regular-string **catalog reverse-fold** folds the human translations in and AI-fills the `needs_review` gaps, staged in the catalog (not shipped) until cutover — exactly as the plural fold does today.
+The tooling to **stage** regular-string translations into that catalog now exists, the same shape as the plural fold. `CatalogStrings.fold_translations!` fills each `(key, locale)` cell of `Localizable.xcstrings` as `human ?? AI ?? English` — human ⇒ `translated`, machine / English ⇒ `needs_review` — and is reuse-aware in the same way: a kept machine cell isn't re-translated, and a human translation supersedes it on the next fold. It runs as three manual lanes: `generate_strings_catalog` (extract the English source), `download_catalog_strings` (pull GlotPress into a gitignored scratch dir), and `localize_catalog` (fold the humans in, AI-fill the rest, commit the catalog).
 
-When that's built, two facts established here will carry over:
+Two things keep it from shipping anything today, and both set it apart from the plural fold:
+
+- **Manual, not in the release path.** These lanes aren't wired into `download_localized_strings` or any beta/release step — a run extracts strings, calls the API (cost), and commits a large catalog, so it's run on demand. Only the unit tests run in CI.
+- **Staged, not shipped.** `Localizable.xcstrings` still isn't the runtime store, so the folded translations sit in the catalog unused until the cutover — exactly like the plural catalog.
+
+Two facts the fold relies on, both established by the reverse download:
 
 - **"Undefined by GlotPress" = absent**, not empty. The export omits untranslated strings (`status: current`; verified no empty-valued entries), so absence is the untranslated signal.
 - **Humans always supersede MT**, and machine output never returns to GlotPress — so there's no translation-memory pollution and no manual reconciliation, as long as MT lives in a state-bearing store (the catalog's `needs_review`).
@@ -77,5 +82,5 @@ When that's built, two facts established here will carry over:
 | Brand do-not-translate + per-locale terms | `fastlane/lanes/translation_glossary.rb` |
 | Anthropic SDK glue + Message Batches | `fastlane/lanes/anthropic_batch.rb` |
 | Plural fold (`Localizable.strings` ⇄ `Plurals.xcstrings`) + AI wiring | `fastlane/lanes/plural_strings_helper.rb`, `fastlane/lanes/localization_plurals.rb` |
-| Catalog generation (future regular-string backing store) | `fastlane/lanes/localization_catalog.rb` |
+| Catalog generation + regular-string fold (staged, manual) | `fastlane/lanes/localization_catalog.rb`, `fastlane/lanes/catalog_strings_helper.rb` |
 | Download/upload orchestration | `fastlane/lanes/localization.rb` |

fastlane/lanes/ai_translator.rb

@@ -160,16 +160,26 @@ def translate_plural(english_forms:, categories:, locale:, note: nil, anchors: {
   # strings already sorted by key so each batch naturally groups one feature (reader.*, editor.*) — better
   # terminology consistency within a batch.
   #
+  # A batch spanning many keys is many sequential requests. If one FAILS mid-run (network / rate limit / SDK
+  # error), the batches that already succeeded are KEPT and the run stops — the remaining strings fall back to
+  # English and are retried on the next run, rather than the whole locale's completed (and billed) work being
+  # discarded. Stopping (vs. continuing) avoids hammering the API when the failure is systemic.
+  #
   # @param strings [Array<Hash>] each { key:, source:, comment: } (string or symbol keys both accepted).
   # @param locale [String] target lproj code.
   # @param batch_size [Integer] strings per request.
   def translate_all(strings, locale:, batch_size: DEFAULT_BATCH_SIZE)
     items = batchable_items(strings)
     return {} if items.empty?
 
-    items.each_slice(batch_size).with_object({}) do |chunk, out|
-      out.merge!(translate_batch(chunk, locale))
+    translated = {}
+    items.each_slice(batch_size).with_index do |chunk, index|
+      translated.merge!(translate_batch(chunk, locale))
+    rescue StandardError => e
+      warn_batch_failure(locale: locale, index: index, kept: translated.size, error: e)
+      break
     end
+    translated
   end
 
   # Builds Message Batch jobs for many strings across many locales (the async / cheaper bulk path). Returns
@@ -297,6 +307,14 @@ def to_string_keys(hash)
     (hash || {}).each_with_object({}) { |(key, value), acc| acc[key.to_s] = value }
   end
 
+  # A batch request failed mid-run. Keep the batches that already succeeded and STOP rather than hammer the rest:
+  # the untranslated strings fall back to English and are retried next run (the fold's reuse-awareness means the
+  # kept ones aren't re-billed). Surfaced on stderr — this class is deliberately fastlane-free, so no UI here.
+  def warn_batch_failure(locale:, index:, kept:, error:)
+    warn "AITranslator: batch #{index + 1} for '#{locale}' failed (#{error.message}); " \
+         "kept #{kept} translation(s), the rest fall back to English (retry next run)."
+  end
+
   # One batched request: number the chunk, ask for a JSON {number => translation}, keep the validated ones.
   def translate_batch(chunk, locale)
     numbered = number_chunk(chunk)

fastlane/lanes/ai_translator_test.rb

@@ -4,6 +4,7 @@
 # Uses a canned-reply lambda for `complete:`, so it exercises all of the prompt-building / validation logic
 # without the `anthropic` gem or the network.
 require 'minitest/autorun'
+require 'stringio'
 require_relative 'ai_translator'
 
 # Exercises prompt-building and the validator gate via a canned-reply `complete:` lambda (no gem / network).
@@ -17,6 +18,16 @@ def translator(reply:, prompts: nil)
     AITranslator.new(complete: complete)
   end
 
+  # Runs the block with $stderr captured, returning what it wrote (the class surfaces batch failures via warn).
+  def capture_stderr
+    original = $stderr
+    $stderr = StringIO.new
+    yield
+    $stderr.string
+  ensure
+    $stderr = original
+  end
+
   def test_returns_cleaned_translation
     t = translator(reply: %("Réglages"\n)) # wrapped in quotes + trailing newline
     assert_equal 'Réglages', t.translate(source: 'Settings', locale: 'fr')
@@ -182,6 +193,37 @@ def test_translate_all_bad_json_batch_falls_back
     assert_empty out
   end
 
+  def test_translate_all_keeps_completed_batches_when_a_later_batch_fails
+    calls = 0
+    complete = lambda do |**|
+      calls += 1
+      raise 'rate limited' if calls == 2 # the second batch fails mid-run
+
+      '{"1":"x","2":"y"}'
+    end
+    out = nil
+    warnings = capture_stderr do
+      out = AITranslator.new(complete: complete).translate_all(
+        [{ key: 'a', source: 'One' }, { key: 'b', source: 'Two' },
+         { key: 'c', source: 'Three' }, { key: 'd', source: 'Four' },
+         { key: 'e', source: 'Five' }, { key: 'f', source: 'Six' }],
+        locale: 'fr', batch_size: 2
+      )
+    end
+
+    assert_equal 2, calls, 'must stop after the failing batch, not hammer the remaining ones'
+    assert_equal({ 'a' => 'x', 'b' => 'y' }, out, 'the first completed batch is kept, not discarded')
+    assert_match(/batch 2 for 'fr' failed \(rate limited\)/, warnings)
+  end
+
+  def test_translate_all_returns_empty_without_raising_when_the_first_batch_fails
+    complete = ->(**) { raise 'network down' }
+    out = nil
+    capture_stderr { out = AITranslator.new(complete: complete).translate_all([{ key: 'a', source: 'One' }], locale: 'fr') }
+
+    assert_empty out, 'a total failure degrades to English (empty), and must not propagate the exception'
+  end
+
   def test_translate_all_empty_input_makes_no_call
     called = false
     complete = lambda do |**|