Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 71 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@

<br />

**Potomatic** is a command-line tool for translating `.pot` (Portable Object Template) files into multiple languages using AI (currently OpenAI). We built it to streamline large-scale localization of WordPress products, a process we detail in [this behind‑the‑scenes article](https://www.gravitykit.com/translating-wordpress-plugins-using-chatgpt/).
**Potomatic** is a command-line tool for translating `.pot` (Portable Object Template) files into multiple languages using AI. We built it to streamline large-scale localization of WordPress products, a process we detail in [this behind‑the‑scenes article](https://www.gravitykit.com/translating-wordpress-plugins-using-chatgpt/).

While [`gpt-po`](https://github.com/ryanhex53/gpt-po) helped us get started, we needed smarter retry logic, cost controls, and better visibility into large jobs, among other things. **Potomatic** delivers those improvements and more, as well adds fine‑grained prompt tuning through a built‑in [A/B testing utility](#-ab-testing-for-prompt-optimization).

Supports multiple AI providers: **OpenAI**, **Google Gemini**, and **Google Translate API**.

## 📢 Disclaimer

Translation quality varies depending on factors such as model selection, prompt design, and the complexity of the source text. **Potomatic** can generate a baseline translation, but the output should always be reviewed and verified before use.
Expand Down Expand Up @@ -40,7 +42,7 @@ For improved results, consider refining your prompt, using a higher-tier model,

## 🚀 Key Features

* **🤖 AI‑powered translations** – Translate into any language supported by OpenAI models.
* **🤖 AI‑powered translations** – Translate into any language supported by OpenAI, Google Gemini, or Google Translate API.
* **📦 Smart batch handling** – Tune batch size, concurrency and retries for the right balance of cost and speed.
* **💰 Cost‑conscious execution** – Accurately estimate costs and tokens, and control the maximum cost of a job.
* **🔄 Incremental & resumable workflows** – Resume interrupted jobs, merge with existing `.po` files, or force a re‑translation.
Expand Down Expand Up @@ -285,6 +287,73 @@ Using this example, "Block Editor" and other terms will not be translated to tar

---

## 🤖 AI Providers

Potomatic supports multiple AI providers for translation:

### Google Gemini

Uses Google's Gemini models for translation. Set the provider to `gemini` and configure your API key.

**Available Models:**

| Model | Prompt Cost | Completion Cost | Best For |
|-------|-------------|-----------------|----------|
| `gemini-3.1-pro-preview` | $0.00125/1K tokens | $0.005/1K tokens | Complex translations, highest quality |
| `gemini-2.5-pro` | $0.0005/1K tokens | $0.0015/1K tokens | High-quality general use |
| `gemini-2.5-flash` | $0.000175/1K tokens | $0.000525/1K tokens | Fast, cost-effective translations |
| `gemini-flash-latest` | $0.000175/1K tokens | $0.000525/1K tokens | Latest optimized flash model |
| `gemini-3.1-flash-lite-preview` | $0.0000375/1K tokens | $0.00015/1K tokens | Ultra-budget translations |
Comment on lines +300 to +306
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Pricing in this table will be wrong as soon as you fix config/gemini-pricing.json.

This README table mirrors the per‑1K rates from config/gemini-pricing.json, which I flagged as significantly below Google’s real pricing. Whatever values you settle on in the config file should be reflected here verbatim to avoid the README diverging from runtime cost calculations. Consider generating this table from the JSON to prevent future drift.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 300 - 306, The README's static pricing table is out
of sync with config/gemini-pricing.json; update README.md so the per‑1K prompt
and completion rates exactly match the values in config/gemini-pricing.json (or
replace the hardcoded table with a build step or script that generates the table
from that JSON at docs build time); locate and change the table rows for the
Gemini models (the current entries for gemini-3.1-pro-preview, gemini-2.5-pro,
gemini-2.5-flash, gemini-flash-latest, gemini-3.1-flash-lite-preview) to reflect
the JSON or implement the generation approach to avoid future drift.


**Usage:**
```bash
# Using Gemini 2.5 Flash
./potomatic -l fr_FR -p translations.pot --provider gemini --model gemini-2.5-flash -k $GOOGLE_API_KEY

# Using the latest flash model
./potomatic -l fr_FR -p translations.pot --provider gemini --model gemini-flash-latest -k $GOOGLE_API_KEY

# Using budget-friendly flash-lite
./potomatic -l fr_FR -p translations.pot --provider gemini --model gemini-3.1-flash-lite-preview -k $GOOGLE_API_KEY
```

### Google Translate API

Uses Google Cloud Translate API for fast, cost-effective translations. Ideal for straightforward text without complex nuances.

**Pricing:** Google Translate API charges based on character count:
- $20 per 1M characters (standard tier)
- Volume discounts available for higher usage

**Usage:**
```bash
./potomatic -l fr_FR -p translations.pot --provider google-translate -k $GOOGLE_CLOUD_API_KEY
```

### OpenAI

Uses OpenAI's GPT models for translation. This was the original provider and offers the widest model selection.

**Available Models:**

| Model | Prompt Cost | Completion Cost | Best For |
|-------|-------------|-----------------|----------|
| `gpt-4o-mini` | $0.00015/1K tokens | $0.0006/1K tokens | Fast, budget-friendly |
| `gpt-4o` | $0.0025/1K tokens | $0.01/1K tokens | High-quality |
| `gpt-4o-mini-2024-07-18` | $0.00015/1K tokens | $0.0006/1K tokens | Specific dated version |

**Usage:**
```bash
./potomatic -l fr_FR -p translations.pot --provider openai --model gpt-4o-mini -k $OPENAI_API_KEY
```

**Environment Variables:**
- `GOOGLE_API_KEY` - For Gemini provider
- `GOOGLE_CLOUD_API_KEY` - For Google Translate provider
- `API_KEY` - Falls back to OpenAI
Comment on lines +350 to +353
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Documented GOOGLE_API_KEY / GOOGLE_CLOUD_API_KEY env vars are not actually wired up.

src/config/index.js only reads API_KEY (via envSchema) and merges that into config.apiKey. There is no provider‑specific env var fallback, so users who follow this section and set GOOGLE_API_KEY=... will hit “API key required” at validation time. Either add the env‑var fallbacks in parseEnvironmentConfig() / createConfiguration(), or correct the README to say only API_KEY works for all providers.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 350 - 353, The README mentions GOOGLE_API_KEY and
GOOGLE_CLOUD_API_KEY but the code only reads API_KEY; update
parseEnvironmentConfig() / createConfiguration() to accept provider-specific env
fallbacks by reading GOOGLE_API_KEY and GOOGLE_CLOUD_API_KEY (via envSchema or
explicit process.env reads) and merging them into config.apiKey when API_KEY is
absent, or alternatively update envSchema to include those keys mapped into the
same config.apiKey value so validation succeeds; look for the functions
parseEnvironmentConfig(), createConfiguration(), and the config.apiKey usage to
implement the fallback logic.


---

## ⚙️ Configuration Files

**Potomatic** uses several configuration files in the `config/` directory to customize its behavior:
Expand Down
28 changes: 28 additions & 0 deletions config/gemini-pricing.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"models": {
"gemini-2.5-pro": {
"prompt": 0.0005,
"completion": 0.0015
},
"gemini-2.5-flash": {
"prompt": 0.000175,
"completion": 0.000525
},
"gemini-3.1-pro-preview": {
"prompt": 0.00125,
"completion": 0.005
},
"gemini-flash-latest": {
"prompt": 0.000175,
"completion": 0.000525
},
"gemini-3.1-flash-lite-preview": {
"prompt": 0.0000375,
"completion": 0.00015
}
},
"fallback": {
"prompt": 0.0005,
"completion": 0.0015
}
Comment on lines +2 to +27
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

Gemini API pricing per 1M tokens 2026 official: gemini-2.5-pro, gemini-2.5-flash, gemini-3.1-pro-preview, gemini-flash-latest, gemini-3.1-flash-lite-preview

💡 Result:

Gemini API Paid Tier pricing per 1M tokens (USD) as of April 2026 from official Google AI for Developers documentation (standard tier, text/image/video input unless noted; output includes thinking tokens; audio input higher where specified; Pro models tiered by prompt length <=200K / >200K tokens): - gemini-2.5-pro: Input $1.25 / $2.50, Output $10.00 / $15.00 - gemini-2.5-flash: Input $0.30 (text/image/video), $1.00 (audio); Output $2.50 - gemini-3.1-pro-preview: Input $2.00 / $4.00, Output $12.00 / $18.00 - gemini-flash-latest: Likely maps to latest stable Flash such as gemini-2.5-flash or gemini-3-flash-preview at Input $0.30-$0.50 / Output $2.50-$3.00 (exact alias pricing follows underlying model; no explicit separate rate found) - gemini-3.1-flash-lite-preview: Input $0.25 (text/image/video), $0.50 (audio); Output $1.50 Free tier available with limits (e.g., requests per day). Context caching and grounding extra. Batch/Priority tiers offer discounts/multipliers. Prices from ai.google.dev/gemini-api/docs/pricing (results 1,17+), Vertex AI docs (4,9,10), and consistent secondary confirmations (2,20,23,26,29). Vertex AI matches Developer API standard rates. "gemini-flash-latest" points to latest Flash release (e.g., 2.5 Flash per models docs), using 2.5 Flash rates conservatively as most cited stable.

Citations:


Pricing values are significantly below Google's published rates — cost estimates and --max-cost enforcement will be inaccurate.

Per Google's official Gemini API pricing (April 2026), the per‑1K-token costs in this file are 2–10× lower than reality:

Model Official (per 1K) This file (per 1K)
gemini-2.5-pro $0.00125 / $0.01 $0.0005 / $0.0015
gemini-2.5-flash $0.0003 / $0.0025 $0.000175 / $0.000525
gemini-3.1-pro-preview $0.002 / $0.012 $0.00125 / $0.005
gemini-3.1-flash-lite-preview $0.00025 / $0.0015 $0.0000375 / $0.00015

Since GeminiProvider.calculateCost() and dry‑run estimation read directly from this config (and the README pricing tables mirror it), a user who sets --max-cost 5.00 could be billed multiples of that on real runs. Note also that 2.5 Pro and 3.1 Pro have a long‑context (>200K tokens) tier at 2× — worth documenting if not modeling.

📐 Corrected values (from official ai.google.dev/gemini-api/docs/pricing)
   "models": {
     "gemini-2.5-pro": {
-      "prompt": 0.0005,
-      "completion": 0.0015
+      "prompt": 0.00125,
+      "completion": 0.01
     },
     "gemini-2.5-flash": {
-      "prompt": 0.000175,
-      "completion": 0.000525
+      "prompt": 0.0003,
+      "completion": 0.0025
     },
     "gemini-3.1-pro-preview": {
-      "prompt": 0.00125,
-      "completion": 0.005
+      "prompt": 0.002,
+      "completion": 0.012
     },
     "gemini-flash-latest": {
-      "prompt": 0.000175,
-      "completion": 0.000525
+      "prompt": 0.0003,
+      "completion": 0.0025
     },
     "gemini-3.1-flash-lite-preview": {
-      "prompt": 0.0000375,
-      "completion": 0.00015
+      "prompt": 0.00025,
+      "completion": 0.0015
     }
   },
   "fallback": {
-    "prompt": 0.0005,
-    "completion": 0.0015
+    "prompt": 0.00125,
+    "completion": 0.01
   }
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@config/gemini-pricing.json` around lines 2 - 27, The pricing constants in
config/gemini-pricing.json are materially lower than Google's published April
2026 Gemini rates, which will make GeminiProvider.calculateCost() and
dry-run/--max-cost checks inaccurate; update each model entry (e.g.,
"gemini-2.5-pro", "gemini-2.5-flash", "gemini-3.1-pro-preview",
"gemini-flash-latest", "gemini-3.1-flash-lite-preview") and the "fallback"
values to the official per‑1K token prompt and completion prices from Google's
Gemini pricing page, and add/annotate the long‑context (>200K) tier (2×
multiplier) for 2.5 Pro and 3.1 Pro where appropriate so cost calculations and
README tables match production billing and enforcement performed by
GeminiProvider.calculateCost().

}
12 changes: 12 additions & 0 deletions config/google-translate-pricing.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"models": {
"default": {
"prompt": 0,
"completion": 0
}
},
"fallback": {
"prompt": 0,
"completion": 0
}
}
32 changes: 20 additions & 12 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
},
"scripts": {
"translate": "./potomatic",
"translate:gemini": "./potomatic --provider gemini",
"ab-prompt-test": "node tools/ab-prompt-test",
"test": "vitest run",
"test:watch": "vitest",
Expand All @@ -45,6 +46,7 @@
"node": ">=18"
},
"dependencies": {
"@google/generative-ai": "^0.24.1",
"chalk": "^5.3.0",
"commander": "^12.0.0",
"dotenv": "^16.5.0",
Expand Down
13 changes: 11 additions & 2 deletions src/config/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,8 @@ export function parseCliArguments() {
.option('--locale-format <format>', 'Format to use for locale codes in file names: `wp_locale` (ru_RU), `iso_639_1` (ru), `iso_639_2` (rus), or `target_lang` (default)', DEFAULTS.LOCALE_FORMAT)

// === Translation Options ==.=
.option('-k, --api-key <key>', 'OpenAI API key (overrides API_KEY env var)')
.option('--provider <provider>', 'AI provider to use (e.g., "openai", "gemini", "google-translate")', DEFAULTS.PROVIDER)
.option('-k, --api-key <key>', 'Provider API key (overrides API_KEY env var)')
.option('-m, --model <model>', 'AI model name (e.g., "gpt-4o-mini")', DEFAULTS.MODEL)
.option('--temperature <number>', 'Creativity level (0.0-2.0); lower = more deterministic, higher = more creative', (val) => Math.max(0, Math.min(2, parseFloat(val))), DEFAULTS.TEMPERATURE)
.option('-F, --force-translate', 'Re-translate all strings, ignoring any existing translations', DEFAULTS.FORCE_TRANSLATE)
Expand Down Expand Up @@ -302,7 +303,15 @@ export function validateConfiguration(options) {
const errors = [];

if (!options.dryRun && !options.apiKey && !process.env.API_KEY) {
errors.push('🔑 API key required (set API_KEY env var, use --api-key, or try --dry-run)');
// Google Translate doesn't require an API key
if (options.provider !== 'google-translate') {
errors.push('🔑 API key required (set API_KEY env var, use --api-key, or try --dry-run)');
}
}

// Google Translate requires source language
if (options.provider === 'google-translate' && !options.sourceLanguage && !process.env.SOURCE_LANGUAGE) {
errors.push('🌐 Source language required for Google Translate (use -s or --source-language)');
}
Comment on lines 305 to 315
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Provider comparisons here are case‑sensitive while ProviderFactory lower‑cases — and the sourceLanguage check is effectively dead code.

Two issues:

  1. options.provider !== 'google-translate' and options.provider === 'google-translate' will both miss --provider Google-Translate / GOOGLE-TRANSLATE, while ProviderFactory.createProvider happily accepts those (it does providerName.toLowerCase()). Result: API key validation will incorrectly fire (or be skipped) depending on case. Normalize once.
  2. options.sourceLanguage is wired in parseCliArguments (Line 209) with default DEFAULTS.SOURCE_LANGUAGE (which itself defaults to 'en'). So !options.sourceLanguage is essentially always falsy unless someone explicitly passes --source-language ''. The Google‑Translate sourceLanguage validation never fires in practice — and the provider's own validateConfig already enforces this via ProviderFactory.createAndValidateProvider. Either drop the duplicate check here, or also strip the CLI default so the absence is detectable.
-	if (!options.dryRun && !options.apiKey && !process.env.API_KEY) {
-		// Google Translate doesn't require an API key
-		if (options.provider !== 'google-translate') {
-			errors.push('🔑 API key required (set API_KEY env var, use --api-key, or try --dry-run)');
-		}
-	}
-
-	// Google Translate requires source language
-	if (options.provider === 'google-translate' && !options.sourceLanguage && !process.env.SOURCE_LANGUAGE) {
-		errors.push('🌐 Source language required for Google Translate (use -s or --source-language)');
-	}
+	const provider = (options.provider || '').toLowerCase();
+	if (!options.dryRun && !options.apiKey && !process.env.API_KEY) {
+		// Google Translate doesn't require an API key.
+		if (provider !== 'google-translate') {
+			errors.push('🔑 API key required (set API_KEY env var, use --api-key, or try --dry-run)');
+		}
+	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/config/index.js` around lines 305 - 315, The provider comparisons in this
block are case-sensitive while ProviderFactory.createProvider lowercases
provider names, causing mismatches; normalize options.provider (e.g., use
options.provider && options.provider.toLowerCase()) before comparing to
'google-translate' so both the API key branch and the provider-specific branch
behave consistently with ProviderFactory.createProvider, and remove the
redundant source-language presence check for Google Translate
(options.sourceLanguage / DEFAULTS.SOURCE_LANGUAGE) since parseCliArguments
already applies a default and ProviderFactory.createAndValidateProvider enforces
required fields—either drop the kludgy validation here or change
parseCliArguments/DEFAULTS handling so absence is detectable if you prefer
keeping this check.


if (!options.targetLanguages || options.targetLanguages.length === 0) {
Expand Down
Loading