Skip to content

fix(policy,adapters,plugins): typed errors for every user-authored TOML read (#440, #473) - #688

Merged
pbean merged 7 commits into
mainfrom
pbean/policy-toml-funnel
Aug 22, 2026
Merged

fix(policy,adapters,plugins): typed errors for every user-authored TOML read (#440, #473)#688
pbean merged 7 commits into
mainfrom
pbean/policy-toml-funnel

Conversation

@pbean

@pbean pbean commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

The problem

Three separate parsers read user-authored TOML — policy.loads, adapters/profile.load_profiles,
and plugins/loader — and each has a typed domain error (PolicyError, ProfileError,
PluginError) that every consumer keys its fault handling on. But a file whose content was
wrong escaped as a raw ValueError / TypeError / UnicodeDecodeError instead, walking
straight past the handlers written to survive exactly that:

  • cli.py's _configure_muxexcept (PolicyError, OSError), and it runs before dispatch on
    every subcommand, so one bad character traced back at you from an unrelated command.
  • tui/app.py and tui/screens/dashboard.py — the TUI died at construction rather than falling
    back to defaults and letting the operator open the settings editor to fix the file.
  • tui/settings.pyexcept (PolicyError, PluginError), the degrade that renders the message
    into the settings screen's error line.
  • cmd_validate's role loop — the raw escape hit main's bare-except backstop, which prints one
    error: line and no document, breaking machine.py's one-object --json contract.

Two of the policy faults never announced themselves at all: notify.desktop = "false" is a truthy
string, so bool() turned the feature on; verify.commands = "pytest" iterated a bare string
into six one-character commands that read back as applied configuration.

What changed, phase by phase

# Commit Change
1 d2e728c6, 025d57ca policy crash-class. The four [limits]-only helpers generalized to section-aware _typed_int / _typed_float / _typed_bool / _typed_str(d, where, key, default) (+ _typed_str_tuple); every remaining raw int()/float() in loads() routed through them; _opt_grace/_opt_nudges given inline type guards; array shape guards on verify.commands and both extra_args.
2 cda395d2 policy silent-class. Every remaining bare bool()/str() routed through the typed helpers — 16 boolean fields, 22 string fields, the plugins.enabled entries, and the deprecated-[engine] fold. Allowlist fields keep their allowlist and blame the type first.
3 bcddfed9 profile read guard. _read_profile_text(entry, source) at both load_profiles read sites (packaged + overlay), converting UnicodeDecodeError/OSError into ProfileError naming the file.
4 66d6e2e8 plugin loader read guard + ship (#689). The same _read_manifest_text(toml, source) at both plugins/loader read sites (builtin + project); CHANGELOG.

[limits] messages are byte-identical throughout — all call sites pass where="limits". Valid
policies parse to identical Policy objects: this changes only what happens to invalid values.
usage_grace_s = 30 is still a legal float, and an unset extra_args still means "inherit the
profile's flags" where [] means "none". The lenient snapshot readers (_snapshot_extra_args,
adapter_policy_from_snapshot) are deliberately untouched — they parse the json-round-tripped
RunState.policy_snapshot and must never crash; 025d57ca documents why.

plugins/manifest.py needed no work: its value-coercion funnel already exists and its messages
are pinned by tests. The live plugin gap was the unguarded read, not the parse.

Repros, re-measured on this branch

Repro Before After
[scm] max_parallel = "x" raw ValueError PolicyError: scm.max_parallel must be an integer: got 'x'
[cleanup] run_retention = "x" raw ValueError PolicyError: cleanup.run_retention must be an integer: got 'x'
[limits] dev_contract_nudge = "false" typed already unchanged, byte-identical (unregressed)
[notify] desktop = "false" silently ON PolicyError: notify.desktop must be a boolean: got 'false'
non-UTF-8 profile overlay, validate --project proj --json bare error: line, empty stdout, rc 1 one JSON document at rc 1 with an adapter.profile finding naming the file; stderr empty
non-UTF-8 project plugin.toml raw UnicodeDecodeError through the TUI settings degrade PluginError: plugin <path>: not valid UTF-8: …, settings surface degrades

Verification

  • uv run pytest -q -n logical6397 passed, 53 skipped (6225 before phase 1; +172 cases).
  • uv run pyright — 0 errors. trunk fmt + trunk check --all (258 files) — clean.
  • Every new gate was ablated singly and confirmed to redden, restored from a cp backup with
    an md5 check rather than git checkout. Where a fix had two axes, both were measured and the
    redden-sets are disjoint — which is what proves neither half stands in for the other:
    • call-site axis (phase 4): reverting the project read reddens 2 tests, the builtin read 1.
    • arm axis (phase 4): dropping the UnicodeDecodeError arm reddens 1, the OSError arm 2.
    • phase 2's wiring axis: reverting one call site (scm.keep_failed) to a bare bool() with
      both helpers intact reddens exactly 3 rows while 62 sibling rows stay green.
  • The validate tests assert on the parsed machine_json document, never on rc alone: main's
    bare-except backstop also returns 1, so an rc-only assertion is green with the fix reverted.

Closes #440. Closes #473. Closes #689.

#689 was filed for the plugin-manifest leg after review: #473's repro and contract are both
the validate command, which never reaches the plugin loader, so that leg had no issue of its
own even though it is the same fault at the third and last parser reading user-authored TOML.
Filed rather than split out — the guard is 37 lines in loader.py mirroring the profile guard,
and shipping two of three parsers fixed would leave a known crash on main.

Context, no status change intended: #587 (the earlier [limits] leg this extends), #474
(a duplicate of #440, contributing the validate-document carry test), #278 (where the
truthy-string hazard was recorded).

Summary by CodeRabbit

  • Bug Fixes
    • Policy validation now rejects incorrectly typed TOML values with clear, field-specific errors instead of coercing them.
    • Unreadable or invalid-encoding profile files and plugin manifests now produce structured, file-specific diagnostics.
    • Dashboard startup gracefully falls back to default settings when unrelated policy values are invalid.
  • Validation
    • Expanded coverage for policy fields, profiles, and plugin manifests.
    • JSON validation output now includes actionable findings and an appropriate failure status.

t added 5 commits August 21, 2026 22:01
…fields (#440)

Generalize the four [limits]-only readers into section-aware `_typed_int` /
`_typed_float` / `_typed_bool` / `_typed_str(d, where, key, default)` and add
`_typed_str_tuple` for the argv-shaped arrays, then route every remaining bare
`int()` in `loads()` through them: verify.stream_capture_kb, the four sweep
counters, scm.max_parallel, scm.failed_diff_max_mb, and the two cleanup
retention knobs. `_opt_grace`/`_opt_nudges` return None for "inherit" so they
cannot take a default — they carry the same guard inline.

All ~17 [limits] call sites pass where="limits", so every message #587 pinned
stays byte-identical; scm.preserve_keep folds into `_typed_int`, which is the
guard it already hand-rolled.

The arrays had two failure modes, not one: a bare TOML string is iterable, so
`commands = "pytest"` exploded per character into six one-character commands
that read as applied configuration, while a scalar raised a bare TypeError out
of `loads`. Both now raise PolicyError naming section.key, and unset stays
distinct from empty (extra_args None = inherit the profile's flags).

Every new gate is ablation-verified singly; the pairs redden disjoint row sets.
The validate carry (#474) asserts on the machine_json document, not rc — with
the conversion reverted the backstop returns 1 too, and what actually bites is
stdout carrying no document at all.
…dropped

_snapshot_extra_args is now the one place carrying the pre-fix shape. Its
input is a json round-tripped asdict(Policy), already validated on the way in,
and adapter_policy_from_snapshot wraps it in except Exception: return None
because it feeds display surfaces that must never crash — so a PolicyError
here would only be swallowed, at the cost of blanking the display.
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@pbean, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7051974c-8790-4add-b053-bd00f9b37bf7

📥 Commits

Reviewing files that changed from the base of the PR and between f844c0e and db0aac8.

📒 Files selected for processing (1)
  • CHANGELOG.md

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 67903e4f-bf08-477e-9da6-313ed2b10ae1

📥 Commits

Reviewing files that changed from the base of the PR and between 66d6e2e and f844c0e.

📒 Files selected for processing (1)
  • CHANGELOG.md

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

Policy parsing now rejects wrong-typed TOML values with contextual PolicyError findings. Profile overlays and plugin manifests now convert unreadable or undecodable files into typed errors that identify the affected path. Tests cover CLI validation, loader behavior, and TUI fallback handling.

Changes

Configuration and resource error handling

Layer / File(s) Summary
Strict policy validation
src/bmad_loop/policy.py, tests/test_policy.py, tests/test_cli.py, tests/test_tui_app.py, CHANGELOG.md
Shared validators reject invalid scalar and array types across policy sections while preserving existing range, enum, path, and default behavior. Tests verify structured findings and TUI fallback handling.
Profile read error handling
src/bmad_loop/adapters/profile.py, tests/test_profile.py, tests/test_cli.py, CHANGELOG.md
Packaged and project profile reads convert UTF-8 and filesystem failures into contextual ProfileError exceptions.
Plugin manifest read error handling
src/bmad_loop/plugins/loader.py, tests/test_plugin_loader.py, tests/test_tui_settings.py, CHANGELOG.md
Builtin and project manifest reads convert UTF-8 and filesystem failures into contextual PluginError exceptions. Validation tests preserve file-path diagnostics.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to f844c

The change routes invalid user-authored TOML through typed errors while preserving valid configuration behavior, and no actionable merge-blocking risk remains after normal checks and review.

Poem

A rabbit checks each TOML line,
Rejects wrong types with errors fine.
Profile files name their plight,
Plugins report bad bytes right.
Clear paths and tests, I cheer,
Then nibble a carrot near.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning Plugin manifest error handling and related tests are not required by the directly linked issues [#440] and [#473]. Move plugin manifest changes to a linked issue or a separate pull request, unless the linked issue scope is explicitly expanded.
Docstring Coverage ⚠️ Warning Docstring coverage is 79.63% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 54 functions across 9 files. (1 skipped: 1 unsupported.) Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy [#440] typed policy validation and [#473] typed profile read errors, file diagnostics, JSON reporting, and degradation handling.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: typed errors for policy, profile, and plugin TOML loading.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pbean/policy-toml-funnel

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pbean

pbean commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66d6e2e849

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread CHANGELOG.md
Comment on lines +19 to +20
- **A policy field of the wrong TOML type now raises `PolicyError` naming `section.key`
(#440).** `loads()` coerced with bare `int()`/`float()`/`bool()`/`str()` outside the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Condense the Unreleased changelog entries

Condense these additions to terse, scannable release-note bullets. The two new entries span 36 lines and include implementation history, exception-flow analysis, and exhaustive field lists, making the Unreleased section difficult to scan and directly violating the repository's changelog convention.

AGENTS.md reference: AGENTS.md:L68-L70

Useful? React with 👍 / 👎.

Cut reviewer-facing rationale from the two Unreleased entries: the
rejected-alternative note on why widening the conversion funnel could
not have caught the read fault, the per-handler exception tracing, and
the 15-section enumeration. Every user-visible fact is kept -- the
symptoms, the two silent coercions, both arms of the read guard, and
the compatibility notes.
@pbean

pbean commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: f844c0e772

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@pbean

pbean commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

#473's repro and contract are both the validate command, which never
reaches the plugin loader, so the plugin half of this entry was
attributed to an issue that does not cover it.
@pbean

pbean commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: db0aac814a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@pbean
pbean merged commit 6f58848 into main Aug 22, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant