Skip to content

Map disability benefit categories in UK data#383

Merged
MaxGhenis merged 1 commit intomainfrom
codex/disability-benefit-categories-data
May 1, 2026
Merged

Map disability benefit categories in UK data#383
MaxGhenis merged 1 commit intomainfrom
codex/disability-benefit-categories-data

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

Summary

  • Add dataset-side mapping from reported PIP, DLA, and Attendance Allowance amounts to PE-UK category inputs using current-law PE-UK rates.
  • Keep disability reported amounts as internal FRS/imputation intermediates, then strip them before saved public datasets.
  • Update public transfer data to emit PIP category inputs directly and remove obsolete disability reported amount uprating rows.

Companion to PolicyEngine/policyengine-uk#1656.

Tests

  • uv run ruff check policyengine_uk_data/datasets/disability_benefits.py policyengine_uk_data/datasets/frs.py policyengine_uk_data/datasets/create_datasets.py policyengine_uk_data/datasets/enhanced_cps.py policyengine_uk_data/datasets/imputations/frs_only.py policyengine_uk_data/datasets/imputations/income.py policyengine_uk_data/tests/test_disability_benefits.py policyengine_uk_data/tests/test_policybench_transfer.py
  • uv run pytest -q policyengine_uk_data/tests/test_disability_benefits.py policyengine_uk_data/tests/test_frs_only_imputation.py policyengine_uk_data/tests/test_policybench_transfer.py::test_policybench_transfer_writes_only_valid_leaf_inputs policyengine_uk_data/tests/test_policybench_transfer.py::test_policybench_transfer_runs_uk_microsimulation
  • PYTHONPATH=/private/tmp/pe-uk-pr1656 uv run pytest -q policyengine_uk_data/tests/test_disability_benefits.py policyengine_uk_data/tests/test_policybench_transfer.py::test_policybench_transfer_writes_only_valid_leaf_inputs policyengine_uk_data/tests/test_policybench_transfer.py::test_policybench_transfer_runs_uk_microsimulation
  • uv run pytest -q policyengine_uk_data/tests/test_policybench_transfer.py

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

MaxGhenis commented May 1, 2026

Addressed cross-PR review findings:

  • Regenerated enhanced_cps_2025.h5 so the checked-in artifact now has pip_dl_category / pip_m_category and no PIP reported amount columns.
  • Added a regression test that loads the checked-in H5 and verifies those columns.
  • Recomputed is_disabled_for_benefits, is_enhanced_disabled_for_benefits, and is_severely_disabled_for_benefits after stage-2 disability amount imputation, with tests.
  • Kept current-law PE-UK rates for category assignment and documented that choice in the mapper.

@MaxGhenis MaxGhenis force-pushed the codex/disability-benefit-categories-data branch from c766af1 to c750776 Compare May 1, 2026 13:13
@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Small follow-up after checking DLA aggregate preservation: category assignment now uses the same baseline DWP thresholds that the removed PE-UK category formulas used. The disability flag recomputation still uses the current-law DWP rates, matching the existing UK-data FRS derivation. This should keep DLA/PIP/AA category assignment behavior aligned with the pre-PR model while still publishing concrete category inputs instead of reported amount intermediates.

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Verified aggregate preservation on the cached private enhanced FRS. I converted the old reported-amount dataset into category inputs with this PR's mapper, then compared current PE-UK on the old dataset against PE-UK #1656 on the converted dataset. Totals are exactly identical for the affected disability benefits: DLA £4.449897bn, DLA self-care £2.936218bn, DLA mobility £1.513680bn, PIP £25.045953bn, AA £7.966628bn.

@MaxGhenis MaxGhenis merged commit bdd69a5 into main May 1, 2026
3 checks passed
@MaxGhenis MaxGhenis deleted the codex/disability-benefit-categories-data branch May 1, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant