Add Modal GPU calibration#279

Open

nwoodruff-co wants to merge 25 commits intomainfrom

feature/modal-gpu-calibration

Commits on Feb 20, 2026

Add Modal GPU calibration to speed up CI
nikhilwoodruff
committed
Fix black formatting, dataset.copy(), and add changelog entry
nikhilwoodruff
committed
Reduce peak memory in Modal path by building matrices sequentially
nikhilwoodruff
committed
ci: re-trigger CI after runner preemption
nikhilwoodruff
committed
fix: spawn constituency Modal job before building LA arrays to reduce peak memory
nikhilwoodruff
committed
fix: interleave national/local matrix builds with explicit gc.collect()
nikhilwoodruff
committed
fix: build national matrix once before app.run() to halve peak memory
nikhilwoodruff
committed
fix: pass frs.copy() to create_national_target_matrix to avoid int time_period TypeError
nikhilwoodruff
committed
fix: use serialized=True on Modal function to avoid policyengine_uk import in container
nikhilwoodruff
committed
feat: return per-epoch weight checkpoints from Modal and write calibration logs
nikhilwoodruff
committed
feat: offload imputation pipeline to high-CPU Modal container
nikhilwoodruff
committed
Reduce peak memory by serialising matrices before building next one
nikhilwoodruff
committed
Trigger CI
nikhilwoodruff
committed
Fix Modal CPU image: install system HDF5 deps and pin policyengine-uk-data>=1.40.0
nikhilwoodruff
committed
Enable Modal output for image build debugging
nikhilwoodruff
committed
Fix Modal CPU image: use Python 3.13 for policyengine-uk-data compatibility
nikhilwoodruff
committed
Use uv for CPU image installs; pre-install CPU-only torch to avoid CUDA download
nikhilwoodruff
committed
Fix uv_pip_install index_url args; wrap app.run() with modal.enable_output()
nikhilwoodruff
committed
Fix CPU image: use run_commands with pip to ensure policyengine-uk installs
nikhilwoodruff
committed
Add serialized=True to run_imputation to avoid container importing policyengine_uk_data __init__
nikhilwoodruff
committed
Install deps via uv in CPU image for faster cached builds
nikhilwoodruff
committed
Copy local policyengine_uk_data source into CPU image instead of installing from PyPI
nikhilwoodruff
committed
Fix: use add_local_dir not copy_local_dir
nikhilwoodruff
committed
Upgrade Modal specs: 16 CPU/32GB for imputation, A10G GPU for calibration
nikhilwoodruff
committed
Revert imputation offloading; Modal GPU for calibration only
nikhilwoodruff
committed