Conversation
* implement most of issue 1 * update cp_surface * correct period meaning * update docstrings * finish issue 1 * minor comment updates * remove turbine_key mirrors change in resgroup/hill-of-towie-open-source-analysis#53 * address PR comments * Update hill_of_towie.py
* issue 2 initial effort * Simplify docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * add example_hot_study.py * address PR comments * improve kwargs enforcement --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* issue 3 WIP * Update example_v0_study.py * fix md has different turbines than cfg.asset.wtgs * write out tdf * add missing save_plots guards * fix lint * remove HOT details from era5.py * fix format * fix test_explicit_mplbackend_is_respected * address PR comments
* update issue 4 * WIP * cache per year, turbine * format * improve naive plots * improve naive plots * improve issue 4 descr * rename example prepost * rename example prepost * rename example prepost * run fast methods first * fix TypeError * address PR comment * address PR comment
* refactor out scadadf * improve e2e tests * address PR comments * fix test flake
* update issue text * RLearner WIP * RLearner WIP * RLearner tested against naive * update gitignore * add overnight scripts * fix typo Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * improve overnight scripts * add findings.md * address PR comments --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* add inspect_prepost_hard_case.py * add diagnostics modules * mandatory availability filter * fix naive plot bug * fix docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * update findings.md * address PR comments --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* power_model * update docstrings * improve histograms * address PR comments
* add study_power_model_compare.py * add study_power_model_compare baseline * uplift by condition WIP * update power_model baseline results * add by condition diagnostic plots * add by condition diagnostic plots * update findings.md * plan issues 6-8 * address PR comment
* add prediction clip * add condition dependent comparison * add _select_profiles * add --accept-candidate * update baseline * address pr comments
* add ERA5 future work * issue 8 substantially complete * address PR comments * accept new conditional uplift method * update issue 8 text * address PR comments
* issues 9-11 * address PR comments * explore feature removal * improvements from feature removal * address PR comments
* issue 12 * address PR comments
* issue 13 complete * further investigate half life * address PR comments * skip rlearner e2e tests * address PR comments
* issue 14 WIP * add 1 and 2 months to baseline * accept matched count 50 * issue 14 full benchmark * address PR comments
* issue 15 WIP * re-number issues * re-number issues * fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* prune opt-in behaviour * update docstring
* update findings.md * add power conditional WIP * add power conditional WIP * improve conditional diagnostic plots * add power plot * rerun study_power_model_compare * address PR comment
* Update naive_ratio.py * Update method.py * make benchmarking importable * use shared build_toggle_df * fix log bug * address PR comments
* make columns required * address PR comments
* add ToggleSpecialistMethod * Update test_naive_ratio.py * Update toggle_specialist.py * complete issue 17 * feat: add campaign_weeks grid + toggle-methods regression harness Adds a weeks-based campaign-length grid alongside the existing months grid, and a study script that regression-tests the two toggle-capable methods (toggle_specialist, power_model) on Hill of Towie. The weeks grid is additive: StudyConfig takes exactly one of campaign_months / campaign_weeks, and leaderboard/plot_campaign_curves take a length_col defaulting to campaign_months. Every existing months-based call is therefore bit-identical, so the committed power_model baseline stays valid. months-only accessors (CampaignWindow.months, StudyConfig.max_activity_months) raise on a weeks study rather than silently reporting weeks as months. study_toggle_methods_compare.py scores a placebo plus a symmetric +/-2% Cp pair over 1/2/4/8 weeks -- the small-signal, short-campaign regime a real toggle campaign lives in. It reports raw deltas against a committed benchmark: ground truth is deterministic in (config, seed), so an unchanged method must diff to exactly 0.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * add validate_conditions * Create study_toggle_methods_compare_baseline.json Recorded on f6b509b's method code. The stamp reads 04f36d6-dirty because validate_conditions (a pure, unwired addition) landed mid-run and test files were uncommitted; neither method's source differed. * enforce kwargs * add binning to toggle specialist * Update density.py * update vocab * update study_toggle_methods_compare_baseline.json * address PR comments * Update study_toggle_methods_compare.py * Update study_toggle_methods_compare_baseline.json * V1 ts uncertainty (#127) * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * toggle specialist uncertainty WIP * clean up baseline loose end * address PR comments * add labeled_rows * make power bins symmetric * address PR comments * format --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* exclude_row WIP * address PR comments
* WIP * WIP * WIP * address PR comments * address further PR comments * add inspect_wake_steering_case.py * create issues_campaigns.md
…133) Drop the rlearner package entirely, carrying forward only the shared piece power_model still needs. - Relocate make_outcome_model (+ its _COMMON params and lazy lightgbm import) verbatim from rlearner/nuisance.py into power_model/fitting.py; repoint power_model/method.py and inspect_era5_matching_importance.py. Estimator construction is byte-identical, so power_model behaviour is unchanged. - Carry over the factory's direct unit test into test_power_model_fitting.py (TestMakeOutcomeModel) so relocating it does not drop its unit coverage. - Delete the rlearner package, its seven tests, and the rlearner-only inspect_prepost_feature_ablation.py. Rename the "rlearner" fixture labels in test_plots.py (they were plain strings, no import). - Drop the now-stale rlearner references in era5_sync.py and method.py docstrings. Acceptance: poe all-fast green; power_model reads UNCHANGED against both committed benchmarks (study_toggle_methods_compare max delta 0.06 pp; study_power_model_compare 0 material moves), i.e. identical to within LightGBM's same-machine noise floor. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…nt drivers (#134) Post-C7 housekeeping ahead of the campaigns tranche. - Reword all "R-learner" references left behind by the rlearner package deletion (C7) to describe current behaviour. Notably the overnight studies advertised "oracle + naive + R-learner + v0" but now run power_model. - Strip finding-number citations (F1/F5/F13/F30/...) from source and tests per the CLAUDE.md rule; the rationale stays in docs/v1/findings.md. - Move spent one-off / manual inspection drivers into benchmarking/baselines/old/: migrate_toggle_baseline_v2_to_v3, inspect_era5_matching_importance, inspect_short_campaigns, inspect_naive, inspect_v0_run (no tests, not imported by the active tree). poe all-fast green; no scored-method or benchmark behaviour changes. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* W0: src/ layout — rename legacy wind_up to wind_up_v0, add v1 wind_up skeleton Adopt a src/ layout so the v1 package can claim the `wind_up` import name while the legacy tool is retained as `wind_up_v0` (distribution stays `res-wind-up`; only import names change). - move legacy `wind_up/` to `src/wind_up_v0/`; add a minimal `src/wind_up/` v1 skeleton (docstring + __version__ + py.typed) for W1 to fill - repoint every importer (v0_binned baseline, benchmarking, tests, examples, and the smarteole notebook) to `wind_up_v0` - constants.py PROJECTROOT_DIR parents[1] -> parents[2] (repo root is one level deeper) - pyproject packaging (where=["src","."]), coverage source, ruff paths, CODEOWNERS - document Git LFS as a prerequisite; gitignore the smarteole example download Behaviour-preserving: the v0 output-schema key "wind_up_version" is unchanged, poe lint is green (mypy 120 files), 819 tests pass, and the smarteole/wedowind example plots and numbers are byte/pixel-identical before and after. benchmarking/ stays packaged temporarily (imported by a separate project); dropping it from the release artifact and deleting the legacy config/input_data/cache root folders are deferred to W2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * update v1 README * W0 review: fix 3.10 mypy, repoint README to wind_up_v0, harden baseline guards Address PR #135 review feedback: - benchmarking/synthetic/upgrades.py: pin shape-agnostic npt.NDArray[np.float64] annotations on cp_ratio/ws_factor/nacelle_delta and cast the _is_waked argument to float64, so mypy passes under the numpy resolved on Python 3.10. This lint failure predates W0 (it came in with the wake-steering work and the v1 branch was already red on 3.10); the fix is type-only, no runtime change. - README.md: repoint the quick-start imports from `wind_up` to `wind_up_v0`. The `wind_up` package is now the empty v1 skeleton, so the old snippets raised ModuleNotFoundError. - src/wind_up_v0/py.typed: PEP 561 marker for the retained, fully typed legacy API. - test_naive_ratio / test_toggle_specialist: the independence guards now reject both `wind_up` and `wind_up_v0`, so an "independent" baseline cannot quietly import either. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* begin C1 work * C1: campaign declaration + runner + farm uplift + placebo campaign Stands up the whole declaration -> runner -> farm-uplift -> reporting pipeline on a placebo (zero injected uplift) whole-farm campaign, so a campaign is declared rather than hand-wired. - src/wind_up/farm.py: farm_uplift(), the pure headline function, with the capacity-factor cap and non-negativity floor, per-turbine guard flags and spread. - benchmarking/campaigns/: SyntheticCampaign (private, holds the injected upgrades) derives CampaignSpec (public facts only); CampaignRunner; the report; and the two placebo campaigns with their driver. - true_farm_uplift(): the N-turbine generalisation of true_net_uplift. - conditional_truth_vs_estimate moved from inspect_prepost_hard_case into harness/plots.py, so campaigns/report.py does not import the v0 pipeline for a frame-shaping helper. Re-exported, so existing callers are unchanged. Consumers read per-turbine facts through spec.timing_for()/usable_mask() and the mode through spec.mode, so C8's per-turbine change histories change accessor bodies rather than every call site. Results on real Hill of Towie SCADA are logged in docs/v1/findings_campaigns.md (CF1-CF5): truth is exactly 0 in both modes; toggle beats prepost by an order of magnitude; the farm result reaches +0.148% with six test turbines; and T06 is identified as the best failure-mode fixture turbine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: drop "treated" from src, and "headline" -> "result" PR #136 review. Two terminology changes in the product surface: - TurbineUplift.treated_energy -> actual_energy. The codebase already pairs "actual" with "counterfactual" -- power_model documents its headline as ``sum_actual / sum_counterfactual - 1`` -- and farm.py already had counterfactual_energy, so "treated" was the odd one out. The docstring formula now reads the same way in both places. "actual" is also neutral for the analyses C8 covers that are not upgrades: confirming stable performance, or quantifying a loss event. - "headline" -> "result" throughout, since a farm is not necessarily the top level of aggregation. src/wind_up/ no longer contains "treated". The benchmarking layer keeps it (including the shared treated_mask helpers) for C8 to sweep once the neutral umbrella term is settled, rather than leaving a third vocabulary in play. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Review: guard non-finite energy and unusable ratings in farm_uplift Copilot review on #136. The guards accepted non-finite actual energy and non-finite or non-positive rated power, and each failed in its own way: - NaN energy passed with used=True and no guard, then vanished from the result entirely, because the summation skips NaN. The farm number looked clean and n_guarded reported 0. - Infinite energy produced an infinite farm result. - A negative rating made the capacity cap clip the counterfactual to a *negative* value, violating the documented non-negativity floor, and gave a result below -100%. - A NaN rating silently disabled the cap. All four are now dropped with explicit guard reasons (non_finite_energy, invalid_rating), so they are visible in n_guarded and the per-turbine detail rather than silent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: pool each method's farm truth over the turbines it actually used Copilot review on #136. farm_uplift can drop a turbine (a non-finite estimate, say), so a method's farm estimate may cover a subset of the campaign while its truth was pooled over every upgraded turbine. signed_error then subtracted one estimand from another -- and it did so precisely when a method misbehaved, which is when the number matters most. Each farm row's truth is now pooled over that method's used turbines. n_guarded flags the rows where turbines were dropped, since a method that dropped some is not directly comparable with one that used them all. CampaignResult.truth_farm_uplift is unchanged: it remains the all-turbine campaign truth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: excluded turbines are not candidate references; ERA5 only when needed Copilot review on #136, two independent placebo fixes: - candidate_references included excluded turbines, so the spec simultaneously offered a turbine as a reference and forbade its use. Behaviour was correct because the runner drops those rows, but C3 is "automatic reference selection from the CampaignSpec" and would have read the contradictory field. - build_hot_v0_context fetches ERA5, and only the power model reads it, so include_power_model=False still paid the network dependency. It is now built only when the power model is enabled, which makes the fast path genuinely offline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Review: plot only the conditions a method actually reported Copilot review on #136. The report built truth for every condition whenever a method reported any conditional output, so a method reporting one axis still got plots for the others. toggle_specialist reports only "power", so every toggle run wrote ws and ti charts whose method series was entirely NaN while being labelled method-vs-truth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C1 scope: v0 is out of scope for the placebo The placebo is a whole-farm campaign and v0 enumerates test/reference combinations per turbine, so a whole-farm v0 run is not tractable. Recording the decision rather than leaving an unmet acceptance criterion: Copilot correctly flagged on #136 that the scope asked for v0 as an optional slow method and the driver did not provide one. The seam still accepts V0BinnedMethod unchanged, so a later campaign over a small turbine subset can include it. C4 still expects v0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C8: add the harness "window" disambiguation; track the robustness design C8 scope gains two items that belong with its vocabulary sweep: - Disambiguate "window" in benchmarking/harness/campaign.py, which uses it for two different spans in one docstring: CampaignWindow is the whole baseline-plus-activity span, while its prose says "post window" / "activity window" for the treated part alone. That is the ambiguity C1 renamed `window` to `analysis_period` to escape, so a reader who knows the harness will misread the spec field. - Retire "treated" from the benchmarking layer in the same pass (443 uses, 64 of them the shared treated_mask helpers). src/ is already clear of it. Also drops "treated period" as the example neutral fallback, since the same issue retires that word. The robustness design note was untracked and had picked up the C1 naming pass ("brief" -> "campaign spec"), so those edits existed only in the working tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Mark C0, W0 and C1 done in the campaigns issues list C1 merged as PR #136. C0 and W0 were already complete but unmarked, so the list did not show where the tranche stood. Each now carries a Status line in the same shape C7 uses, and the suggested order marks the finished issues and names C2 as next. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * add spec * C2: campaign context on the method seam Methods learn their candidate references and row validity from a CampaignContext the runner derives from the CampaignSpec, instead of each deriving "every turbine except the test one" -- nine such derivations across four methods, and a declaration nothing read. CampaignContext (harness/context.py) is a narrow per-test-turbine view, so methods are not coupled to the analyst-facing declaration that C8 generalizes and W2 promotes to public API. context_for (campaigns/context.py) is the one translation point, and the one place to audit that no truth reaches a method. Declared validity (exclusions, C8 histories, C5 wake gating) rides the context; screened validity (R3) stays method-internal. valid_for_uplift is named for its purpose: data can be invalid for uplift and still valid for a northing analysis. MethodInput gains a lazily-built default context that is today's implicit contract, so existing call sites are unchanged. Adds ColumnSchema.northed(role) for R1's derived column. Behaviour change: the campaign path now honours the declared candidate_references, so the placebo's six upgraded turbines stop acting as each other's references. Placebo CF1-CF5 in findings_campaigns.md are stale as a result. The study path is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * C2 review: enforce context invariants, cover declared turbines Addresses the PR 137 review. MethodInput now rejects two silent-failure modes: an input with neither a campaign_context nor an upgrade_timing (which built a context with timing=None and failed later in resolve_toggle), and a context whose test_wtg differs from the input's (which would estimate one turbine while reading another's references and row validity). context_for now covers every declared turbine present in the frame, not just the test turbine and its references, and select() raises rather than silently keeping the rows of a turbine it has no validity for. A method co-analysing several turbines keeps them via select(also=...); those rows were previously exempt from declared validity. Also hoists a set() rebuilt per column in the two wide-format methods. Not changed: build_toggle_df already de-duplicates its index internally and documents itself as indexed by unique timestamps, so passing a long-format index is correct and unchanged from before this branch. Docs: each C-issue with a real Hill of Towie counterpart now runs it alongside the synthetic campaign to test the shape of the declaration (no ground truth, not scored); C3/C4/C5 name their counterpart. W2 gains a YAML-declared CampaignSpec, with those three real campaigns as the acceptance test for "easy to use". C2's done-when records the placebo re-record as its closing step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * CF6: re-record the placebo under the C2 reference rule C2 made the campaign path honour the declared candidate_references, so the six upgraded placebo turbines no longer serve as each other's references. Re-runs both placebo campaigns and records the result as CF6, controlled against the same driver run from the pre-C2 commit a1f96af. The control reads +0.1485%, matching the +0.148% CF3 recorded, so the comparison is a like-for-like A/B of the reference rule. power_model's prepost farm error falls from +0.148% to +0.039%, while its mean per-turbine absolute error rises slightly (0.481 -> 0.515 pp): the headline gain is cancellation across turbines, not better individual estimates. The mechanism is explicitly not established. Truth is still exactly 0.0 in both modes. CF1 is marked superseded; CF2-CF5 carry a note that they predate the reference rule and are not reproducible as written, their sweeps not having been re-run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D * Re-record the toggle compare baseline (stale since #126/#128) The portable baseline dated from de85f84 (2026-07-15). #126 (V1 toggle specialist) and #128 (custom filter to toggle specialist) then added ~270 lines to toggle_specialist.py without re-recording it, so toggle_specialist had been reading MOVED on every run since: 67 of 84 cells, worst 1.52 pp, against a band of 1e-5 pp. The drift is confined to the conditional per-bin cells (condition == "power"); no overall row moved, so the headline P50 was never affected. That is consistent with those PRs reworking the conditional path deliberately and forgetting the re-record. Not caused by the C2 work on this branch: an A/B with the context calls neutralised is bit-identical, and a control sweep on the pre-C2 commit a1f96af produces the same 19 moved cp_0pct cells to the digit. Recorded from a clean tree at 2e13ac3 on linux, then verified with a fresh compare: power_model UNCHANGED (max delta 0.0531 pp, band 0.1) and toggle_specialist UNCHANGED (max delta 5e-07 pp, band 1e-5) — its effectively bit-exact band is usable again as the sharp instrument for small regressions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECPVDa4P3dbiWYCQKbz18D --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0 is a single-method tool: it measures turbine-upgrade uplift with a binned power-curve, test-vs-reference method. v1 turns wind-up into a platform in which alternative uplift methods are pluggable and objectively benchmarked on synthetic datasets with known ground truth. The driving goals are accurate and precise results from short campaigns and richer conditional information about how an upgrade performs (uplift by wind speed, uplift in wakes vs free-stream, day vs night, by direction/stability, etc.).
Metrics:
Design considerations: