Add WatchSleepNet full pipeline: IBISleepDataset + SleepStagingIBI + WatchSleepNet#1034
Open
gregschrock wants to merge 2 commits intosunlabuiuc:masterfrom
Open
Add WatchSleepNet full pipeline: IBISleepDataset + SleepStagingIBI + WatchSleepNet#1034gregschrock wants to merge 2 commits intosunlabuiuc:masterfrom
gregschrock wants to merge 2 commits intosunlabuiuc:masterfrom
Conversation
…WatchSleepNet Summary Implements the complete PyHealth pipeline for IBI-based sleep staging from consumer wearables, based on: ▎ Wang et al. (2025). WatchSleepNet: A Scalable Deep Learning Model for Wearable Sleep Staging. CHIL 2025, PMLR 287:1–20. https://proceedings.mlr.press/v287/wang25a.html The paper introduces WatchSleepNet, a deep learning architecture designed for sleep stage classification from inter-beat interval (IBI) signals derived from wrist-worn PPG/ECG wearables. It demonstrates competitive accuracy against PSG-supervised baselines while remaining deployable on low-power consumer devices (e.g., Apple Watch, Empatica E4). This PR adds three new components following the established SleepEDFDataset / SleepStagingSleepEDF pattern: IBISleepDataset (pyhealth/datasets/ibi_sleep.py) - A single, cohort-agnostic BaseDataset subclass that unifies DREAMT, SHHS, and MESA under one API via a source parameter — in contrast to the existing per-cohort classes (DREAMTDataset, SHHSDataset) which each load raw, cohort-specific file formats. IBISleepDataset operates on a common intermediate NPZ representation (25 Hz IBI time series + per-sample stage labels) produced by the included preprocessing scripts, making it straightforward to train or evaluate across cohorts without any dataset-specific loading code. - prepare_metadata() scans a directory for *.npz files, writes ibi_sleep-metadata.csv, skips unreadable files with a warning, and stores NaN for missing AHI values - default_task property returns SleepStagingIBI() SleepStagingIBI (pyhealth/tasks/sleep_staging_ibi.py) - BaseTask subclass following the SleepStagingSleepEDF API convention - Converts patient records into per-epoch sample dicts: 750-sample (30 s × 25 Hz) IBI windows with mapped sleep stage labels - Supports both 3-class (Wake / NREM / REM) and 5-class (W / N1 / N2 / N3 / REM) label spaces via num_classes parameter WatchSleepNet (pyhealth/models/watchsleepnet.py) - BaseModel subclass implementing the paper's architecture: 4-block 1D ResNet → dilated TCN → BiLSTM → Multi-head Attention → global average pooling → linear classifier - Accepts signal tensor of shape (B, 750) and returns {"loss", "y_prob", "y_true"} - Supports configurable number of output classes, hidden dimension, and attention heads Preprocessing scripts (examples/) - preprocess_dreamt_to_ibi.py — raw Empatica E4 BVP CSV → NPZ via neurokit2 peak detection - preprocess_shhs_to_ibi.py — SHHS EDF + profusion XML → NPZ via biosppy R-peak detection - preprocess_mesa_to_ibi.py — MESA EDF → NPZ; same ECG pipeline as SHHS - All three are standalone CLI scripts with lazy optional-dependency imports; dst_dir output serves directly as IBISleepDataset(root=...) End-to-end example (examples/watchsleepnet_sleep_staging.py) — demonstrates the full pipeline from dataset construction through training and evaluation using synthetic data. Docs — RST stubs added to docs/api/datasets.rst, docs/api/models.rst, and docs/api/tasks.rst with autoclass directives. Test plan - tests/core/test_ibi_sleep_dataset.py — 12 unit tests covering all three dataset sources, patient ID extraction, AHI NaN passthrough, corrupt NPZ skipping, missing AHI key, dev mode, empty directory, and set_task integration (3-class and 5-class) - tests/core/test_sleep_staging_ibi.py — 9 unit tests covering label mapping (3-class and 5-class), invalid label skipping, max-epoch cap, short-record empty return, signal shape, AHI NaN passthrough, and wrong sample-rate error - tests/core/test_watchsleepnet.py — 7 unit tests covering instantiation, forward shape, loss computation (with and without labels), wrong input length error, and backward pass - All tests use synthetic data only (no real patient data) - CI runs on Python 3.13 (test.yml); set_task tests are gated on Python ≥ 3.12 due to itertools.batched Co-authored-by: Manasa Hegde <manasah3@illinois.edu> Co-authored-by: Janani Jayanth <jananij2@illinois.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributor
Contribution Type
Option 4: Full Pipeline — Dataset + Task + Model
Paper
Wang et al. (2025). WatchSleepNet: A Scalable Deep Learning Model for Wearable Sleep Staging. CHIL 2025, PMLR 287:1–20.
https://proceedings.mlr.press/v287/wang25a.html
Description
Implements the complete PyHealth pipeline for IBI-based sleep staging from consumer wearables, following the WatchSleepNet paper methodology. The paper introduces a deep learning architecture for sleep stage classification from inter-beat interval (IBI) signals derived from wrist-worn PPG/ECG wearables (e.g., Apple Watch, Empatica E4), demonstrating competitive accuracy against PSG-supervised baselines while remaining deployable on low-power consumer devices.
This PR adds three new components following the established SleepEDFDataset / SleepStagingSleepEDF pattern:
IBISleepDataset — a single, cohort-agnostic
BaseDatasetsubclass that unifies DREAMT, SHHS, and MESA under one API via asourceparameter, in contrast to the existing per-cohort classes which each load raw, cohort-specific file formats. Operates on a common intermediate NPZ representation (25 Hz IBI time series + per-sample stage labels) produced by the included preprocessing scripts.SleepStagingIBI —
BaseTasksubclass following the SleepStagingSleepEDF API convention. Converts patient records into per-epoch sample dicts: 750-sample (30 s × 25 Hz) IBI windows with mapped sleep stage labels. Supports both 3-class (Wake / NREM / REM) and 5-class (W / N1 / N2 / N3 / REM) vianum_classes.WatchSleepNet —
BaseModelsubclass implementing the paper's architecture: 4-block 1D ResNet → dilated TCN → BiLSTM → Multi-head Attention → global average pooling → linear classifier. Accepts signal tensor of shape(B, 750)and returns{"loss", "y_prob", "y_true"}. Supports configurablenum_classes,hidden_dim, andattn_heads.File Guide
Core implementation
pyhealth/datasets/ibi_sleep.py—IBISleepDatasetpyhealth/datasets/configs/ibi_sleep.yaml— dataset YAML schemapyhealth/tasks/sleep_staging_ibi.py—SleepStagingIBIpyhealth/models/watchsleepnet.py—WatchSleepNet,ResidualBlockExample / ablation
examples/ibi_sleep_staging_ibi_watchsleepnet.py— end-to-end two-phase transfer learning pipeline; run with--syntheticfor no real data requiredPreprocessing scripts
examples/preprocess_dreamt_to_ibi.py— Empatica E4 BVP CSV → NPZ (neurokit2)examples/preprocess_shhs_to_ibi.py— SHHS EDF + profusion XML → NPZ (biosppy)examples/preprocess_mesa_to_ibi.py— MESA EDF → NPZ (same ECG pipeline as SHHS)Tests (synthetic data only, complete in milliseconds)
tests/core/test_ibi_sleep_dataset.py— 12 tests: all three sources, patient IDextraction, AHI NaN passthrough, corrupt NPZ skipping, missing AHI key, dev mode, empty directory,
set_taskintegration (3-class and 5-class)tests/core/test_sleep_staging_ibi.py— 9 tests: label mapping, invalid label skipping, max-epoch cap, short-record empty return, signal shape, wrong fs errortests/core/test_watchsleepnet.py— 8 tests: instantiation, forward shape, loss with/without labels, wrong input length, gradient flow, output dict keysDocs
docs/api/datasets/pyhealth.datasets.IBISleepDataset.rstdocs/api/tasks/pyhealth.tasks.SleepStagingIBI.rstdocs/api/models/pyhealth.models.WatchSleepNet.rstdocs/api/datasets.rst,docs/api/tasks.rst,docs/api/models.rst