Add chance level visualization, distribution plots, and CVD-friendly palette#1019
Open
bruAristimunha wants to merge 20 commits intodevelopfrom
Open
Add chance level visualization, distribution plots, and CVD-friendly palette#1019bruAristimunha wants to merge 20 commits intodevelopfrom
bruAristimunha wants to merge 20 commits intodevelopfrom
Conversation
Implement per-dataset chance level support based on Combrisson & Jerbi (2015) binomial significance thresholds. - Add moabb/analysis/chance_level.py with theoretical_chance_level(), adjusted_chance_level(), and get_chance_levels() functions - Add moabb/analysis/_utils.py with shared string parsing utilities (_match_int, _match_float, _compute_n_trials) - Modify score_plot() and paired_plot() to accept a chance_level parameter instead of hardcoding 0.5 - Add distribution_plot() combining violin (KDE) and strip plots - Extract _prepare_plot_data() and _extract_color_dict() helpers to reduce duplication across plot functions - Refactor _get_dataset_parameters() to use shared _compute_n_trials() - Add comprehensive tests for all new functionality
- Create moabb/analysis/style.py with MOABB logo color palette, font specs, apply_moabb_style() for accent lines, titles, and source text - Update all plot functions to use MOABB palette, percentage scores, minimal spines, and consistent typography - Add annotated chance level lines with Combrisson & Jerbi (2015) citation and adjusted significance threshold lines (p<0.05/0.01/0.001) - Center meta-analysis subtitle on zero-effect line with arrows - Export MOABB_PALETTE and apply_moabb_style from analysis __init__ - Add style verification tests
…e levels - Add _draw_paired_chance_region helper with crosshair lines at theoretical chance level and shaded L-shaped band at adjusted significance threshold - Fix paired_plot diagonal to span (min_chance, 100) instead of (0, 100) - Increase bottom margin on score_plot, distribution_plot, codecarbon_plot, paired_plot, and meta_analysis_plot to prevent axis label / citation overlap - Add paradigm parameter to get_chance_levels so it uses paradigm.used_events() for the correct class count (e.g. 2 for LeftRightImagery on a 4-class dataset) - Fix _match_int sentinel to use _RAISE instead of None, fixing crashes on datasets with "varies" trial counts (BNCI2015_007, BNCI2019_001, etc.) - Update tutorial explanation with insights from Combrisson & Jerbi (2015) and use get_chance_levels with paradigm parameter - Add tests for paired plot crosshair, shaded bands, and paradigm override
- Remove subtitles from score_plot and distribution_plot - Add shaded band (axhspan/axvspan) to _draw_chance_lines when adjusted levels are available, matching the paired_plot visual style - Annotate the band edge with "Chance by chance (%, p<0.05)" label referencing Combrisson & Jerbi (2015)
Guard _draw_chance_lines and _draw_paired_chance_region calls behind chance_level is not None checks in score_plot, distribution_plot, and paired_plot. When the user does not pass chance_level, no lines, bands, or annotations are drawn.
Swap the 6-color palette for a colorblind-friendly set (Wong 2011 / ColorBrewer principles) that spreads hues across navy, green-teal, sky-blue, purple, amber, and red axes. Rename constants to match their new roles (MOABB_SKY, MOABB_PURPLE, MOABB_AMBER). Move adjusted-chance-level annotations to the right edge of the plot so they no longer overlap the y-axis or each other, and bump the "Score (%)" axis label font size for readability.
Update 7 example scripts to use the chance level framework: - plot_benchmark, plot_benchmark_grid_search: add chance_level to score_plot - plot_select_electrodes_resample, plot_mne_and_scikit_estimators: add chance_level to paired_plot - plot_cross_session_motor_imagery, plot_cross_subject_ssvep, plot_within_session_p300: migrate from seaborn to MOABB plotting functions with chance level support Also fix get_chance_levels to use dataset.code instead of class name for consistent key matching with plotting functions.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Add changelog entries for chance level visualization, distribution plot, Economist-style restyling, and CVD-friendly palette. Apply black/isort formatting and fix codespell typo (unparseable → unparsable).
Signed-off-by: Bru <b.aristimunha@gmail.com>
…nce levels Add n_samples_test and n_classes to evaluation results so plotting functions can compute adjusted chance levels directly from the DataFrame without dataset objects or summary table parsing. - Store test set size and class count in _build_scored_result and the non-per-split path in WithinSessionEvaluation - Extend HDF5 schema with samples_test and n_classes columns - Add chance_levels_from_dataframe() for DataFrame-based computation - Support chance_level="auto" in _resolve_chance_levels()
Remove theoretical_chance_level, get_chance_levels, _extract_n_trials, _extract_n_classes, and defensive alpha validation. Inline 1/n_classes with comments. Condense whats_new entries and revert unrelated conf.py change. Simplify plotting tests to use chance_level="auto" throughout.
Remove unused logging import, shrink docstrings to one-liners, simplify _max_adjusted_threshold and _resolve_chance_levels, reuse _to_percentage in paired_plot.
Update expected column counts in test_evaluations.py (+2 per eval) and drop new columns in acceptance test to match reference CSVs.
The CrossSessionEvaluation may not produce n_samples_test/n_classes columns, so drop them only if present.
Update all examples to use the new DataFrame-based API instead of the removed get_chance_levels function.
The Results.to_dataframe() stores the column as 'samples_test' (not 'n_samples_test'), so the drop must use the correct name.
- Fix column name mismatch: n_samples_test → samples_test in chance_by_chance() - Add backward compat in to_dataframe() for old cached HDF5 files missing samples_test/n_classes columns - Update test DataFrames to use the correct column name
Use all 12 Kalunga2016 subjects instead of 2 for meaningful cross-subject results. Set overwrite=True to avoid stale cached results missing n_classes. Move "Chance by chance" annotation to the left side of the plot to prevent overlap with p-value labels.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
distribution_plotwith violin + strip overlay for richer accuracy inspectionTest plan
python examples/advanced_examples/plot_statistical_analysis.pyand verify score/distribution plots render with new colors, chance lines, and annotationspytest moabb/tests/test_chance_level.py moabb/tests/test_plotting.py— all tests should pass