In 5c, run_eval() mutates the training ini in place, replacing settings such as agent count, buffer count, frozen-bank fraction and horizon with evaluation values. The same mutated ini is then written to the native run log.
Constellation imports that log as experiment data, so it associates the model’s results with hparams that were never used for training. This makes runs appear under the wrong config and invalidates comparisons or plots involving those fields.
For example: a self-play run trained with train.horizon=64, vec.total_agents=4096 and vec.frozen_bank_pct=0.25 will be logged after eval as horizon=1, total_agents=8192 and frozen_bank_pct=1.
Problematic code: run_eval() and configuration serialization
Expected: run evaluation on a copied ini and leaving the original training metadata intact for logging and Constellation imports.
In
5c,run_eval()mutates the traininginiin place, replacing settings such as agent count, buffer count, frozen-bank fraction and horizon with evaluation values. The same mutatediniis then written to the native run log.Constellation imports that log as experiment data, so it associates the model’s results with hparams that were never used for training. This makes runs appear under the wrong config and invalidates comparisons or plots involving those fields.
For example: a self-play run trained with
train.horizon=64,vec.total_agents=4096andvec.frozen_bank_pct=0.25will be logged after eval ashorizon=1,total_agents=8192andfrozen_bank_pct=1.Problematic code: run_eval() and configuration serialization
Expected: run evaluation on a copied
iniand leaving the original training metadata intact for logging and Constellation imports.