CI: add mpi/distributed module to free CI pipe - #814
Conversation
|
added author info in the header per convention, not sure if this is correct, might be better placed in the headers of the new test instead. Happy to change if needed. |
06475eb to
9d7618d
Compare
Credits: AccelCom @ Barcelona Supercomputing Center
Credits: AccelCom @ Barcelona Supercomputing Center
Credits: AccelCom @ Barcelona Supercomputing Center
Credits: AccelCom @ Barcelona Supercomputing Center
|
i run the tests on mare nostrum 5 and on minidmr local and gh actions, same results on all of them |
|
Hi @iarejula-bsc, thanks for this! We're currently trying to (finally) get the 4.3 release ready, as well as buried under proposal writing, so will probably park this at least until 4.3 is out, but I like it! Note that I currently don't expect the tests to pass when run on multiple processes, but that is one of the items on my list for post-4.3. Your contribution will certainly help with working towards that. |
Glad to hear that!
I woudl like to say that minidmr can we also used in the local machine as a local shell to compile/run/test/debug code, if you feel this will help u to develop faster the MPI code dont hestiate on pinging me and will try to explain how to use it :D |
Adds three free CI jobs, all using minidmr, an open-source tool built by our team that allows creating a reproducible HPC cluster on Docker to simulate a multi-node MPI cluster . minidmr is already integrated into the CI pipelines of two of our other projects, and is also used to run MPI tests locally.
By using it on QuEST, this lets your CI pipeline run MPI tests on a single free
ubuntu-latestrunner.test_paid.ymlis untouched for now; however, after a careful review, it could be removed in a follow-up to save money.This PR adds:
test_free.yml: distributed-minidmr-test: full functional tests over real MPI (minidmr, 4 simulated nodes).audit.yml: sanitisation-test-mpi: ASan overcomm/*(the MPI code), minidmr with 2 nodes : the first time this code has run under a sanitiser.audit.yml: coverage-test-mpi: lcov overcomm/*via minidmr, 2 nodes: complements the existing serial coverage job, which can't exercise MPI.Impact:
mpi=ON, cuda=OFFintest_paid.yml, ~$4.80/run, only triggerable manually viaworkflow_dispatch) with free jobs on every push/PR, saving ~$5/run.comm/*goes from 0% measured (explicitly excluded from the serial coverage job, which doesn't compile with MPI) tocomm/comm_config.cppat 77.4% andcomm/comm_routines.cppat 4.3% line coverage.