# E2E tests (#2042) - [x] E2E tests are failing on Django 4.2.0 pull request (#1959). We're remove it them but need to add them again. - [x] E2E tests are randomly failing (~ 1 time out of 4). # Podman test - [ ] The tests only run Docker, not Podman (#1871). We need to add this. # More E2E tests Add to the E2E tests: - [x] A Codabench results submission (#2302) - [x] Another Codabench code submission, typically Mini-AutoML (#2302) - [x] A failing submission (#2302) - [ ] Several submissions made to the same benchmark - [ ] Add a "ingestion during scoring" test - [ ] Multi-task tests are disabled on CI - [ ] Test submission on custom queue - [ ] All tests on `codalab/codalab-legacy:py312`? It is much lighter E2E test files: https://github.com/codalab/codabench/tree/develop/tests/test_files/competitions https://github.com/codalab/codabench/tree/develop/tests/test_files/submissions The test: https://github.com/codalab/codabench/blob/develop/tests/test_submission.py The templates: https://github.com/codalab/competition-examples/ # Improve E2E - [x] Wait until `Finished` or `Failed`, and then test if `Finished`. (#2302) This wait we do not wait in case of failure, and we have more informative tests.
E2E tests (#2042)
E2E tests are failing on Django 4.2.0 pull request (Django to 4.2.0 #1959). We're remove it them but need to add them again.
E2E tests are randomly failing (~ 1 time out of 4).
Podman test
More E2E tests
Add to the E2E tests:
codalab/codalab-legacy:py312? It is much lighterE2E test files:
https://github.com/codalab/codabench/tree/develop/tests/test_files/competitions
https://github.com/codalab/codabench/tree/develop/tests/test_files/submissions
The test:
https://github.com/codalab/codabench/blob/develop/tests/test_submission.py
The templates:
https://github.com/codalab/competition-examples/
Improve E2E
FinishedorFailed, and then test ifFinished. (Update E2E tests #2302)This wait we do not wait in case of failure, and we have more informative tests.