Improve test multipass validator. #393
Open
+41
−12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
cc @mzuenni |
meisterT
added a commit
to meisterT/domjudge
that referenced
this pull request
Sep 28, 2024
Note that this does not include support for interactive multi-pass problems. Currently, we only upload the result of the last pass of each test case run, which we should obviously fix in the long run but it's easier to do in separate changes. General overview of multi-pass problems: A multi-pass problem requires a validator and that validator is in charge of deciding whether another pass is being run. In that case, the file `feedbackdir/nextpass.in` will contain the input of the next pass. The problem specification includes an upper limit of passes (most often 2), and if the validator generates more passes than the limit, it is treated as internal error. Progress towards DOMjudge#2307. Tested with RagnarGrootKoerkamp/BAPCtools#393: ``` [Sep 28 09:49:05.007] judgedaemon[301583]: Judge started on goo-0 [DOMjudge/8.4.0DEV/3c8c16477] [Sep 28 09:49:05.007] judgedaemon[301583]: 🔏 Executing chroot script: 'chroot-startstop.sh check' [Sep 28 09:49:05.015] judgedaemon[301583]: Registering judgehost on endpoint default: http://localhost/domjudge/api [Sep 28 09:49:05.194] judgedaemon[301583]: No submissions in queue (for endpoint default), waiting... [Sep 28 09:49:22.049] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:22.050] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/100/118 [Sep 28 09:49:22.051] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:22.194] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:22.578] judgedaemon[301583]: 💻 Compilation: (absolute.py) 'correct' [Sep 28 09:49:22.646] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:22.647] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:22.882] judgedaemon[301583]: ✔ ...done in 0.075s (CPU: 0.030s), result: correct [Sep 28 09:49:22.882] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:23.117] judgedaemon[301583]: ✔ ...done in 0.044s (CPU: 0.026s), result: correct [Sep 28 09:49:23.196] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:23.196] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/101/119 [Sep 28 09:49:23.196] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:23.217] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:23.370] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:23.675] judgedaemon[301583]: 💻 Compilation: (deterministic.py) 'correct' [Sep 28 09:49:23.745] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:23.745] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:23.923] judgedaemon[301583]: ✔ ...done in 0.042s (CPU: 0.026s), result: correct [Sep 28 09:49:23.923] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:24.102] judgedaemon[301583]: ✔ ...done in 0.044s (CPU: 0.025s), result: correct [Sep 28 09:49:24.174] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:24.175] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/102/120 [Sep 28 09:49:24.175] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:24.193] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:24.336] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:24.708] judgedaemon[301583]: 💻 Compilation: (rand.py) 'correct' [Sep 28 09:49:24.780] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:24.780] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:25.039] judgedaemon[301583]: ✔ ...done in 0.129s (CPU: 0.094s), result: correct [Sep 28 09:49:25.039] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:25.304] judgedaemon[301583]: ✗ ...done in 0.120s (CPU: 0.091s), result: wrong-answer [Sep 28 09:49:25.734] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:25.734] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/103/121 [Sep 28 09:49:25.734] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:25.951] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:26.271] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:26.778] judgedaemon[301583]: 💻 Compilation: (no_output.py) 'correct' [Sep 28 09:49:26.846] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:26.847] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:27.092] judgedaemon[301583]: ✗ ...done in 0.055s (CPU: 0.025s), result: no-output [Sep 28 09:49:27.298] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:27.298] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/104/122 [Sep 28 09:49:27.298] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:27.319] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:27.454] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:27.763] judgedaemon[301583]: 💻 Compilation: (string.py) 'correct' [Sep 28 09:49:27.836] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:27.836] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:28.012] judgedaemon[301583]: ✗ ...done in 0.068s (CPU: 0.027s), result: wrong-answer [Sep 28 09:49:28.225] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:28.225] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/105/123 [Sep 28 09:49:28.225] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:28.246] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:28.381] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:28.727] judgedaemon[301583]: 💻 Compilation: (negative.py) 'correct' [Sep 28 09:49:28.803] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:28.803] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:29.045] judgedaemon[301583]: ✗ ...done in 0.089s (CPU: 0.056s), result: wrong-answer [Sep 28 09:49:29.250] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:29.250] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/106/124 [Sep 28 09:49:29.250] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:29.270] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:29.425] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:29.761] judgedaemon[301583]: 💻 Compilation: (negative_second_pass.py) 'correct' [Sep 28 09:49:29.833] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:29.833] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:30.034] judgedaemon[301583]: ✔ ...done in 0.080s (CPU: 0.057s), result: correct [Sep 28 09:49:30.034] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:30.252] judgedaemon[301583]: ✗ ...done in 0.082s (CPU: 0.060s), result: wrong-answer [Sep 28 09:49:30.446] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:30.446] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/107/125 [Sep 28 09:49:30.446] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:30.467] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:30.612] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:30.923] judgedaemon[301583]: 💻 Compilation: (extra_output.py) 'correct' [Sep 28 09:49:30.992] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:30.992] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:31.170] judgedaemon[301583]: ✔ ...done in 0.055s (CPU: 0.025s), result: correct [Sep 28 09:49:31.170] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:31.366] judgedaemon[301583]: ✗ ...done in 0.060s (CPU: 0.028s), result: wrong-answer [Sep 28 09:49:31.569] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:31.570] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/108/126 [Sep 28 09:49:31.570] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:31.590] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:31.725] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:32.062] judgedaemon[301583]: 💻 Compilation: (float.py) 'correct' [Sep 28 09:49:32.132] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:32.132] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:32.327] judgedaemon[301583]: ✗ ...done in 0.043s (CPU: 0.022s), result: wrong-answer [Sep 28 09:49:32.522] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:32.541] judgedaemon[301583]: No submissions in queue (for endpoint default), waiting... ```
RagnarGrootKoerkamp
requested changes
Sep 28, 2024
test/problems/multipass/output_validators/multipass_validator/multipass_validator.py
Outdated
Show resolved
Hide resolved
| @@ -2,18 +2,36 @@ | |||
| import sys | |||
| import pathlib | |||
|
|
|||
| judgemessage = pathlib.Path(sys.argv[3]) / 'judgemessage.txt' | |||
There was a problem hiding this comment.
'we' kinda always just use stderr instead of judgemessage, but yes I suppose this is better for a proper example
There was a problem hiding this comment.
I would say ideally both should have an example? the system should be able to concat judgemessage.txt as well as stderr output.
There was a problem hiding this comment.
I did not use stderr because I am not sure how it will be displayed in other systems like Kattis. I am fine with adding one message on stderr, if you think it makes sense though
meisterT
added a commit
to meisterT/domjudge
that referenced
this pull request
Sep 29, 2024
Note that this does not include support for interactive multi-pass problems. Currently, we only upload the result of the last pass of each test case run, which we should obviously fix in the long run but it's easier to do in separate changes. General overview of multi-pass problems: A multi-pass problem requires a validator and that validator is in charge of deciding whether another pass is being run. In that case, the file `feedbackdir/nextpass.in` will contain the input of the next pass. The problem specification includes an upper limit of passes (most often 2), and if the validator generates more passes than the limit, it is treated as internal error. Progress towards DOMjudge#2307. Tested with RagnarGrootKoerkamp/BAPCtools#393: ``` [Sep 28 09:49:05.007] judgedaemon[301583]: Judge started on goo-0 [DOMjudge/8.4.0DEV/3c8c16477] [Sep 28 09:49:05.007] judgedaemon[301583]: 🔏 Executing chroot script: 'chroot-startstop.sh check' [Sep 28 09:49:05.015] judgedaemon[301583]: Registering judgehost on endpoint default: http://localhost/domjudge/api [Sep 28 09:49:05.194] judgedaemon[301583]: No submissions in queue (for endpoint default), waiting... [Sep 28 09:49:22.049] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:22.050] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/100/118 [Sep 28 09:49:22.051] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:22.194] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:22.578] judgedaemon[301583]: 💻 Compilation: (absolute.py) 'correct' [Sep 28 09:49:22.646] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:22.647] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:22.882] judgedaemon[301583]: ✔ ...done in 0.075s (CPU: 0.030s), result: correct [Sep 28 09:49:22.882] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:23.117] judgedaemon[301583]: ✔ ...done in 0.044s (CPU: 0.026s), result: correct [Sep 28 09:49:23.196] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:23.196] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/101/119 [Sep 28 09:49:23.196] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:23.217] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:23.370] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:23.675] judgedaemon[301583]: 💻 Compilation: (deterministic.py) 'correct' [Sep 28 09:49:23.745] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:23.745] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:23.923] judgedaemon[301583]: ✔ ...done in 0.042s (CPU: 0.026s), result: correct [Sep 28 09:49:23.923] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:24.102] judgedaemon[301583]: ✔ ...done in 0.044s (CPU: 0.025s), result: correct [Sep 28 09:49:24.174] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:24.175] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/102/120 [Sep 28 09:49:24.175] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:24.193] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:24.336] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:24.708] judgedaemon[301583]: 💻 Compilation: (rand.py) 'correct' [Sep 28 09:49:24.780] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:24.780] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:25.039] judgedaemon[301583]: ✔ ...done in 0.129s (CPU: 0.094s), result: correct [Sep 28 09:49:25.039] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:25.304] judgedaemon[301583]: ✗ ...done in 0.120s (CPU: 0.091s), result: wrong-answer [Sep 28 09:49:25.734] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:25.734] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/103/121 [Sep 28 09:49:25.734] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:25.951] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:26.271] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:26.778] judgedaemon[301583]: 💻 Compilation: (no_output.py) 'correct' [Sep 28 09:49:26.846] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:26.847] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:27.092] judgedaemon[301583]: ✗ ...done in 0.055s (CPU: 0.025s), result: no-output [Sep 28 09:49:27.298] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:27.298] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/104/122 [Sep 28 09:49:27.298] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:27.319] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:27.454] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:27.763] judgedaemon[301583]: 💻 Compilation: (string.py) 'correct' [Sep 28 09:49:27.836] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:27.836] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:28.012] judgedaemon[301583]: ✗ ...done in 0.068s (CPU: 0.027s), result: wrong-answer [Sep 28 09:49:28.225] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:28.225] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/105/123 [Sep 28 09:49:28.225] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:28.246] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:28.381] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:28.727] judgedaemon[301583]: 💻 Compilation: (negative.py) 'correct' [Sep 28 09:49:28.803] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:28.803] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:29.045] judgedaemon[301583]: ✗ ...done in 0.089s (CPU: 0.056s), result: wrong-answer [Sep 28 09:49:29.250] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:29.250] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/106/124 [Sep 28 09:49:29.250] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:29.270] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:29.425] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:29.761] judgedaemon[301583]: 💻 Compilation: (negative_second_pass.py) 'correct' [Sep 28 09:49:29.833] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:29.833] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:30.034] judgedaemon[301583]: ✔ ...done in 0.080s (CPU: 0.057s), result: correct [Sep 28 09:49:30.034] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:30.252] judgedaemon[301583]: ✗ ...done in 0.082s (CPU: 0.060s), result: wrong-answer [Sep 28 09:49:30.446] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:30.446] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/107/125 [Sep 28 09:49:30.446] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:30.467] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:30.612] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:30.923] judgedaemon[301583]: 💻 Compilation: (extra_output.py) 'correct' [Sep 28 09:49:30.992] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:30.992] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:31.170] judgedaemon[301583]: ✔ ...done in 0.055s (CPU: 0.025s), result: correct [Sep 28 09:49:31.170] judgedaemon[301583]: 🔄 Running pass 2... [Sep 28 09:49:31.366] judgedaemon[301583]: ✗ ...done in 0.060s (CPU: 0.028s), result: wrong-answer [Sep 28 09:49:31.569] judgedaemon[301583]: ⇝ Received 1 'judging_run' judge tasks (endpoint default) [Sep 28 09:49:31.570] judgedaemon[301583]: Working directory: /home/sitowert/domjudge/output/judgings/goo-0/endpoint-default/108/126 [Sep 28 09:49:31.570] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:31.590] judgedaemon[301583]: 🔒 Executing chroot script: 'chroot-startstop.sh start' [Sep 28 09:49:31.725] judgedaemon[301583]: 📋 Verifying versions. [Sep 28 09:49:32.062] judgedaemon[301583]: 💻 Compilation: (float.py) 'correct' [Sep 28 09:49:32.132] judgedaemon[301583]: 🏃 Running testcase 7... [Sep 28 09:49:32.132] judgedaemon[301583]: 🔄 Running pass 1... [Sep 28 09:49:32.327] judgedaemon[301583]: ✗ ...done in 0.043s (CPU: 0.022s), result: wrong-answer [Sep 28 09:49:32.522] judgedaemon[301583]: 🔓 Executing chroot script: 'chroot-startstop.sh stop' [Sep 28 09:49:32.541] judgedaemon[301583]: No submissions in queue (for endpoint default), waiting... ```
This makes the validator more robust against broken team submissions - see additional submissions. Also add better messages so that you can understand what fails and why. All of this sets a good example (people will copy validators) and is helpful when implementing multipass problem support (e.g. DOMjudge/domjudge#2307).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes the validator more robust against broken team submissions - see additional submissions.
Also add better messages so that you can understand what fails and why.
All of this sets a good example (people will copy validators) and is helpful when implementing multipass problem support (e.g. DOMjudge/domjudge#2307).