Design the compilation and verification pipeline about sztos HOT 2 CLOSED

This is what I'm thinking of currently:

1. Download all the solution files and store them in a temporary directory.
2. Run a dedicated Docker container which will build the program (it should try to sandbox the compilation process as much as possible, apply timeouts, etc.).
3. If compilation succeeded download a test case, mount the resulting binary in a new container, run it with the test case as input and store the output.
4. Update the state:
   • If the process exited cleanly compare the actual output to the expected output (or run a verification script in an extended scenario) and update the state appropriately.
   • If errors occurred save them in the database.
5. Delete files specific to this test case.
6. Repeat 3-5 for all test cases in a problem.
7. Clean up the entire directory made for this solution.

from sztos.

Alright, this seems to make sense, I'm moving this list to the wiki.

from sztos.