Parallel execution on re-submission about autolabjs HOT 8 CLOSED

The main server recieves the ID of the student submitting the code. One solution would be to create an Array on the main server that holds the pending submissions. Therefore a second submission by the same student would not be transferred to the load balancer and an error message would be displayed.

Modified Algorithm -

1. Main server recieves a request for submission.
2. Check if the ID is already present in the 'submission_pending' array.
3. If it is then send an error message, else pass the information to the load_balancer as in a normal execution.
4. When the load balancer sends back the results, remove the ID from the array.

The above changes in the algorithm is currently pushed to the dev branch.

Files that were altered ->

1. /main_server/main_server.js
2. /main_server/public/index.html
3. /main_server/public/js/user_logic.js

from autolabjs.

I don't think blocking the second submission is what we are looking for. The actual issue is that when two concurrent similar submissions are made - docker executes them serially even though they are assigned to different containers. We end up losing the parallelism.

In my opinion, writing code to avoid a situation is not the same as fixing the actual problem. Take an example - Let's say we are re-evaluating a given lab and submissions keep coming from the student end as well. In your suggested and coded implementation, the student submissions will be denied straight away because there is another submission in the pending array. But the software must accept any genuine submissions (different from DoS attacks).

My two cents about moving forward:
We need to explore why docker serializes them by replicating the scenario and then actually working towards making the parallel evaluation work. If dedicating two nodes for the exact same submission (till the level of commit hash in real time) is a cause for worry - we should detect the duplicate and return to both requests, the results generated by evaluating one of the submissions, instead of denying one of them and showing no score at all.
It is also always better to have a good discussion on the choices we make and how it impacts use cases and the overall system before writing any code.

from autolabjs.

We discussed the design choice during the last meeting on 10-Aug-2016. It might be better to put in the proposed changes in the issues before implementing the solution. Rajat would have a chance to comment on them.
The solution implemented is the one desirable for regular lab use. In regular lab, a student submits once and waits for the result. There is not much value to implementing the use case of concurrent evaluations for the same student.
For now, it is a valid to assume concurrent evaluation requests as DoS attack. With this assumption, one student may have to wait for a long-time to get evaluated in presence of persistent DoS attacks.
Rajat, it may be better for you to drop in during weekly meetings scheduled at 2:30PM, Wednesdays. BTW, there is an unscheduled meeting today at 2:30PM. You can come in during that time.

from autolabjs.

solution implemented by @tejas-sangol need to incorporate issue #11 in order to truly protect against DoS attacks.

from autolabjs.

This issue no longer exists with Docker version 17.06.1-ce. I verified with a deployment of code available in commit a301bfd.

from autolabjs.

The issue is actually due to accessing the same files in two or more containers at the same time. Since we have the problem only for execution nodes, it is better to replicate execution_nodes/ directory for each execution node. The suggested directory structure for two execution nodes is given below for illustrative purpose.

/opt/execution_nodes
    execution_node_1/   (copy all the code from AutolabJS/execution_nodes/ and rename keys in ssl/)
    execution_node_2/   (copy all the code from AutolabJS/execution_nodes/ and rename keys in ssl/)

@AnkshitJain Do not let this serialization issue creep back into the system. The only simple way to avoid the serialization problem is to keep a single copy of code for each execution node.

This issue must be carefully addressed during changing volume mappings for installation (issue #199).

from autolabjs.

The installation structure removes the serialisation issue coming out of docker + AuFS combination. We need a test to confirm this capability.

from autolabjs.

Not a priority right now. Will be looked at later.

from autolabjs.