Difido TaskRejectedException about difido-reports HOT 3 OPEN

The queue size is set to 100000. You may raise this number to 500000 and more, but, if the server got to more then 100000 operations that are pending, then my guess is that you are having some problem with your IO. Even when I stress test the server, I never get near this number, and I'm running scenarios that are not realistic in any way.
Maybe you are using some kind of a NAS? If you do then I suggest changing the location of the docRoot folder to a local folder and see if it solves the problem.

Another option is to reduce the number of calls that are performed to the server. In most binders this can easily achieve this by editing the difido.properties file and increasing the value of the min.time.between.writes property. This will change the minimum time in milliseconds that updates are done to the server. You may experience some delays in the refresh intervals of the reports but it can reduce the IO and network usage significantly.

from difido-reports.

There is no NAS there.
this worker contain all persistency jobs?
including:
delete old reports
and all the live api calls (reports, new exec etc.)
??

BTW we did not implement min.time.between.writes in python binder....
maybe this is the time to do so

from difido-reports.

Hi Benadam!

The worker is responsible for all operations done on the file system. The majority of it is updating the json files of the executions and tests. You can see it as a kind of a shock absorber for smoothing the operation.

Not having the min.time.between.writes can add more stress on the IO and it is a good idea to implement it in the binder. It is not even have to be configurable. In the official binders it is set to 100ms by default and it good enough for most cases, but still, 100,000 operations waiting in queue it's a lot. It is like you are writing to a floppy drive J.

By the way, I'm really interested in your Python binder! and also, you have my number ;)

from difido-reports.