about preprocess_raw_data.py about equidock_public HOT 5 OPEN

Generating the full DIPS training data takes a lot of time and you have to check if you have enough resources for it. Can you try generating just a fraction of it first, e.g., -data_fraction 0.1 ?

from equidock_public.

Thank you for your reply! I can successfully run the command by modifying parameters! Thank you very much for help!

from equidock_public.

Thank you for your reply! I can successfully run the command by modifying parameters! Thank you very much for help!

I run it with 160GB RAM for five hours, still failed get the same error. that's really nedd a huge resources.
mark it hope usefull for others

from equidock_public.

marke it , i used 25 cpu 400GB RAm processed for 15 hours.

from equidock_public.

I had the same problem. The main reason for this is insufficient memory. The pre-processing of the training data of DIPS dataset did require a large amount of memory, which I could not complete this at one time with a server with 256G memory.

One way is to batch. /DIPS/data/DIPS/interim/pairs-pruned/pairs-postprocessed-train.txt stores all the PDB files waiting to be pre-processed. So, you can divide the txt file into several parts and preprocessing respectively. After this you just need to merge the generated files together. I divided the training data into two parts and finished the pre-processing successfully with 256G memory server.

from equidock_public.