Multi-processing about duplicate-image-finder HOT 4 CLOSED

Hi @thecodingchicken,
Thank you very much for reaching out and for sharing your feedback!
We already received a few requests similar to yours, and therefore decided to implement batch processing in the next upcoming release of difPy. Therefore, with this there will also be an option to multi-thread the process. This release is in ongoing development, and there is no confirmed release date yet. Please therefore, be patient and soon processing on difPy will become a lot more flexible, and scalable. :-)
Again thanks and all the best,
Elise

from duplicate-image-finder.

Just an update. Code is almost working, just have to sort out a few minor issues that break everything. Seems to be the norm for me, as I haven't really done any coding in years.
Glad to hear that you are working on it as well. Is it multithreaded in another language? As I know the GIL prevents you from running many threads that are CPU-bound. Hence why multiprocessing seems to be the only option for me, assuming that you stick in CPython, as compared to IronPython or Jython

from duplicate-image-finder.

Hi @thecodingchicken,
Great to hear you are working on it as well and that you're almost done! Congrats. Feel free to open a pull request when you're done - I would love to see what approach you took.
I am working on it as well yes, but as I have currently quite a few other things I am working on, it will still take a bit to be finalized. I am planning to use the Python multiprocessing library, as well as implement the ability to input images to difPy in batches, and save the process state so that it can be resumed if interrupted.
All the best and happy coding!
Elise

from duplicate-image-finder.

Hi @thecodingchicken,

I'm happy to let you know that difPy v4 now comes with multiprocessing. Thanks for your suggestion!

All the best,
Elise

from duplicate-image-finder.