calcPSF halts in certain specific circumstances. about poppy HOT 5 CLOSED

Are you using multiprocessing? And what version of Python? We've seen issues where parallel FFTW calculations don't play well with multiple instances invoked in parallel at a higher level. There are also issues in general with multi processing on macOS on python 2.7, but the code should be raising an error if you even try that.

First thing to try is just disabling FFTW in the poppy configuration settings, so you just use the regular python FFT. Let me know if that makes the problem go away, and if so it's diagnostic that it is definitely FFTW.

from poppy.

I am indeed using multiprocessing on Python 2.7.13, on Mac OS X 10.12 Sierra.

from poppy.

Disabling FFTW fixed the halting problem.

from poppy.

See #23 and numpy/numpy#5752

The short version: The way that multiprocessing is implemented in Python 2.7 has fundamental incompatibilities with Apple's Accelerate framework, so if you're using anything compiled against that library you may (will) see random hangs like this. The best solution we found is to move to Python ≥ 3.4, which provides an different method for forking off processes that works reliably.

We've also seen issues with using FFTW inside multiprocessing. I don't know if that's because FFTW is invoking multiprocessing, or if it's somehow just not thread-safe itself. That said, if you're using FFTW, you can just ask FFTW itself to parallelize calculations across all your CPU cores, in which case there is no performance gain from wrapping that in another layer of parallelization.

from poppy.

Ok, that makes sense. I'm ready to consider this ticket resolved.

from poppy.