Linking downstream against fftw on Windows, using clang-cl, raises an error:

lld-link: error: could not open 'm.lib': no such file or directory

ref: conda-forge/warpx-feedstock#11

I think this is caused by the unconditional linking of m.lib, which usually should not be required for MSVC:
https://github.com/FFTW/fftw3/blob/fftw-3.3.9/CMakeLists.txt#L109-L112

The work-around for this could be to pass: -DLIBM_LIBRARY=LIBM-NOTFOUND on windows builds.

Issue:

The fftw.pc file shipped with fftw-3.3.8-nompi_hbc43ac9_1110 declares

Version: 3.3.7

rather than the actual packaged version...

#91 fixed the feedstock for running on 64-bit Windows on old CPUs without more advanced vector units than SSE2. The fix is technically a hack around FFTW's CMakeLists.txt (see FFTW/fftw3#292) and may fail in the future when/if the CMake configuration is changed/fixed. When that happens, we should switch configuration back to using ENABLE_SSE2.

It is more a question than a proper issue for this feedstock.

I'd need fftw3_mpi in conda-forge. Similar to these Debian package:

• https://packages.debian.org/stretch/libfftw3-mpi
• https://packages.debian.org/stretch/libfftw3-mpi-dev

These packages depend on libfftw3 and openmpi. They do not contain the libfftw3 libraries.

It would be nice to have the same structure on conda, i.e. that the conda packages fftw and fftw_mpi would be compatible (fftw would be a dependence of fftw_mpi).

The code for fftwmpi-feedstock would be very similar than the one for this feedstock and it would also produce the same libraries as this recipe, so I'm wondering if it's a good thing to have another feedstock or if it would be possible with one feedstock to create two packages.

I was just reading the 3.3.5 release notes

I see that AVX2 support has been added, but I don't know that we should enable it on conda-forge, as the release notes state:

"Special note for distribution maintainers: Although FFTW supports a zillion SIMD instruction
sets, enabling them all at the same time is a bad idea, because it increases the planning time
for minimal gain.   We recommend that general-purpose x86 distributions only enable SSE2
and perhaps AVX. Users who care about the last ounce of performance should recompile
FFTW themselves."

We are currently in line with this recommendation, but thought I would open an issue here for future reference and in case others have thoughts/comments

@conda-forge-admin, please rerender

Would be nice to get a Windows package supported here.

Hi!Thanks for the share of Fftw packages on conda.
However, I've successfully installed the packages with "conda install -c conda-forge fftw=3.3.6" on mac os and I can find the package installed in anaconda environment. But I just can't import the module with the name fftw3 or fftw, it always tells me that there is no such a module.
I wonder if you know why?
My project deadline is coming soon and this is all I need!! I appreciate it if you can offer me a solution! Thank you!!


By the way, the environment is MacOs Sierra 10.12.5, python 3.5.3!

I'm trying to compile a library with this and I get the following linker error:

/tmp/libemos-4.4.6-Source/interpolation/jsymgg.F:340: undefined reference to `dfftw_plan_many_dft_c2r_'
/tmp/libemos-4.4.6-Source/interpolation/jsymgg.F:341: undefined reference to `dfftw_execute_'
/tmp/libemos-4.4.6-Source/interpolation/jsymgg.F:342: undefined reference to `dfftw_destroy_plan_'

It turns out this is due to missing double precision support.
Could this feature be added to the feedstock?

Just wondering: why are the tests in meta.yaml set to exit immediately on success? https://github.com/conda-forge/fftw-feedstock/blob/master/recipe/meta.yaml#L30 This way, only the first test is run, i.e. only the check for the existence of lib/libfftw3f.a is performed, while a whole array of additional tests are setup below that.

Additional comment:

Just testing

Since the mamba upgrade in October, I see the following concretization issue in my downstream project:

The reported errors are:
- Encountered problems while solving:
-   - package fftw-3.3.10-nompi_h427eb53_101 requires libgfortran5 >=9.4.0, but none of the providers can be installed

My project is a plain C/C++ project:
conda-forge/warpx-feedstock#33
and and declares the FFTW dependency here:
https://github.com/conda-forge/warpx-feedstock/blob/fe5437ec5282ebdbf4e43af96b8a894e26debb31/recipe/meta.yaml#L43

Should we try to rerender the feedstock and see if this solves it?

We appear to be going over the Travis CI log length limit. While it says it will terminate our jobs, it seems to let them run anyways, which is fortunate. That said, we can't expect this to hold up forever. Would be good to fix our build to stay within this log length limit if possible or switch to CircleCI, whichever is easiest.

Comment:

In #88 we disabled the PPC tests on Azure. If anybody knows how to fix them, a PR would be highly appreciated.

Hoping @jayfurmanek has some ideas :)

So fftw 3.3.4 has this compatibility version

Load command 3
          cmd LC_ID_DYLIB
      cmdsize 56
         name @rpath/libfftw3f.3.dylib (offset 24)
   time stamp 1 Wed Dec 31 18:00:01 1969
      current version 8.4.0
compatibility version 8.0.0

However version 3.3.8 looks like this

Load command 3
          cmd LC_ID_DYLIB
      cmdsize 56
         name @rpath/libfftw3f.3.dylib (offset 24)
   time stamp 1 Wed Dec 31 18:00:01 1969
      current version 9.8.0
compatibility version 9.0.0

but the libraries themselves have file names like this

(base) localhost:lib Matt$ ls
libfftw3.3.dylib          libfftw3_threads.3.dylib  libfftw3f.3.dylib         libfftw3f_threads.3.dylib libfftw3l.3.dylib         libfftw3l_threads.3.dylib pkgconfig
libfftw3.a                libfftw3_threads.a        libfftw3f.a               libfftw3f_threads.a       libfftw3l.a               libfftw3l_threads.a
libfftw3.dylib            libfftw3_threads.dylib    libfftw3f.dylib           libfftw3f_threads.dylib   libfftw3l.dylib           libfftw3l_threads.dylib
libfftw3.la               libfftw3_threads.la       libfftw3f.la              libfftw3f_threads.la      libfftw3l.la              libfftw3l_threads.la

So my questions are

1. Why aren't the dylibs named with their compat versions (e.g., libfftw3.8.dylib or whatever)?
2. As a practical issue, do we need to break up the build so that the dylibs can be properly versioned?

I suppose also we could pin fftw down to the micro version globally

This would fix this issue when using cmake (with clean scripts) to build packages that depend on fftw.

See http://www.fftw.org/release-notes.html

There is no VC14 build of this package for windows listed in conda-forge files. AFAICT, this causes python 3.6 windows conda package of pyFFTW not to be build.

The recipe/meta.yaml specifies that, on Windows, a conda package should be build with VC14 whereas appveyor config only launches VC9.

Is it because a pass of conda smithy has been missed?

After reading #16. Current Windows packages do not provide the Fortran bindings.

I'd be happy to try, but I'd need some guidance on what the recommended strategy is:

• Uncommenting that line in meta.yaml? Seems too easy :)
• Migrating to msys2 compilers? Drafted here.

Thanks!

Is there a way to install through Conda for a build with OpenMP support? I'm using a project that needs libfftw3_omp.

It seems that in
#50
A discussion started on whether or not we should include the static files.

I believe conda-forge has since pushed away from static libraries.

Would it be acceptable to remove the static libraries from this package and deal with the issues when they arise downstream?
Can we split the package instead into two?

fftw-static   # not recommended but for downstream projects to migrate
fftw  # only dynamic modules

Thoughts?

Comment:

This feedstock has been tremendously valuable for building stacks for Dedalus, but in some cases when building on HPC systems, it may be preferable to use an existing optimized FFTW build on the cluster. However, this currently makes it hard to use other conda builds that rely on FFTW, since conda tries to install it's own FFTW binaries.

I think this could be avoided by creating "external" dummy builds on this feedstock like what has been done with MPI. Then upstream conda builds that rely on FFTW should work with either the true builds from the feedstock, or just pass through to the existing system libraries if the user pre-installs the dummy build.

I'm still pretty new to conda forge builds, so I might eventually be able to try this myself and open a PR, but if it's trivial for anyone else to do in the meantime, it's a feature that I think would be greatly appreciated!

The build matrix seems to have multiple versions of python in it which is weird since this is a binary package.

Neon acceleration is currently quite slow on osx-arm64 platforms (see FFTW/fftw3#129).

For example, if I compile fftw with --enable-neon it takes 98.8(8) µs per loop to compute a 256x256x256 fftn, compared with 28.7(4) µs without --enable-neon.

I am not sure exactly what neon is for (is it something for iOS?), but it would be good if a faster version could be provided. (I am new to conda-forge, so don't know how this would be specified.)

Here is the file I used to profile (must be run with ipython):

# perf.py
import numpy as np
import pyfftw

from IPython import get_ipython

ipython = get_ipython()

N = 256
rng = np.random.default_rng(seed=2)
A = rng.normal(size=(2, N, N, N)).view(dtype=complex)

At = np.fft.fftn(A)

At1 = pyfftw.interfaces.numpy_fft.fftn(A)
assert np.allclose(At1, At)

# By default, pyfftw does not cache the plans.  Here we enable the cache
# and set the keepalive time to an hour.
pyfftw.interfaces.cache.enable()
pyfftw.interfaces.cache.set_keepalive_time(60 * 60)

_fftn = pyfftw.builders.fftn(
    a=A.copy(),
    planner_effort="FFTW_MEASURE",
    threads=8,
    overwrite_input=False,
    auto_align_input=True,
    auto_contiguous=True,
    avoid_copy=False,
)

At2 = _fftn(A)
assert np.allclose(At2, At)

# Also, the number of threads is set by default to 1.  Here we set the
# default value to 8 and use FFT_MEASURE to actually check.


ipython.run_line_magic("timeit", "np.fft.fftn(A)")
ipython.run_line_magic("timeit", "pyfftw.interfaces.numpy_fft.fftn(A)")
ipython.run_line_magic("timeit", "_fftn(A)")

I am building fftw with the following commands:

MY_FLAGS="--enable-threads --enable-neon --enable-armv8-cntvct-el0 --prefix=$(grealpath ~/fftw)"
MY_FLAGS="--enable-threads --enable-armv8-cntvct-el0 --prefix=$(grealpath ~/fftw)"
make clean
./configure --enable-float $MY_FLAGS 
make
make install
make clean
./configure $MY_FLAGS
make
make install
make clean

Once this is built, I create a conda env and copy the libraries. See also https://github.com/andrej5elin/howto_fftw_apple_silicon

conda create -y -p envs/py3.11 "python~=3.11" ipython
conda activate envs/py3.11
cp ~/fftw/bin/fftw-wisdom* envs/py3.11/bin/
cp ~/fftw/include/fftw3* envs/py3.11/include/
cp ~/fftw/lib/libfftw3* envs/py3.11/lib/
cp ~/fftw/lib/pkgconfig/fftw3* envs/py3.11/lib/pkgconfig/
CFLAGS="-Wno-implicit-function-declaration" pip --no-cache-dir install pyfftw

I then run

ipython perf.py

which gives

# Without --enable-neon
587 ms ± 4.07 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
579 ms ± 6.38 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
29.9 ms ± 1.14 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

# With --enable-neon
597 ms ± 6.48 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
568 ms ± 7.53 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
98.8 ms ± 759 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

This is comparable to what I get if I just do

$ conda create -y -p envs/py3.11 -c conda-forge "python~=3.11" ipython pyfftw
...
$ envs/py3.11/bin/ipython perf.py
599 ms ± 7.56 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
550 ms ± 4.44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
92.5 ms ± 750 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

For comparison, installing with rosetta (CONDA_SUBDIR=osx-64) gives the following timings:

$ CONDA_SUBDIR=osx-64 conda create -y -p envs/py3.11 -c conda-forge "python~=3.11" ipython pyfftw
...
$ envs/py3.11/bin/ipython perf.py
1.34 s ± 12.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
1.29 s ± 205 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
25.9 ms ± 468 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

I am not sure why running under Rosetta is better. Am I missing some ARM optimizations?

This is running on a Macbook Pro (16-inch, 2021) with an Apple M1 Max processor and 64GB of RAM.

Could fftw be configured without Fortran support or would this better be a separate feedstock?