Comments (4)
wannesm
commented on June 14, 2024
1
The setup.py file in the master branch was corrected for M1/M2 and has picked up on your installation of libomp. Turns out the precompiled version was dynamically linking libomp and at a different location than the default homebrew location. Even though the static library was also available. Your detailed feedback made that clear and was thus very useful since I didn't realize before that macos had this default behavior.
from dtaidistance.
wannesm
commented on June 14, 2024
The M1/M2 have indeed some strange behaviour wrt OpenMP (I noticed quite some projects with similar issues). Thanks for trying so many options. As you figured out already, the compiler thinks omp is available but the library is missing. Brew installing omp should be sufficient (potentially together with brew install llvm) but on the new system macos appears more specific about the paths. I have updated the setup file accordingly.
But ... we recently got access to M1/M2 macs (Github Actions doesn't support them yet for automated compilation). So I was able to compile and upload to PyPI a precompiled wheel that should match your setup (Python 3.10 and macos-arm). This should now work by reinstalling (pip install --force-reinstall --no-cache dtaidistance) or by downloading the wheel from https://pypi.org/project/dtaidistance/#files (dtaidistance-2.3.9-cp310-cp310-macosx_12_0_arm64.whl).
(Unrelated, our unit tests run noticeably faster on the M1/M2 macs, so I hope you also experience this speed bump :-) )
from dtaidistance.
tommedema
commented on June 14, 2024
This sounds very promising, thanks @wannesm! Unfortunately when trying pip install --force-reinstall --no-cache dtaidistance I now get:
Cannot import OMP-based library (dtw_cc_omp)
Cannot import Numpy-based library (dtw_cc_numpy)
Not all libraries are available in your installation.
You can rerun the compilation from source or pip install in verbose mode:
pip install -vvv --upgrade --force-reinstall --no-deps --no-binary dtaidistance dtaidistance
In case you need to use an older version of numpy, compile against your current installation:
pip install -vvv --upgrade --force-reinstall --no-deps --no-build-isolation --no-binary dtaidistance dtaidistance
Share the following information when submitting a bug report:
== Packages ==
- Cannot import OMP-based library (dtw_cc_omp)
- dlopen(/Users/playground/.pyenv/versions/3.10.6/lib/python3.10/site-packages/dtaidistance/dtw_cc_omp.cpython-310-darwin.so, 0x0002): Library not loaded: '/opt/local/lib/libomp/libomp.dylib'
Referenced from: '/Users/playground/.pyenv/versions/3.10.6/lib/python3.10/site-packages/dtaidistance/dtw_cc_omp.cpython-310-darwin.so'
Reason: tried: '/opt/local/lib/libomp/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)
- Cannot import Numpy-based library (dtw_cc_numpy)
- dlopen(/Users/playground/.pyenv/versions/3.10.6/lib/python3.10/site-packages/dtaidistance/dtw_cc_numpy.cpython-310-darwin.so, 0x0002): Library not loaded: '/opt/local/lib/libomp/libomp.dylib'
Referenced from: '/Users/playground/.pyenv/versions/3.10.6/lib/python3.10/site-packages/dtaidistance/dtw_cc_numpy.cpython-310-darwin.so'
Reason: tried: '/opt/local/lib/libomp/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)
- Numpy version: 1.23.3
- Matplotlib version: 3.6.0
- Scipy version: 1.9.1
== System information ==
namespace(name='cpython', cache_tag='cpython-310', version=sys.version_info(major=3, minor=10, micro=6, releaselevel='final', serial=0), hexversion=50988784, _multiarch='darwin')
== Compilation information ==
Compiler type: unix
--noopenmp: 0
--forceopenmp: 0
--noxpreprocessor: 0
--forcellvm: 0
--forcegnugcc: 0
Cython found (during compilation)
- Version: 0.29.32
- Locations: <module 'Cython' from '/Users/wannes/Projects/Research/2016-DTW/repo_dtw/venv-310/lib/python3.10/site-packages/Cython/__init__.py'>
Numpy found (during compilation):
- Version: 1.23.3
- Location: <module 'numpy' from '/Users/wannes/Projects/Research/2016-DTW/repo_dtw/venv-310/lib/python3.10/site-packages/numpy/__init__.py'>
Using LLVM settings (/usr/bin/clang)
Checking for OpenMP availability for /usr/bin/clang
/usr/bin/clang -dM -E -Xpreprocessor -fopenmp - # with stdout=-1, stderr=-1, input=, encoding=ascii
#define _OPENMP 201811
... found OpenMP
All extensions:
[<setuptools.extension.Extension('dtaidistance.dtw_cc') at 0x117f27640>, <setuptools.extension.Extension('dtaidistance.ed_cc') at 0x117f27b50>, <setuptools.extension.Extension('dtaidistance.dtw_cc_omp') at 0x117f34040>, <setuptools.extension.Extension('dtaidistance.dtw_cc_numpy') at 0x117f359c0>]
==
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
Cell In [2], line 4
2 from dtaidistance import dtw
3 if not dtw.try_import_c(verbose = False):
----> 4 raise Exception('Failed to import C version of dtaidistance')
Exception: Failed to import C version of dtaidistance
When I try to pull the new code and install from source, I still get:
% python3 setup.py build_ext --inplace
dtaidistance/dtw_cc.c:781:10: fatal error: 'omp.h' file not found
#include <omp.h>
^~~~~~~
1 error generated.
ERROR: The C extension could not be compiled
error: command '/usr/bin/clang' failed with exit code 1
running build_ext
Installed the plain Python version of the package.
If you need the C extension, try reinstalling.
dtaidistance-master 2 %
Perhaps relevant:
% brew list libomp
/opt/homebrew/Cellar/libomp/14.0.6/include/ (3 files)
/opt/homebrew/Cellar/libomp/14.0.6/lib/libomp.dylib
/opt/homebrew/Cellar/libomp/14.0.6/lib/libomp.a
This is where homebrew has linked libomp:
% brew unlink libomp --dry-run | xargs ls -l
ls: Would: No such file or directory
ls: remove:: No such file or directory
lrwxr-xr-x 1 playground admin 43 Sep 17 15:49 /opt/homebrew/include/omp-tools.h -> ../Cellar/libomp/14.0.6/include/omp-tools.h
lrwxr-xr-x 1 playground admin 37 Sep 17 15:49 /opt/homebrew/include/omp.h -> ../Cellar/libomp/14.0.6/include/omp.h
lrwxr-xr-x 1 playground admin 38 Sep 17 15:49 /opt/homebrew/include/ompt.h -> ../Cellar/libomp/14.0.6/include/ompt.h
lrwxr-xr-x 1 playground admin 36 Sep 17 15:49 /opt/homebrew/lib/libomp.a -> ../Cellar/libomp/14.0.6/lib/libomp.a
lrwxr-xr-x 1 playground admin 40 Sep 17 15:49 /opt/homebrew/lib/libomp.dylib -> ../Cellar/libomp/14.0.6/lib/libomp.dylib
from dtaidistance.
tommedema
commented on June 14, 2024
this started working when I did:
eval "$(/opt/homebrew/bin/brew shellenv)"
pip install -vvv --upgrade --force-reinstall --no-deps --no-build-isolation --no-binary dtaidistance git+https://github.com/wannesm/dtaidistance.git#egg=dtaidistance
I don't know why I have to run the eval statement since this is already on my path from ~/.zprofile
and somehow it was no longer enough to restart my notebook kernel -- I also had to completely stop the jupyter server, then restart it
note that I actually had to change it to git+https://github.com/wannesm/dtaidistance.git#egg=dtaidistance from dtaidistance or otherwise I would get this issue: TypeError: subsequence_search() got an unexpected keyword argument 'dists_options'
really appreciate your excellent help and advice
from dtaidistance.
Related Issues (20)
- Subsequence search kbest_matches with k = None broken since commit 31f7703 HOT 1
- kbest_matches_fast is 5.5x slower on master than on 26c3962; k = None is 2.3x faster than k = len(series)-1 HOT 4
- sed: can't read s/^typedef when using make float HOT 4
- Different results between Fedora and raspberry pi HOT 2
- How is the DTW calculated between 3D time series? HOT 1
- Change Reqest | Custom cost function HOT 2
- conda-forge osx-arm64 build HOT 6
- error in plotting HOT 1
- Best path issue using subsequence in C HOT 2
- Install fails HOT 2
- I have two long audios, how do I find out the repetitions of them
- inconsistency between the documentation and implementation of use_ndim HOT 2
- Broken figures on readthedocs HOT 3
- Kernerl restarts on using dtw.warping_path_fast HOT 2
- Python 3.12 / NumPy 1.26 support HOT 2
- Maybe incorrect to calculate the dtaidistance.dtw_ndim.distance HOT 2
- Clustering labels HOT 1
- dtw_ndim.distance do not have inner_dist='euclidean' HOT 2
- dtw_ndim.distance_fast path problem
- trouble in dtw_ndim.distance HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dtaidistance.