Comments (18)
Just a quick update on this:
@DanielLenz: It seems like matplotlib
was indeed the issue. The kernel only dies when I import both nmt
and matplotlib
together. I didn't try the different backends, but os.environ["KMP_DUPLICATE_LIB_OK"] = "True"
resolves the issue.
Tagging @lluism and @izzyswafford because we ran into a similar issue a while ago.
I'm still looking forward to comments from other people as well.
from namaster.
What's the jupyter error message, i.e. what is printed in the terminal that runs your jupyter server?
from namaster.
Hey Daniel,
This is what I get in the terminal:
Adapting from protocol version 5.1 (kernel 7b812c0f-23d8-4dd2-9542-d8014eb96ca1) to 5.3 (client). OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized. OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
Is there a quick fix for this?
from namaster.
I haven't experienced this issue myself, but quickly googling the error (e.g. dmlc/xgboost#1715) suggests a couple of different approaches:
- Re-installing pymaster (did you install using conda? I think that's the recommend way nowadays)
- Use the "unsafe, unsupported, undocumented workaround ", i.e.
os.environ["KMP_DUPLICATE_LIB_OK"] = "True"
- Don't use notebooks (worthwhile talk by Joel Grus)
- Use a different matplotlib backend (
TkAgg
can apparently cause issues) conda install nomkl
- Wait for others to chime in!
Good luck!
from namaster.
Thanks, Daniel. I did install pymaster using conda, but I'll definitely try your other suggestions and post the updates here soon.
Cheers
from namaster.
I am also running into similar issues and if you import namaster first, the problem doesn't show (at least in my case).
from namaster.
@fjaviersanchez: Thanks for the update and sorry for my late reply. I just tested this and the problem still persists.
The kernel dies regardless of the import order for namaster
and matplotlib
.
from namaster.
@syasini thanks for trying this workaround and reporting the results here.
from namaster.
Hello all,
I am facing a somewhat similar issue and can't crack it so I was wondering if someone out there would have any tip. I installed Namaste from source and can test the C routines check just fine (make check) but running the python test crashed my python (very unusual) with a segfault.
Below is the terminal output. It suggests something is off in my setup but it usually works well so I am confused. Any suggestion welcome and thanks for making this package public.
===
bash-3.2$ python -m unittest discover -v
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/healpy/fitsfunc.py:351: UserWarning: If you are not specifying the input dtype and using the default np.float64 dtype of read_map(), please consider that it will change in a future version to None as to keep the same dtype of the input file: please explicitly set the dtype if it is important to you.
warnings.warn(
test_bins_flat_alloc (test.test_nmt_bins.TestBinsFsk) ... ok
test_bins_flat_binning (test.test_nmt_bins.TestBinsFsk) ... ok
test_bins_flat_errors (test.test_nmt_bins.TestBinsFsk) ... ok
test_bins_binning (test.test_nmt_bins.TestBinsSph) ... ok
test_bins_binning_f_ell (test.test_nmt_bins.TestBinsSph) ... ok
test_bins_constant (test.test_nmt_bins.TestBinsSph) ... ok
test_bins_edges (test.test_nmt_bins.TestBinsSph) ... ok
test_bins_errors (test.test_nmt_bins.TestBinsSph) ... ok
test_bins_variable (test.test_nmt_bins.TestBinsSph) ... ok
test_workspace_covar_flat_benchmark (test.test_nmt_covar.TestCovarFsk) ... ok
test_workspace_covar_flat_errors (test.test_nmt_covar.TestCovarFsk) ... ok
test_workspace_covar_benchmark (test.test_nmt_covar.TestCovarSph) ... /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/healpy/fitsfunc.py:351: UserWarning: If you are not specifying the input dtype and using the default np.float64 dtype of read_map(), please consider that it will change in a future version to None as to keep the same dtype of the input file: please explicitly set the dtype if it is important to you.
warnings.warn(
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/healpy/fitsfunc.py:351: UserWarning: If you are not specifying the input dtype and using the default np.float64 dtype of read_map(), please consider that it will change in a future version to None as to keep the same dtype of the input file: please explicitly set the dtype if it is important to you.
warnings.warn(
Segmentation fault: 11
bash-3.2$
from namaster.
Hi @olivierdore, this looks like conflicting healpix-related libraries (maybe NaMaster
is calling one healpix/libsharp
version and healpy
a different one?) because it breaks the first time it's calling healpy, essentially. I also thought it could be python3.8 problems but I just installed from source using py38 and it seems to work for me. Have you tried using pip install pymaster
(this is likely still not going to fix it but who knows?) or if you use conda conda install pymaster -c conda-forge
? @damonge, any ideas?
from namaster.
This does not exactly answer @olivierdore's question, but a good workaround solution for the problem would be to use the Docker image from here: https://github.com/simonsobs/PSpipe
I hope this helps.
from namaster.
Thank you @fjaviersanchez and @syasini . I will fight a bit more before installing the Docker which I am sure would work. @fjaviersanchez I have been careful to link directly to my healpix/healpy installation (through MacPorts) and not reinstall it. For some reason, I do need however to reinstall libsharp as using my default installation (through MacPorts) leads to a segfault in the C "make check" (TEST 23/66 nmt:he_sht_car /bin/sh: line 1: 79903 Segmentation fault: 11 ${dir}$tst), whereas linking to a fresh install leads to the segfault in python... Very odd. But maybe libsharp is indeed the culprit...
from namaster.
I think that NaMaster is still reinstalling healpix and libsharp. A hacky option is to copy or create a symlink of libchealpix.a
and libsharp.a
in the ./_deps/lib
subdirectory, where ./
is NaMaster's top directory (where setup.py
lives) and also copy (or symlink) the files from the include
subdirectories from libsharp and healpix and put them in _deps/include
. That way the reinstall part is skipped. Another hacky option is to comment out lines 76 and 77 in setup.py
and remove "./_deps/lib/libchealpix.a"
from line 91 and see if it still compiles.
from namaster.
Thank you Javier. I tried all the above already but it did not solve my problem. Either I break the C install or I break the python wrappers!
from namaster.
We've never tested the code on python3.8. I doubt that's the issue, but I'll give it a try.
Can I check if you have:
- Removed all tracers of priors installations of libsharp and namaster
- Reinstalled namaster through pip.
?
from namaster.
Travis passes on 3.8, so that's not the problem
from namaster.
Hi @damonge . Thanks for the answer. I have libsharp install in my libary as it comes with healpy (all installed through MacPorts) and can not remove it without breaking healpy. Pip (also from MacPorts) does not work for me as it does not find the dependencies (fftw, gsl...) even though they are there (this is true whether I use CC=gcc or clang). That's why I tried to install from the source. I now suspect the issue is that sharp was installed with clang (that's the only option for macports) and I was trying to install namaster with gcc. It does not affect the C build up/check though... so I am retrying to reinstall namaster with clang now but still face some segfault testing the C library (it compiles fine).
from namaster.
Hey all, just wanted to give you an update. After reinstalling from scratch, linking to the dependent libraries by hand (ie explicit sim links) to make sure I can control all the dependencies, and deciding to use only clang (although it should not matter), I can run properly the C routines "make check" but I still get a segfault 11 using the python wrapper in (test_workspace_covar_benchmark (test.test_nmt_covar.TestCovarSph). I am running out of ideas so will give up for now and will move to another machine. Thanks for the suggestions. In the process, I discovered that I could not link properly with the macports installed sharp library (installed for healpix/healpy) and I had to reinstall a new version to get it to work properly. Maybe that is the issue after all but why it would affect the python call of this library and not the C call is beyond me. Thanks.
from namaster.
Related Issues (20)
- nan's with map read from fits file HOT 2
- random seed in synfast_flat
- I am not seeing a real improvement in performance, but it might be my own compiler being a bit outdated. Still worth merging, since libsharp2 is the one being maintained, but will open an issue to investigate further.
- configure fails to find cfitsio HOT 7
- Missing version number in pymaster
- Help installing NaMaster on M2 macOS HOT 17
- cannot find _nmtbin module HOT 4
- Error when setting lmax < 3 * nside - 1 in compute_full_master HOT 2
- Missing different lmax for mask and coupling coefficients for covariances HOT 1
- f_ell does not seem to do anything in NmtBin HOT 1
- Conda installation fails if conda installs gsl=2.7.1 instead of gsl=2.7
- Inconsistency in results between MacOS and Ubuntu HOT 13
- Make exception message more informative
- Weird power in compute_coupled_cell HOT 3
- pymaster installation problem HOT 3
- Pseudo-cl's in the covariance computation - pixel window function HOT 3
- Floating point errors with full sky CAR pixellization HOT 1
- NaMaster installation CFITSIO issue
- Apodisation window functions HOT 3
- Noise bias for flat sky HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from namaster.