Comments (17)
We can assert the MPI version by parsing 'mpirun --version'. However, we can add the '--oversubscribe' flag to the 'or' string in order to avoid a fail just due to the lack of that flag.
We can, alternativelly, put a 'mpi -np 2 ...' . 2 should be always fine, isn't it?
from xmipp.
No objection.. thank you both @dmaluenda @DStrelak
from xmipp.
By the way, note that the whole config-checking can be skipped just by manually stepping the build
./xmipp config ./xmipp compileAndInstall
(note the mising
./xmipp checkConfig
in between)
Thanks a lot for this hint. I don't think a flag is necessary in this case. I will try this option soon and comment on the result.
from xmipp.
Hi @MohamadHarastani ,
can you try if this works for you?
from xmipp.
Thanks @DStrelak , I will check it out asap
from xmipp.
can you try if this works for you?
Hi @DStrelak ,
I checked it and it worked.
Here is the corresponding output of xmipp compile:
mpirun -np 4 --oversubscribe echo ' > This sentence should be printed 4 times if mpi runs fine (by mpirun).'
> This sentence should be printed 4 times if mpi runs fine (by mpirun).
> This sentence should be printed 4 times if mpi runs fine (by mpirun).
> This sentence should be printed 4 times if mpi runs fine (by mpirun).
> This sentence should be printed 4 times if mpi runs fine (by mpirun).
These were the line of codes that I edited (starting from this line):
if checkProgram("mpirun",False):
ok=(runJob("mpirun -np 4 --oversubscribe echo '%s (by mpirun).'" % echoString) or
runJob("mpirun -np 4 --allow-run-as-root --oversubscribe echo '%s (by mpirun).'" % echoString))
elif checkProgram("mpiexec",False):
ok=(runJob("mpiexec -np 4 --oversubscribe echo '%s (by mpiexec).'" % echoString) or
runJob("mpiexec -np 4 --oversubscribe --allow-run-as-root echo '%s (by mpiexec).'" % echoString))
I can't verify if the flag --oversubscribe works with mpiexec
I checked increasing 4 to 10 and it worked (to make sure its the flag --oversubscribe that is solving the issue as I have 4 mpi slots that could by chance be empty)
This is what I used to test:
[mohamad@localhost ~]$ which mpirun
/usr/lib64/openmpi3/bin/mpirun
[mohamad@localhost ~]$ mpirun --version
mpirun (Open MPI) 3.1.3
Regards
from xmipp.
'--oversubscribe' has been added in MPI 2.1. and e.g. Travis uses version 1.6
@dmaluenda , do we / can we detect MPI version?
from xmipp.
We can, alternativelly, put a 'mpi -np 2 ...' . 2 should be always fine, isn't it?
In theory, yes. I doubt that anybody would be brave enough to use xmipp with less than two cores.
How about we link it with the number of jobs used for build?
from xmipp.
I thought the same "Who want to run Xmipp with less than 4 cores?" But the answer is "What about the login node in clusters?" Damn!
I agree on linking the number of mpi-jobs to the number of cores for the compilation. Indeed, in the hypothetical case than N=1, if
mpirun -np 1 echo whatever
works, it's fine. We are checking that mpirun works and this prove that.
from xmipp.
Can we close this issue?
from xmipp.
I think yes. I forced to use only 2 cores, that is the minimum that makes sense...
If the problem persist, please, don't hesitate to reopen this to be able to make an more accurate approach.
from xmipp.
Hello again,
I have just had another similar issue. I am trying to compile xmipp on a supercomputer. Usually, we have a script to run mpi jobs using sbatch. Here is what I get:
[uhj53dz@jean-zay4: xmipp]$ ./xmipp
'xmipp.conf' detected.
Checking configuration ------------------------------
Checking compiler configuration ...
g++ 8 detected
g++ -c -w -mtune=native -march=native -std=c++11 -O3 xmipp_test_main.cpp -o xmipp_test_main.o -I../ -I/gpfswork/rech/nvo/uhj53dz/miniconda3/include -I/gpfswork/rech/nvo/uhj53dz/miniconda3/include/python3.8 -I/gpfswork/rech/nvo/uhj53dz/miniconda3/lib/python3.8/site-packages/numpy/core/include
g++ -L/gpfswork/rech/nvo/uhj53dz/miniconda3/lib xmipp_test_main.o -o xmipp_test_main -lfftw3 -lfftw3_threads -lhdf5 -lhdf5_cpp -ltiff -ljpeg -lsqlite3 -lpthread
rm xmipp_test_main*
Checking MPI configuration ...
mpicxx -c -w -I../ -I/gpfswork/rech/nvo/uhj53dz/miniconda3/include -mtune=native -march=native -std=c++11 -O3 xmipp_mpi_test_main.cpp -o xmipp_mpi_test_main.o
mpicxx -L/gpfswork/rech/nvo/uhj53dz/miniconda3/lib xmipp_mpi_test_main.o -o xmipp_mpi_test_main -lfftw3 -lfftw3_threads -lhdf5 -lhdf5_cpp -ltiff -ljpeg -lsqlite3 -lpthread
rm xmipp_mpi_test_main*
mpirun -np 1 echo ' > This sentence should be printed 2 times if mpi runs fine.'
This version of Spack (openmpi ~legacylaunchers schedulers=slurm)
is installed without the mpiexec/mpirun commands to prevent
unintended performance issues. See https://github.com/spack/spack/pull/10340
for more details.
If you understand the potential consequences of a misconfigured mpirun, you can
use spack to install 'openmpi+legacylaunchers' to restore the executables.
Otherwise, use srun to launch your MPI executables.
mpirun -np 1 --allow-run-as-root echo ' > This sentence should be printed 2 times if mpi runs fine.'
This version of Spack (openmpi ~legacylaunchers schedulers=slurm)
is installed without the mpiexec/mpirun commands to prevent
unintended performance issues. See https://github.com/spack/spack/pull/10340
for more details.
If you understand the potential consequences of a misconfigured mpirun, you can
use spack to install 'openmpi+legacylaunchers' to restore the executables.
Otherwise, use srun to launch your MPI executables.
mpirun or mpiexec have failed.
Cannot compile with MPI or use it
rm xmipp_mpi_test_main*
rm: cannot remove 'xmipp_mpi_test_main*': No such file or directory
I will try to workaround this by commenting this test.. I will reply again here my progress.
Regards,
Mohamad
from xmipp.
Hi @MohamadHarastani ,
Thanks for reporting this problem.
However, I don't think that there's anything we can / should do about this particular case (unless, of course, it turns out that it's a wide-spread issue).
As you surely understand, we can't prepare our script for all possible environments.
The admin of your machine will be able to resolve this problem. I'm however a bit worried how / if Scipion will work fine in that environment (as we should have the support of Slurm, but AFAIK it's not often used [read: well tested]).
If in doubts, feel free to contact us, and we'll gladly help!
KR,
David
from xmipp.
Thanks @DStrelak for your reply.
I commented the lines that require mpirun and the installation continued. We have using the previous xmipp version on the same super computer (compatible with scipion2) and now I am trying to compile the new version. Of course, we don't need support for all environments or for slurm, we prepare our slurm scripts manually, all what we need is a successful xmipp compilation (it is just a linux redhat, compilation with conda).
I have limited experience in slurm, but it is used on the two supercomputers that we have access to here.
I passed this step now. Just for the record, I commented these lines from starting from here.
# if not (runJob("%s -np 2 echo '%s.'" % (configDict['MPI_RUN'], echoString)) or
# runJob("%s -np 2 --allow-run-as-root echo '%s.'" % (configDict['MPI_RUN'], echoString))):
# print(red("mpirun or mpiexec have failed."))
# return False
We can close this issue and rediscuss a solution if needed (maybe a flag to pass this mpirun test with an error message that shows the option to run with this flag).
Regards,
Mohamad
from xmipp.
Hi @MohamadHarastani ,
I'm glad that it was the only hurdle you've met.
The flag to skip (a specific) config test sounds good to me. What do you think, @dmaluenda ?
from xmipp.
I agree on a bypassing flag. I vote for an environ var like XMIPP_NOMPICHECK=True
or something like that. In this way we can add it to the https://github.com/I2PC/xmipp/wiki/Xmipp-configuration-(version-20.07) guide.
By the way, note that the whole config-checking can be skipped just by manually stepping the build
./xmipp config
./xmipp compileAndInstall
(note the mising ./xmipp checkConfig
in between)
from xmipp.
Should be resolved .
from xmipp.
Related Issues (20)
- Performance of tomo_confidence_map
- Debian package HOT 1
- wrong path to the libcifpp HOT 9
- angular_project_library produces deterministic results when perturbed HOT 2
- Cannot fetch googletest HOT 1
- xmipp.conf's CUDA ignored for compiling cuFFTAdvisor HOT 1
- xmipp_volum_from_pdb gives different contrast according to the pixel size HOT 1
- libcifpp installation error stops xmipp compilation HOT 2
- libcifpp compilation failed HOT 6
- Installer: Popen.poll() returning None in unexpected situations
- non-standard path gcc compile googltest failed in centos7 HOT 1
- New Xmipp installer HOT 3
- python 3.12 error installing HOT 2
- local resolution to B factor has a zero value for residue 1000
- New CMake based compilation script
- error on installation of xmipp through scipion3 HOT 3
- Clean bin without removing xmipp.conf HOT 1
- MADV_HUGEPAGE error HOT 1
- xmipp3-volume consensus fails
- convert pdb to volume with mmcif HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xmipp.