Dear Michael and colleagues, The Peregrine nodes that have been boug

Test gelifes nodes about peregrine HOT 2 CLOSED

richelbilderbeek commented on July 4, 2024

Test gelifes nodes

from peregrine.

Comments (2)

richelbilderbeek commented on July 4, 2024

From ?Hanno Hildenbrandt?:

Our shiny new gelifes-nodes on peregrine feature AMD EPYCS CPUs, not Intel Xeons as any(?) other node. Unfortunately, that means that software compiled with Intel's C/C++/Fortran compilers will fail on gelifes nodes if the '-xHost' or '-fast' option was given (pretty default usage IMHO):

-------- hello_world.cpp ------

#include <iostream>

int main()

{

       std::cout << "hello world\n";

       return 0;

}

--------------------------------

p@pg-interactive:~ module load icc

p@pg-interactive:~ icc hello_world.cpp -O3 -xHost -ohello_world

p@pg-interactive:~ srun --time=00:00:10 --ntasks=1 --partition=short ./hello_world

hello world

p@pg-interactive:~srun --time=00:00:10 --ntasks=1 --partition=gelifes ./hello_world
 

Please verify that both the operating system and the processor support Intel(R) X87, CMOV, MMX, FXSAVE, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, MOVBE, POPCNT, F16C, AVX, FMA, BMI, LZCNT and AVX2 instructions.

 

srun: error: pg-node211: task 0: Exited with exit code 1

p@pg-interactive:~ bummer...

It doesn't help to compile with '-xHost' on the gelifes-nodes! icc's auto-dispatcher doesn't recognize that flag on non-Intel CPUS and fail! Don't use '-xHost' (or '-fast') then. But what else if you want max performance? For GCC you can try -O3 -march=znver1. However, now we have heterogenous environments on the cluster. Available instruction sets differ, testing once (on the interactive node) and run anywhere is not guarantied to work anymore. You might argue that this issue is a corner-case for people who compile their own stuff. Nope, it isn't:

p@pg-interactive:~ module purge

p@pg-interactive:~ module load Python

p@pg-interactive:~ srun --time=00:00:10 --ntasks=1 --partition=short python3 -c "print('Hello world')"

Hello world

p@pg-interactive:~ srun --time=00:00:10 --ntasks=1 --partition=gelifes python3 -c "print('Hello world')"

 

Please verify that both the operating system and the processor support Intel(R) X87, CMOV, MMX, FXSAVE, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, MOVBE, POPCNT, F16C, AVX, FMA, BMI, LZCNT and AVX2 instructions.

 

srun: error: pg-node211: task 0: Exited with exit code 1

Let's see why:

p@pg-interactive:~ python3-config --cflags

-I/apps/haswell/software/Python/3.6.4-intel-2018a/include/python3.6m -I/apps/haswell/software/Python/3.6.4-intel-2018a/include/python3.6m  -fno-strict-aliasing -Wsign-compare -Wunreachable-code -O2 -xHost -ftz -fp-speculation=safe -fp-model source -fPIC -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes

Spot the culprit flag(s). There are many modules that defaults to software compiled with icc -xHost. Now that is awful - none of those will run on gelifes-nodes. We have to avoid default modules and select them carefully by hand, e.g.

p@pg-interactive:~ module load Python                    /* WRONG */

p@pg-interactive:~ module load Python/3.6.4-foss-2018a    /* OK, but we stuck on GCC 6.4 for python C++ packages */

For me, this is annoying but doable. Anyhow, I expect many more troubles with ill-configured modules, Makefiles etc.. These EPYCs are otherwise epic thought :)

from peregrine.

richelbilderbeek commented on July 4, 2024

Done. It's easy, just add the different partition at the top of an sbatch bash script.

from peregrine.

Test gelifes nodes about peregrine HOT 2 CLOSED

Comments (2)

Related Issues (11)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent