joeatodd / hidem Goto Github PK

Computing system (please complete the following information):

• Machine name: Puhti @ CSC, Espoo
• HiDEM commit hash [e.g. 53e2eb5]: 924c983

Issue Type
Error message in compilation

Error message or symptoms
mpif-sizeof.h:395.47:
Included at mpif.h:63:
Included at /users/eefvan/sources/puhti/HiDEM/src/typedefs.f90:14:

  COMPLEX(REAL64), DIMENSION(1,1,1,1,1,1,1,*)::x
                                           1

Error: Array specification at (1) has more than 7 dimensions
Fatal Error: Error count reached limit of 25.

This same error (more than 7 dimensions) came up 25 times. Maybe related to:
CMake Warning:
Manually-specified variables were not used by the project:
CMAKE_TOOLCHAIN_FILE

Describe the context
Due to problems on Puhti, I had to recompile HiDEM. I have removed the build directory and tried to use the same compilation file as before, but then the above errors came up.

It would be good to save a copy of a full set of restart files until a new set is written, such that in case the writing of restart files breaks down for some reason (reaching storage limit or running time limit), the latest full set is still available and not partly overwritten.

Computing system (please complete the following information):

• Machine name: Mahti and Archer

Issue Type
Is this issue:
'procedural': VTU files not writing properly

Error message or symptoms:
When bringing into Paraview this message is given:

A similar message is given when trying to work in Python.

Describe the context
I am modifying the io.f90 and wave2.f90 code to write out two additional variables to the STR files. I found that I need to set Double Precision Output = TRUE for these modifications. However, setting this to TRUE does not allow the VTU files to write properly. I am working on the modifications on Mahti but also tried simply setting Double Precision Output = TRUE on old successful sims on Archer to check that it wasn't an issue with the Mahti environment.

Expected behavior
Displacement be written to the VTU files correctly with Double Precision Output set to True.

Files
Nothing was written to the error log. Example files are in this Dropbox folder: https://www.dropbox.com/sh/h4wgd8v7d7ircb9/AADKwTZU1Gp6phPzg8PvJIuna?dl=0

I'm using the HiDEM-develop branch and am looking to write out more bond information, specifically capturing MAXV and (DL-L) for broken bonds.

Is the best way to append these new variables to the STR files using the "BinarySTROutput" subroutine in io.f90? I'm wondering if it is possible to do that. If not, I could copy that routine and modify it to write new STR files containing MAXV and (DL-L)?

Other ideas?

I figure that I can use our python analysis scripts to gather the data I need from broken bonds afterwards.

Computing system (please complete the following information):
Archer2

Issue Type
Compilation on new system

Error message or symptoms
When I run: bash compile_cray.sh, the output reads:
-- checking for thread-local storage - found
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR) (found version "3.1")
-- Could NOT find MPI_Fortran (missing: MPI_Fortran_LIB_NAMES MPI_Fortran_F77_HEADER_DIR MPI_Fortran_MODULE_DIR) (found version "3.1")
CMake Error at /lus/cls01095/work/y07/shared/utils/cmake/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find MPI (missing: MPI_C_FOUND MPI_Fortran_FOUND) (found version
"3.1")

  Reason given by package: MPI component 'CXX' was requested, but language CXX is not enabled.  

Call Stack (most recent call first):
/lus/cls01095/work/y07/shared/utils/cmake/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/lus/cls01095/work/y07/shared/utils/cmake/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindMPI.cmake:1721 (find_package_handle_standard_args)
src/CMakeLists.txt:5 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/n01/n01/annac/HiDEM/HiDEM/build/CMakeFiles/CMakeOutput.log".
make: *** No targets specified and no makefile found. Stop.
make: *** No rule to make target 'install'. Stop.

Describe the context
I'm working with compile_cray.sh and HiDEM-cray.cmake.
As I understand it, PrgEnv-cray is loaded automatically on Archer2. I have commented out #module swap PrgEnv-gnu PrgEnv-cray in compile_cray.sh for this reason.
I also ran module load cmake.

Expected behavior
Successful compilation

Error Log
CMakeOutput.log

Any suspicions/clues as to what the cause might be?
Perhaps I am not loading the MPI component 'CXX' correctly? The Archer2 Quickstart for Developers page mentions using a command like: export CC=cc export CXX=CC export FC=ftn export F77=ftn export F90=ftn. I tried this but obviously don't have it set up correctly.

Computing system (please complete the following information):

• Machine name [e.g. Archer, My Desktop]: PACE @ Georgia Tech, Atlanta
• HiDEM commit hash [e.g. 53e2eb5]: 9b91a0f

Issue Type
SEGFAULT

Error message or symptoms
Segmentation fault (signal 11)

Describe the context
The segmentation fault only occurs when fracture is allowed to initiate. In my modified HiDEM code, I impose a condition where particles within a specific portion of the domain undergo bond breaking and are flushed with some velocity. I use a rectangular domain with fixed lateral and inflow margin boundary conditions with a grid spacing of 5m. I have noticed that segmentation does not occur if I use a particle SCL of 5,4,2m but using a SCL of 3m ends the simulation on fracture initiation due to segmentation. This also seems to be dependent on the amount of processors assigned to the job, i.e. the job does not fail if I use 3 nodes and 28 ppn.

Expected behavior
HiDEM should behave normally regardless of particle scale and amount of nodes assigned to the job.

Error Log

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x2aaaac28727f in ???
#0 0x2aaaac28727f in ???
#0 0x2aaaac28727f in ???
#1 0x441dd4 in ???
#2 0x40280e in ???
#3 0x2aaaac2733d4 in ???
#4 0x402837 in ???
#5 0xffffffffffffffff in ???
#1 0x441dd4 in ???
#2 0x40280e in ???
#3 0x2aaaac2733d4 in ???
#4 0x402837 in ???
#5 0xffffffffffffffff in ???
#1 0x441dd4 in ???
#2 0x40280e in ???
#3 0x2aaaac2733d4 in ???
#4 0x402837 in ???
#5 0xffffffffffffffff in ???
#0 0x2aaaac28727f in ???
#1 0x441dd4 in ???
#2 0x40280e in ???
#3 0x2aaaac2733d4 in ???
#4 0x402837 in ???
#5 0xffffffffffffffff in ???

Any suspicions/clues as to what the cause might be?
Potentially something to do with the amount of processors involved compared to the aprticle scale, however, I don't know why it would be the case that segmentation only occurs as soon as fracture is permitted.