Code Monkey home page Code Monkey logo

vol-log-based's Introduction

Log VOL - an HDF5 VOL connector for storing data in a time-log layout in files

This software repository contains source codes implementing Log VOL connector, an HDF5 Virtual Object Layer (VOL) plugin that stores HDF5 datasets in a storage layout similar to the time log. When using the Log VOL connector, write requests from an MPI process are appended one after another (as logs) in a contiguous space in the file. The contiguous spaces of multiple processes are appended one after another following the increasing order of processes' MPI rank IDs. Such a log layout I/O strategy avoids the expensive inter-process communication required when storing data in the canonical order. One of the pwoerful features of HDF5 VOL is to allow applications to make use a VOL connector by setting two environment variables without changing the application source codes. Using the Log VOL connector, exist HDF5 programs can achieve a better parallel write performance with no changes to their codes. Files created by the Log VOL conform with the HDF5 file format specification, but require the Log VOL to read them back.

  • Current build status:
    • Ubuntu_mpich
    • Ubuntu with OpenMPI
    • Mac with MPICH

HDF5 VOL Connector ID

Documents

Applicaton Case Studies and Experimental Results

Developers

Copyright (C) 2022, Northwestern University. See COPYRIGHT notice in top-level directory.

Project funding supports:

Ongoing development and maintenance of the Log VOL connector are supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of Energy's Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale ecosystem, including software, applications, and hardware technology, to support the nation's exascale computing imperative.

vol-log-based's People

Contributors

brtnfld avatar jhendersonhdf avatar khou2020 avatar wkliao avatar yzanhua avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

vol-log-based's Issues

configure mentions non-existant make command.

As far as I can tell there is no "ptest" for make:

./configure.ac:AC_ARG_VAR(TESTMPIRUN, [MPI run command for "make ptest", @<:@default: mpiexec -n NP@:>@])
./configure.ac:AC_ARG_VAR(TESTOUTDIR, [Output file directory for "make check" and "make ptest", @<:@default: ./@:>@])

h5ldump command-line options

Let's add more command-line options to h5ldump. For example,
-k to print whether the file is a regular HDF5 or a log-based.
Or even a NetCDF class file. This is similar to ncdump -k.
-H to print header only. We can add them gradually.

Wrong values reported when reading from LOG-VOL

While running in Summit and Cori with LOG-VOL I noticed that the reported read values are incorrect when compared to not using the VOL connector. It happened in sequential and parallel runs.

hdf5-iotest fails

hdf5-iotest https://github.com/brtnfld/hdf5-iotest is a performance benchmark that compares the effects of different HDF5 parameters and IO patterns. I tried it on summit with vol-log and it crashes on test #18 in H5Dwrite, #18 is output every step of rank 2 chunked arrays, with fill values on, alignment, metadata block size of 2048, using the earliest format, and MPI collective.

For the same test case on a local Linux box it worked fine.

On summit,

Currently Loaded Modules:

  1. lsf-tools/2.0 3) darshan-runtime/3.3.0-lite 5) DefApps 7) zlib/1.2.11 9) spectrum-mpi/10.4.0.3-20210112 11) netlib-lapack/3.9.1 13) fftw/3.3.9 15) nsight-compute/2021.2.1 17) cuda/11.0.3
  2. hsi/5.0.2.p5 4) xalt/1.2.1 6) gcc/11.1.0 8) cmake/3.21.3 10) essl/6.3.0 12) netlib-scalapack/2.1.0 14) boost/1.77.0 16) nsight-systems/2021.3.1.54 18) python/3.8-anaconda3

I used HDF5 1.13, and hdf5-iotest master. To compile hdf5-iotest:

#!/bin/bash
#autogen.sh

HDF5=$HOME/packages/hdf5-1.13/build/hdf5
export LDFLAGS="-L$HDF5/lib"
export LD_LIBRARY_PATH="$HDF5/lib:$LD_LIBRARY_PATH"
export LIBS="-lhdf5"
export CC="mpicc"
export CPPFLAGS="-I$HDF5/include"
export CFLAGS="-g"
../configure --prefix=$PWD
make 
make install

To run the program, it takes the input file "hdf5_iotest.ini"

[DEFAULT]
version = 0
steps = 10 
arrays = 10 
rows = 42 
columns = 42 
#process-rows = 1764
process-rows = 42
#process-rows = 882
process-columns = 1
# [weak, strong]
scaling = weak
# align along increment [bytes] boundaries
alignment-increment = 16777216 
# minimum object size [bytes] to force alignment (0=all objects)
alignment-threshold = 0
# minimum metadata block allocation size [bytes]
meta-block-size = 2048
# [posix, core, mpi-io-uni]
single-process = mpi-io-uni
[CUSTOM]
one-case = 18
#one-case = 119 
#gzip = 6
#szip = H5_SZIP_NN_OPTION_MASK, 8
#async = 1
#delay = 0s
hdf5-file = hdf5_iotest.h5
csv-file = hdf5_iotest.csv
#split = 1
#restart = 1

To run the program:


#!/bin/bash
###BSUB -P CSC300
#BSUB -P CSC444
#BSUB -W 01:00
# power 42
#BSUB -nnodes 1
#BSUB -J IOTEST 
#BSUB -o IOTEST.%J
#BSUB -e IOTEST.%J
##SMT1 -- 1 HW Thread per physical core
##SMT4 -- All 4 HW threads are active (Default)
##BSUB -alloc_flags smt1
# 42 physical cores, (21 each cpu), per node
# 84 per cpu, 168 total

module unload darshan-runtime

JID=$LSB_JOBID
cd $MEMBERWORK/csc444
mkdir iotest.$JID
cd iotest.$JID
EXEC=hdf5_iotest
cp $LS_SUBCWD/$EXEC .
cp $LS_SUBCWD/hdf5_iotest_000042.csv .
cp $LS_SUBCWD/hdf5_iotest.ini .
NPROCS_MAX=$(($LSB_MAX_NUM_PROCESSORS - 1))
NNODES=$(($NPROCS_MAX / 42))
echo "NUMBER OF NNODES, NPROCS_MAX = $NNODES $NPROCS_MAX"

NPROCS="42"

  HDF_DIR=$HOME/packages/hdf5-1.13/build/hdf5
  export LOGVOL_DEBUG_ABORT_ON_ERR=1
  export LD_LIBRARY_PATH="$HDF_DIR/lib:$HOME/packages/vol-log-based/build/log-vol/lib:$LD_LIBRARY_PATH"
  export HDF5_PLUGIN_PATH="$HOME/packages/vol-log-based/build/log-vol/lib"
  export HDF5_VOL_CONNECTOR="LOG under_vol=0;under_info={}"

for i in ${NPROCS}
do
  foo=$(printf "%06d" $i)
  echo "csv-file = hdf5_iotest${C}_${foo}.csv" >> hdf5_iotest.ini
  echo "process-rows = $i" >> hdf5_iotest.ini
  jsrun -n $i ./$EXEC 
  mv *.csv $LS_SUBCWD
  sed -i '$d' hdf5_iotest.ini
  sed -i '$d' hdf5_iotest.ini
  du hdf5_iotest.h5
  ls hdf5_iotest.h5
  $HOME/packages/hdf5/build/hdf5/bin/h5stat -S hdf5_iotest.h5
  #cp stdio_hdf5_iotest.${foo} $LS_SUBCWD/
#  rm -f *.h5
done

The only information I could get from the core files was:

#0 0x0000200000863618 in raise () from /lib64/power9/libc.so.6
Backtrace stopped: Cannot access memory at address 0x7fffcf4085c0

dread test segfautls on summit

Using HDF5 1.13 and

currently Loaded Modules:

  1. lsf-tools/2.0 3) darshan-runtime/3.3.0-lite 5) DefApps 7) zlib/1.2.11 9) spectrum-mpi/10.4.0.3-20210112 11) netlib-lapack/3.9.1 13) fftw/3.3.9 15) nsight-compute/2021.2.1 17) cuda/11.0.3
  2. hsi/5.0.2.p5 4) xalt/1.2.1 6) gcc/11.1.0 8) cmake/3.21.3 10) essl/6.3.0 12) netlib-scalapack/2.1.0 14) boost/1.77.0 16) nsight-systems/2021.3.1.54 18) python/3.8-anaconda3

the dread test fails with the segfault below, it seems to work with one rank, this is with 2 ranks.

#0 0x0000200014f17a00 in ADIOI_GEN_WriteStrided () from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/spectrum_mpi/mca_io_romio321.so
#1 0x0000200014edba8c in ADIOI_GPFS_WriteStridedColl ()
from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/spectrum_mpi/mca_io_romio321.so
#2 0x0000200014eceda8 in MPIOI_File_write_all () from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/spectrum_mpi/mca_io_romio321.so
#3 0x0000200014ecf908 in mca_io_romio_dist_MPI_File_write_at_all ()
from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/spectrum_mpi/mca_io_romio321.so
#4 0x0000200014ec1c7c in mca_io_romio321_file_write_at_all ()
from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/spectrum_mpi/mca_io_romio321.so
#5 0x0000200000b08a2c in PMPI_File_write_at_all () from /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-11.1.0/spectrum-mpi-10.4.0.3-20210112-6kg6anupjriji6pnvijebfn7ha5vsqp2/container/../lib/libmpi_ibm.so.3
#6 0x000020000009c4d8 in MPI_File_write_at_all (fh=, offset=2436, buf=0x0, count=, datatype=0x3a4abdc0, status=0x7fffe55cfb18) at lib/darshan-mpiio.c:563
#7 0x000020000017e374 in H5VL_log_filei_metaflush(H5VL_log_file_t*) () from /ccs/home/brtnfld/scratch/vol-log-based/build/src/.libs/libH5VL_log.so.0
#8 0x000020000017ab84 in H5VL_log_filei_close(H5VL_log_file_t*) () from /ccs/home/brtnfld/scratch/vol-log-based/build/src/.libs/libH5VL_log.so.0
#9 0x000020000017c3e0 in H5VL_log_filei_dec_ref(H5VL_log_file_t*) () from /ccs/home/brtnfld/scratch/vol-log-based/build/src/.libs/libH5VL_log.so.0
#10 0x000020000018780c in H5VL_log_obj_t::~H5VL_log_obj_t() () from /ccs/home/brtnfld/scratch/vol-log-based/build/src/.libs/libH5VL_log.so.0
#11 0x000020000018a920 in H5VL_log_free_wrap_ctx(void*) () from /ccs/home/brtnfld/scratch/vol-log-based/build/src/.libs/libH5VL_log.so.0
#12 0x00002000008594a4 in H5VL__free_vol_wrapper (vol_wrap_ctx=0x3a2efba0) at ../../src/H5VLint.c:2243
#13 0x000020000085c884 in H5VL_reset_vol_wrapper () at ../../src/H5VLint.c:2431
#14 0x000020000084fd88 in H5VL_file_close (vol_obj=, dxpl_id=792633534417207304, req=0x0) at ../../src/H5VLcallback.c:4163
#15 0x000020000060b9e0 in H5F__close_cb (file_vol_obj=, request=) at ../../src/H5Fint.c:216
#16 0x00002000006a388c in H5I__dec_ref (id=id@entry=72057594037927936, request=0x0) at ../../src/H5Iint.c:1036
#17 0x00002000006a3a40 in H5I__dec_app_ref (id=72057594037927936, request=) at ../../src/H5Iint.c:1108
#18 0x00002000006a3b5c in H5I_dec_app_ref (id=) at ../../src/H5Iint.c:1156
#19 0x0000200000600c3c in H5Fclose (file_id=72057594037927936) at ../../src/H5F.c:1060
#20 0x0000000010002338 in ?? ()
#21 0x0000200000e04078 in generic_start_main.isra () from /lib64/power9/libc.so.6
#22 0x0000200000e04264 in __libc_start_main () from /lib64/power9/libc.so.6
#23 0x0000000000000000 in ?? ()

passthru fails on non-MPI programs

@yzanhua
Tested using program, h5_write.c, from HDF5 examples, which is not an MPI program.

wget -qc https://raw.githubusercontent.com/HDFGroup/hdf5/hdf5-1_13_0/examples/h5_write.c

Run command:

export HDF5_VOL_CONNECTOR="LOG under_vol=0;under_info={}"
export HDF5_PLUGIN_PATH="../../src/.libs"
export H5VL_LOG_PASSTHRU_READ_WRITE 1
./h5_write

HDF5-DIAG: Error detected in HDF5 (1.13.3) MPI-process 0:
  #000: ../../hdf5-1.13.3/src/H5VLcallback.c line 2397 in H5VLdataset_write(): unable to write dataset
    major: Virtual Object Layer
    minor: Unable to initialize object
  #001: ../../hdf5-1.13.3/src/H5VLcallback.c line 2237 in H5VL__dataset_write(): dataset write failed
    major: Virtual Object Layer
    minor: Write failed
  #002: ../../hdf5-1.13.3/src/H5VLnative_dataset.c line 408 in H5VL__native_dataset_write(): can't write data
    major: Dataset
    minor: Write failed
  #003: ../../hdf5-1.13.3/src/H5Dio.c line 534 in H5D__write(): collective access for MPI-based driver only
    major: Dataset
    minor: Feature is unsupported
HDF5-DIAG: Error detected in HDF5 (1.13.3) MPI-process 0:
  #000: ../../hdf5-1.13.3/src/H5F.c line 1061 in H5Fclose(): decrementing file ID failed
    major: File accessibility
    minor: Unable to close file
  #001: ../../hdf5-1.13.3/src/H5Iint.c line 1156 in H5I_dec_app_ref(): can't decrement ID ref count
    major: Object ID
    minor: Unable to decrement reference count
  #002: ../../hdf5-1.13.3/src/H5Iint.c line 1108 in H5I__dec_app_ref(): can't decrement ID ref count
    major: Object ID
    minor: Unable to decrement reference count
  #003: ../../hdf5-1.13.3/src/H5Fint.c line 217 in H5F__close_cb(): unable to close file
    major: File accessibility
    minor: Unable to close file
  #004: ../../hdf5-1.13.3/src/H5VLcallback.c line 4341 in H5VL_file_close(): file close failed
    major: Virtual Object Layer
    minor: Unable to close file
  #005: ../../hdf5-1.13.3/src/H5VLcallback.c line 4310 in H5VL__file_close(): file close failed
    major: Virtual Object Layer
    minor: Unable to close file
HDF5: infinite loop closing library
      L,S_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top,T_top

If the environment variable H5VL_LOG_PASSTHRU_READ_WRITE is not set, then there is no error.
Also, if modifying the test program by adding MPI_Init, MPI_Finalize, and use collective mode file create property list, then it ran fine.

Segmentation fault: H5Pget_vol_cap_flags

Summary

H5Pget_vol_cap_flags gives segmentation fault.

Details

This issue prevents us from running vol-tests. vol-tests will fail in the setup stage (before any actual tests are performed).

A test program to reproduce the problem is provided below.
However, running the test program with the Passthru VOL (provided by HDF5 group) gives the same segmentation error.
Therefore, it's possible that the root cause is outside our library.

Reproducing the issue

Click here to see a test program that reproduces the problem:
#include <stdlib.h>
#include <string.h>
#include "hdf5.h"

#define CHECK_ERR(A)                                             \
    {                                                            \
        if (A < 0) {                                             \
            printf ("Error at line %d: code %d\n", __LINE__, A); \
            goto err_out;                                        \
        }                                                        \
    }

hid_t get_vol_id (int argc, char **argv, int rank);

int main (int argc, char **argv) {
    herr_t err = 0;
    int rank;
    hid_t fapl_id, connector_id;
    uint64_t vol_cap_flags;

    MPI_Init (&argc, &argv);
    MPI_Comm_rank (MPI_COMM_WORLD, &rank);

    // get connector id, return one of the following:
    //      1) the id for Log VOL
    //      2) the id for the Passthru VOL (provided by HDF5 group)
    //      3) -1 (error case)
    connector_id = get_vol_id(argc, argv, rank);
    CHECK_ERR(connector_id);

    // create a file access property list
    fapl_id = H5Pcreate (H5P_FILE_ACCESS);
    CHECK_ERR (fapl_id);

    // set the underlying VOL of fapl_id
    err = H5Pset_vol(fapl_id, connector_id, NULL);
    CHECK_ERR(err);

    // get vol_cap_flags
    err = H5Pget_vol_cap_flags(fapl_id, &vol_cap_flags);  // seg fault happens inside this line

    // following codes not able to run due to the seg fault above


    CHECK_ERR (err);

err_out:;
    if (fapl_id > 0) H5Pclose (fapl_id);
    MPI_Finalize ();
    return err;
}

hid_t get_vol_id (int argc, char **argv, int rank) {
    hid_t connector_id = H5I_INVALID_HID;
    if (argc > 2) {
        if (!rank) printf ("Usage: %s [volname]\n", argv[0]);
        goto err_out;
    }
    
    // return the id for LOG VOL
    if (strcmp(argv[1], "LOG") == 0) {
        if (!rank) printf ("Using connetcor: %s\n", argv[1]);
        connector_id = H5VLregister_connector_by_name ("LOG", H5P_DEFAULT);
        CHECK_ERR (connector_id);
    } 
    
    // return the id for Passthru VOL
    else if (strcmp(argv[1], "pass_through") == 0) {
        if (!rank) printf ("Using connetcor: %s\n", argv[1]);
        connector_id = H5VL_pass_through_register();
        CHECK_ERR (connector_id);
    }

    // error case
    else {
        if (!rank) {
            printf("Not supported for this test program.");
            printf ("Using connetcor: %s\n", argv[1]);
        }
    }
err_out:;
    return connector_id;
}
Click here to see the makefile:
HDF5=/home/HDF5/1.14.0
LOGVOL=/home/Log-Vol/install

all:
	mpicc test.c -g -o test \
	-I${HDF5}/include \
	-L${HDF5}/lib -lhdf5 -lhdf5_hl

run:
	LD_LIBRARY_PATH=${LOGVOL}/lib:${HDF5}/lib:${LD_LIBRARY_PATH} \
	HDF5_PLUGIN_PATH="${LOGVOL}/lib" \
	mpirun -n 1 ./test LOG

passthru:
	LD_LIBRARY_PATH=${HDF5}/lib:${LD_LIBRARY_PATH} \
	HDF5_PLUGIN_PATH="${HDF5}/lib" \
	mpirun -n 1 ./test pass_through

clean:
	rm -rf *.h5 core.* test

make to compile.

make run to run the test program with Log VOL.

make passthru to run the test program with the Passthru VOL (provided by HDF5 group).

gdb output, using Log VOL

gdb output, using the Passthru VOL

Would you please make an official release?

I'd like to push a Spack package for this VOL into the official Spack repository but the Spack maintainer will ask the version number for this software. I'd appreciate if you can tag this repo with an official version number.

null_req test fails.

See the artifact from my GitHub Action [1].

FAIL: null_req
==============

*** TESTING CXX    null_req: Call H5Dwrite with empty selection
HDF5-DIAG: Error detected in HDF5 (1.13.0-7) MPI-process 0:
  #000: H5VL.c line 87 in H5VLregister_connector(): library initialization failed
    major: Function entry/exit
    minor: Unable to initialize object
  #001: H5.c line 279 in H5_init_library(): unable to initialize vol interface
    major: Function entry/exit
    minor: Unable to initialize object
  #002: H5VLint.c line 198 in H5VL_init_phase2(): unable to set default VOL connector
    major: Virtual Object Layer
    minor: Can't set value
  #003: H5VLint.c line 440 in H5VL__set_def_conn(): can't register connector
    major: Virtual Object Layer
    minor: Unable to register new ID
  #004: H5VLint.c line 1372 in H5VL__register_connector_by_name(): unable to load VOL connector
    major: Virtual Object Layer
    minor: Unable to initialize object
HDF5-DIAG: Error detected in HDF5 (1.13.0-7) MPI-process 0:
  #000: H5Pfapl.c line 6114 in H5Pset_vol(): not a file VOL ID
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.0-7) MPI-process 0:
  #000: H5D.c line 1225 in H5Dwrite(): can't synchronously write data
    major: Dataset
    minor: Write failed
  #001: H5D.c line 1172 in H5D__write_api_common(): can't write data
    major: Dataset
    minor: Write failed
  #002: H5VLcallback.c line 2181 in H5VL_dataset_write(): dataset write failed
    major: Virtual Object Layer
    minor: Write failed
  #003: H5VLcallback.c line 2148 in H5VL__dataset_write(): dataset write failed
    major: Virtual Object Layer
    minor: Write failed
  #004: H5VLnative_dataset.c line 345 in H5VL__native_dataset_write(): can't write data
    major: Dataset
    minor: Write failed
  #005: H5Dio.c line 374 in H5D__write(): src and dest dataspaces have different number of elements selected
    major: Invalid arguments to routine
    minor: Bad value
Error at line 69 in null_req.cpp:
fail with 1 mismatches.
FAIL null_req (exit status: 1)

[1] https://github.com/hyoklee/vol-log-based/actions/runs/1397442070

qmcpack fails to open dataset

Looks like it is calling the native VOL instead of the LOG VOLs when trying to open the dataset.

All the binaries and scripts are on summit login4 in qmcpack/build_summit_cpu/bin to reproduce.

HDF5-DIAG: Error detected in HDF5 (1.13.0) MPI-process 0:
#000: ../../src/H5VLcallback.c line 4406 in H5VLgroup_open(): unable to open group
major: Virtual Object Layer
minor: Unable to initialize object
#1: ../../src/H5VLcallback.c line 4335 in H5VL__group_open(): group open failed
major: Virtual Object Layer
minor: Can't open object
#2: ../../src/H5VLnative_group.c line 154 in H5VL__native_group_open(): unable to open group
major: Symbol table
minor: Can't open object
#3: ../../src/H5Gint.c line 397 in H5G__open_name(): group not found
major: Symbol table
minor: Object not found
#4: ../../src/H5Gloc.c line 439 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#5: ../../src/H5Gtraverse.c line 838 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#6: ../../src/H5Gtraverse.c line 614 in H5G__traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#7: ../../src/H5Gloc.c line 396 in H5G__loc_find_cb(): object 'state_0' doesn't exist
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.13.0) MPI-process 0:
#000: ../../src/H5VLcallback.c line 2016 in H5VLdataset_open(): unable to open dataset
major: Virtual Object Layer
minor: Can't open object
#1: ../../src/H5VLcallback.c line 1945 in H5VL__dataset_open(): dataset open failed
major: Virtual Object Layer
minor: Can't open object
#2: ../../src/H5VLnative_dataset.c line 252 in H5VL__native_dataset_open(): unable to open dataset
major: Dataset
minor: Can't open object
#3: ../../src/H5Dint.c line 1437 in H5D__open_name(): not found
major: Dataset
minor: Object not found
#4: ../../src/H5Gloc.c line 439 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#5: ../../src/H5Gtraverse.c line 838 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#6: ../../src/H5Gtraverse.c line 614 in H5G__traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#7: ../../src/H5Gloc.c line 396 in H5G__loc_find_cb(): object 'nprocs_nthreads_statesize' doesn't exist
major: Symbol table
minor: Object not found

Compilation fails.

The previous commit [1] compiled fine but the latest doesn't:

     336     #include "H5VL_logi_profiling.hpp"
     337              ^~~~~~~~~~~~~~~~~~~~~~~~~
     338    compilation terminated.

I tested it on Linux 3.10.0-1160.24.1.el7.x86_64 #1 SMP Thu Apr 8 19:51:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux.

[1] b13778e

strncpy

n = strlen (name);
if (n == 0) { ERR_OUT ("Object name cannot be empty") }
if (name[0] == '_') {
ret = (char *)malloc (n + 2);
ret[0] = '_';
strncpy (ret + 1, name, n + 1);
} else {
ret = (char *)name;
}
return ret;

This code block is problematic.

  1. strncpy cannot copy more than strlen(name) from name.
  2. How is ret freed latter if name[0] == '_' ?

Unable to find H5VL_logi_profiling.m4 files

The Autotools configure assume in-source builds, for out-of-source builds, it can not find the m4 files.

Making all in src
make[2]: Entering directory '/autofs/nccs-svm1_home1/brtnfld/packages/vol-log-based/build/src'
m4 -I../../m4 ../../src/H5VL_logi_profiling.m4h >H5VL_logi_profiling.hpp
m4:../../src/H5VL_logi_profiling.m4h:16: cannot open `H5VL_logi_profiling_timers.m4': No such file or directory

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.