steven-varga / h5cpp Goto Github PK
View Code? Open in Web Editor NEWC++17 templates between [stl::vector | armadillo | eigen3 | ublas | blitz++] and HDF5 datasets
Home Page: http://h5cpp.org
License: Other
C++17 templates between [stl::vector | armadillo | eigen3 | ublas | blitz++] and HDF5 datasets
Home Page: http://h5cpp.org
License: Other
TL;DR
h5::gzip
is used and one chunk is t0o small.Long version:
I am trying to append data and it did not work.
Since i was not sure whether I am doing something wrong, I decided to check the current master for examples and tests.
There i found this problem:
$ ./tests/test-packettable
$ h5dump -d "/stream of struct" test.h5
HDF5 "test.h5" {
DATASET "/stream of struct" {
DATATYPE H5T_COMPOUND {
H5T_STD_U16LE "field1";
H5T_IEEE_F64LE "field2";
H5T_IEEE_F32LE "field3";
H5T_STD_I8LE "field4";
H5T_IEEE_F64LE "field5";
H5T_IEEE_F64LE "field6";
H5T_IEEE_F64LE "field7";
H5T_IEEE_F64LE "field8";
H5T_IEEE_F64LE "field9";
}
DATASPACE SIMPLE { ( 0 ) / ( H5S_UNLIMITED ) }
DATA {
}
}
}
The test test-packettable
creates an empty dataset /stream of struct
even though it writes 200 elements.
I looked at the example examples-packettable
, since it also uses append.
In this example I looked at the dataset /stream of integral
which is created with:
h5::pt_t pt = h5::create<int>(fd, "stream of integral",
h5::max_dims{H5S_UNLIMITED,3,5}, h5::chunk{2,3,5} | h5::gzip{9} | h5::fill_value<int>(3) );
If I dump it, I get this:
$ ./examples/examples-packettable
$ h5dump -d "/stream of integral" example.h5
HDF5 "example.h5" {
DATASET "/stream of integral" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 6, 3, 5 ) / ( H5S_UNLIMITED, 3, 5 ) }
DATA {
(0,0,0): 1, 2, 3, 4, 5,
(0,1,0): 6, 7, 8, 9, 10,
(0,2,0): 11, 12, 13, 14, 15,
(1,0,0): 16, 17, 18, 19, 20,
(1,1,0): 21, 22, 23, 24, 25,
(1,2,0): 26, 27, 28, 29, 30,
(2,0,0): 31, 32, 33, 34, 35,
(2,1,0): 36, 37, 38, 39, 40,
(2,2,0): 41, 42, 43, 44, 45,
(3,0,0): 46, 47, 48, 49, 50,
(3,1,0): 51, 52, 53, 54, 55,
(3,2,0): 56, 57, 58, 59, 60,
(4,0,0): 61, 62, 63, 64, 65,
(4,1,0): 66, 67, 68, 69, 70,
(4,2,0): 71, 72, 73, 74, 75,
(5,0,0): 76, 77, 78, 79, 80,
(5,1,0): 81, 82, 83, 0, 0,
(5,2,0): 0, 0, 0, 0, 0
}
}
}
Here the values are written, but the fill value is 0 instead of 3,
Since my data is one dimensional, I added this code to the example:
try { // centrally used error handling
std::vector<int> stream(83);
std::iota(std::begin(stream), std::end(stream), 1);
// the leading dimension is extended once chunk is full, chunk is filled in row major order
// zero copy writes directly to chunk buffer then pushed through filter chain if specified
// works up to H5CPP_MAX_RANK default to 7
// last chunk if partial filled with h5::fill_value<T>( some_value )
h5::pt_t pt = h5::create<int>(fd, "stream of integral2",
h5::max_dims{H5S_UNLIMITED}, h5::chunk{2} | h5::gzip{9} | h5::fill_value<int>(3) );
for( auto record : stream )
h5::append(pt, record);
} catch ( const h5::error::any& e ){
std::cerr << "ERROR:" << e.what();
}
It is a copy of /stream of integral
, but I changed the dimensions and chunk size (and name).
$ ./examples/examples-packettable
$ h5dump -d "/stream of integral2" example.h5
HDF5 "example.h5" {
DATASET "/stream of integral2" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 83 ) / ( H5S_UNLIMITED ) }
DATA {h5dump error: unable to print data
}
}
}
Dumping it, I get corrupted data (but the dimensions are correct).
I got the same results with:
h5::max_dims{H5S_UNLIMITED }, h5::chunk{1 }
h5::max_dims{H5S_UNLIMITED,1 }, h5::chunk{2,1 }
h5::max_dims{H5S_UNLIMITED,2 }, h5::chunk{2,2 }
h5::max_dims{H5S_UNLIMITED,1,1}, h5::chunk{2,1,1}
h5::max_dims{H5S_UNLIMITED,2,1}, h5::chunk{2,1,1}
h5::max_dims{H5S_UNLIMITED,2,1}, h5::chunk{2,2,1}
h5::max_dims{H5S_UNLIMITED,2,2}, h5::chunk{2,2,1}
h5::max_dims{H5S_UNLIMITED,1,2}, h5::chunk{2,1,2}
h5::max_dims{H5S_UNLIMITED }, h5::chunk{5 }
It worked with (correct values, but wrong fill value):
h5::max_dims{H5S_UNLIMITED,2,2 }, h5::chunk{2,2,2 }
h5::max_dims{H5S_UNLIMITED,2,2,2}, h5::chunk{2,2,2,2}
h5::max_dims{H5S_UNLIMITED,2,2,1}, h5::chunk{2,2,2,1}
h5::max_dims{H5S_UNLIMITED,8 }, h5::chunk{2,8 }
h5::max_dims{H5S_UNLIMITED }, h5::chunk{16 }
h5::max_dims{H5S_UNLIMITED }, h5::chunk{ 8 }
h5::max_dims{H5S_UNLIMITED }, h5::chunk{ 6 }
If I remove h5::gzip{9}
, i can use h5::max_dims{H5S_UNLIMITED}, h5::chunk{2}
.
Hence my guess is: This problem is caused by the minimum gzip block size (or at least the minimum size in the implementation used by hdf5).
I think some diagnostic would be good, since this error is not easy to track down.
I can definitely understand the usage of C++17, but it's still a bit too new for my usages.
I'm not sure how many c++17 features h5cpp uses but one I am seeing is if-constexpr variations.
I see some attempt at constexpr version handling with h5cpp__constexpr, but it seems there's several instances of the if constexprs. I wonder if this is all that stands in the way?
Is there a Windows support?
Can I use this library in the Windows 10 x64 environment?
Complex numbers are prevalent in signal processing and more generally scientific computing. BLAS | LAPACK supports complex numbers therefore it is reasonable to have the ability to store the content of objects with complex datatypes in a straightforward way. While the HDF5 CAPI doesn't provide predefined complex datatypes ie: H5T_IEEE_C32LE
is not present, the library allows creating a custom representation of such types.
Since the in memory layout of std::complex
is guaranteed to be contiguous by the c++11 standard section ยง26.4 see this stack overflow notes the implementation is indeed straightforward and will follow pattern with the newly added half float dataypes
This section is opened to leave comments/observations for the upcoming std::complex
support.
g++ ublas.cpp -lhdf5 -lz -ldl -lm -o ublas
HDF5-DIAG: Error detected in HDF5 (1.11.3) thread 0:
#000: H5Dio.c line 322 in H5Dwrite(): could not get a validated dataspace from file_space_id
major: Invalid arguments to routine
minor: Bad value
#001: H5S.c line 254 in H5S_get_validated_dataspace(): selection + offset not within extent
major: Dataspace
minor: Out of range
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 1.0000 3.0000 1.0000 3.0000 1.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 1.0000 3.0000 1.0000 3.0000 1.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000 3.0000
HDF5-DIAG: Error detected in HDF5 (1.11.3) thread 0:
#000: H5Dio.c line 322 in H5Dwrite(): could not get a validated dataspace from file_space_id
major: Invalid arguments to routine
minor: Bad value
#001: H5S.c line 254 in H5S_get_validated_dataspace(): selection + offset not within extent
major: Dataspace
minor: Out of range
HDF5-DIAG: Error detected in HDF5 (1.11.3) thread 0:
#000: H5Dio.c line 322 in H5Dwrite(): could not get a validated dataspace from file_space_id
major: Invalid arguments to routine
minor: Bad value
#001: H5S.c line 254 in H5S_get_validated_dataspace(): selection + offset not within extent
major: Dataspace
minor: Out of range
HDF5-DIAG: Error detected in HDF5 (1.11.3) thread 0:
#000: H5Dio.c line 322 in H5Dwrite(): could not get a validated dataspace from file_space_id
major: Invalid arguments to routine
minor: Bad value
#001: H5S.c line 254 in H5S_get_validated_dataspace(): selection + offset not within extent
major: Dataspace
minor: Out of range
HDF5-DIAG: Error detected in HDF5 (1.11.3) thread 0:
#000: H5Dio.c line 322 in H5Dwrite(): could not get a validated dataspace from file_space_id
major: Invalid arguments to routine
minor: Bad value
#001: H5S.c line 254 in H5S_get_validated_dataspace(): selection + offset not within extent
major: Dataspace
minor: Out of range
make[1]: Leaving directory '/home/ubuntu/h5cpp/examples/linalg'
make -C raw_memory
make[1]: Entering directory '/home/ubuntu/h5cpp/examples/raw_memory'
g++ -std=c++11 -Wno-deprecated -c -o raw.o raw.cpp
g++ raw.o -lhdf5 -lz -ldl -lm -o raw
valued read: 2 3 4 5 6 7 8 9 0 0
h5dump -d dataset raw.h5
HDF5 "raw.h5" {
DATASET "dataset" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 1, 10 ) / ( 1, 10 ) }
DATA {
(0,0): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
}
}
}
Following the H5Cpp INSTALL
file documentation, I'm trying to configure and install the HDF5 library in my Ubuntu machine.
I downloaded the lasted HDF5 source code from here: official website
Here is my output:
vm@vm:~/Desktop/hdf5-1.10.6$ sudo ./configure --prefix=/usr/local --enable-build-mode=production --enable-shared --enable-static --enable-optimization=high --with-default-api-version=v110
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 3: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 5: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 8: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 13: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 18: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 21: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 26: $'\r': command not found
/home/vm/Desktop/hdf5-1.10.6/bin/missing: line 32: syntax error near unexpected token `$'in\r''
'home/vm/Desktop/hdf5-1.10.6/bin/missing: line 32: `case $1 in
configure: WARNING: 'missing' script is too old or missing
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
configure: error: cannot run /bin/bash bin/config.sub
How can I fix it?
My setup is:
OS: Ubuntu 18 x64
HDF5 Library: 1.10.6
gcc version: 7.4.0
CMake version: 3.10.2
Steve,
I compiled h5cpp using HDF5 1.10.6. When I run the compound
example, I'm getting an exception in H5Pall.hpp on line 260:
const static h5::elink_acc_flags acc_rdwr{H5F_ACC_RDWR};
No problems with 1.10.5.
If you don't see the same issue with your Linux environment, let me know and I'll provide much more detail.
I simplified most of the code in H5Meigen.hpp
by using Eigen base types.
There are 4 parts: size
, data
, get
and decay
.
For size
, data
, it is straightforward to write the function like this (handles all of Matrix
and Array
, bot row-major and column-major):
template<class T>
typename T::Scalar* data(const ::Eigen::PlainObjectBase<T>& ref ){
return const_cast<typename T::Scalar*>( ref.data() );
}
template<class T>
inline std::array<size_t,2> size( const ::Eigen::PlainObjectBase<T>& ref){
if constexpr(T::IsRowMajor) return {(hsize_t)ref.rows(),(hsize_t)ref.cols()};
return {(hsize_t)ref.cols(),(hsize_t)ref.rows()};
}
The others are not as straightforward, as they are class templates (which are matched differently than function templates, like data
and size
โ those use overload resolution).
For get
, I had to change the base template in H5Mstl.hpp, adding typename = void
:
template <class T, typename = void> struct get {
static inline T ctor( std::array<size_t,impl::rank<T>::value> dims ){
return T(); }};
so that the Eigen template is then more specialized:
template<class T>
struct get<T,std::enable_if_t<std::is_base_of_v<Eigen::PlainObjectBase<T>,T>>>{
static inline T ctor( std::array<size_t,2> dims ){
if constexpr(T::IsRowMajor) return T( dims[0], dims[1] );
return T( dims[1], dims[0] );
}};
Finally, for decay
, I could not use the extra parameter trick as decay
is declared with ellipsis template <T, ...>
, so no solution for that one so far.
What do you think? Would this be worth putting as a pull request?
Incidentally, I was checking out HighFive recently (they use too much whitespace in the code for my taste, though ;) ) and was helping there with a similar issue. The most elegant solution seemed to be to put all such common functions into a class template, one for each supported array types (one has to write the type-matching code only once, for the whole class).
An example is here: https://gist.github.com/eudoxos/43f11cfb04d33b4f3e5cf309ad628365 : io_impl
is specialized for each supported array type, containing all the functions. For h5cpp, those 4 would be obviously decay
, data
, size
and ctor
and they would be called as (e.g.) h5::impl::support<T>::decay
rather than h5::impl::decay<T>
and so on.
error:
auto some_fapl_prop = h5::libver_bounds(H5F_LIBVER_LATEST, H5F_LIBVER_LATEST); or h5::libver_bounds{H5F_LIBVER_LATEST, H5F_LIBVER_LATEST}; fails to parse properly @gheber
workaround is to embed arguments in an extra pair of braces, to force the template parser to recognize std::tuple. This suggested workaround will work even after a fix is provided.
correct syntax until fix:
h5::libver_bounds( { H5F_LIBVER_LATEST, H5F_LIBVER_LATEST } );
or the less visually appealing but functional:
h5::libver_bounds {{ H5F_LIBVER_LATEST, H5F_LIBVER_LATEST }};
explanation:
the arguments require std::tuple<args type> and the correct way to denote this is {arg0, arg1, ..., argn} which is turned into:
std::tuple<type0,type1,...,typen>( arg0, arg1, ..., argn );
this is a compile time operation, doesn't affect performance
H5CPP increasing popularity justifies the survey of compiler/OS compatibility starting with Linux, expanding to other mainstream Unix Like OS-s and Windows OS. This thread is to report and accumulate user experiences then take action on acute cases; as well as to aim for thorough treatment across major platform and compilers providing much similar experience as of the HDF5 CAPI.
To facilitate this process feel free to report success or failure on this thread filling in the blanks:
OS: Linux (distro name) | Apple (MAC | IOS | ...) | Windows (version) | ...
C++ Compiler: clang | gcc | ...
version
STL: if different from default
HDF5 CAPI: version
serial | parallel | VOL
File System: local | pvfs | lustre | Kita S3 | ...
Result: success | failure
Current C++17 compilers are not quite accommodating when reporting template (meta) programming errors, to limit the verbose output please do: make my_target |& head -n20
.
Return statement missing from = operator overload in H5Dappend.hpp (lines 26-28):
pt_t& operator=( h5::pt_t&& pt ){
init(pt.ds);
}
Hi, I would like to add support for OpenCV matrices (cv::Mat
). All supported matrix classes in H5M*
are templated based on scalar type (plus other things); cv::Mat
, on the other hand, stores its datatype internally. I usually use a simple switch/case function to map OpenCV type constants to HDF5 constants and vice verse.
Could I get some guidance how to hook that into h5cpp? I appreciate very much compactness of the code, esthetically and as programmer, but need a bit of help at first :)
Steven, please confirm whether or not you see this issue when compiling/running "string" example in Linux. Is this a general issue or only in VC++ that I should look into fixing for VC++.
Issue: H5CPP cannot read a string dataset into std::vector<std::string>
.
Dev Environment: Visual Studio 2017 (141) compiling as x64 with Win 8.1 64-bit.
Issue was discovered and is reproducible while testing the "string" example project.
Details:
Writing std::vector<std::string>
to a dataset works properly.
But reading the same dataset back into a std::vector<std::string>
triggers a std::bad_alloc
exception when iterating the returned vector.
If you read the dataset into a std::vector<const char*>
, the dataset is read properly and iterates without exception.
However, using std::vector<const char*>
won't compile because it can't get by:
static_assert( utils::is_supported<T>, "error: " H5CPP_supported_elementary_types );
in H5Dread.hpp, line 36.
Commenting out that line allows for compilation and successful execution.
The code below (essentially from string.cpp) is a working example.
// CHD !!! This is important for Windows!
#define H5_BUILT_AS_DYNAMIC_LIB
/* Copyright (c) 2018 vargaconsulting, Toronto,ON Canada
* Author: Varga, Steven <[email protected]>
*/
#include <vector>
#include <string>
#include <h5cpp/all>
int main(){
//RAII will close resource, noo need H5Fclose( any_longer );
h5::fd_t fd = h5::create("example.h5",H5F_ACC_TRUNC);
{
// CHD: This works correctly.
std::vector<std::string> vec = h5::utils::get_test_data<std::string>(20);
h5::write(fd, "/strings.txt", vec);
}
{
// CHD: This will result in a std::bad_alloc exception when the returned
// std::vector<std::string> is iterated.
//auto vec = h5::read<std::vector<std::string>>(fd, "strings.txt");
//for( auto i : vec )
//std::cout << i <<"\n";
// CHD: This will return vector properly and will iterate properly.
// But only if the utils::is_supported assert in H5Dread.hpp, line
// 36 is commented out.
auto vec = h5::read<std::vector<const char*>>(fd, "strings.txt");
for( auto i : vec )
std::cout << i <<"\n";
}
}
Where the Matrix is just not enough, there exists Tensor.
This page is to report request or possible mishaps while I am adding tensor support to H5CPP. The list here lifted from StackOverflow
h5cpp struct.cpp -- -std=c++11 -I/usr/local/include -I/usr/local/h5cpp-llvm -Dgenerated.h
H5CPP: Copyright (c) 2018 , VargaConsulting, Toronto,ON Canada
LLVM : Copyright (c) 2003-2010, University of Illinois at Urbana-Champaign.
g++ -o struct.o -std=c++11 -I/usr/local/include -I/usr/local/h5cpp-llvm -c struct.cpp
g++ struct.o -lhdf5 -lz -ldl -lm -o struct
./struct
HDF5-DIAG: Error detected in HDF5 (1.11.4) thread 0:
#000: H5D.c line 164 in H5Dcreate2(): unable to create dataset
major: Dataset
minor: Unable to initialize object
#001: H5VLcallback.c line 1787 in H5VL_dataset_create(): dataset create failed
major: Virtual Object Layer
minor: Unable to create file
#002: H5VLcallback.c line 1754 in H5VL__dataset_create(): dataset create failed
major: Virtual Object Layer
minor: Unable to create file
#003: H5VLnative_dataset.c line 76 in H5VL__native_dataset_create(): not a datatype ID
major: Invalid arguments to routine
minor: Inappropriate type
terminate called after throwing an instance of 'h5::error::io::dataset::create'
what(): /usr/local/include/h5cpp/H5capi.hpp line# 186 : couldn't create dataset...
error:
redefinition of H5CPP_REGISTER_STRUCT( ... )
At most only one unique symbol may be present in TU translation unit. Include guards are a good method to prevent errors in the linking phase.
workaround: be certain the generated header file is included only once.
Hello,
I have a couple of pull requests I'd like to submit, one of which enables Eigen::Tensor
objects. Before I do so is it possible for you to add a .clangformat
file, so I can ensure I stick with your chosen style?
Please include <cstring>
in h5cpp/core
.
Can an example be provided for creating a group, assigning attributes to that group, and assigning one or more datasets to the group?
Perhaps I am missing something.
Due to template system rework the newly added attributes can only handle armadillo objects in addition to scalars, strings and arrays.
Hi, I copy and paste this short example source-code in a new Qt C++ console project (using C++17)
Code example link: http://h5cpp.org/eigen3_8cpp-example.html
And I got these compile time errors:
So I tryed another simple Eigen example:
#include <QCoreApplication>
#include <iostream>
#include <Eigen/Dense> // must include Eigen before <h5cpp/core>
#include <h5cpp/all>
using Eigen::MatrixXd;
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
MatrixXd m(2,2);
m(0,0) = 3;
m(1,0) = 2.5;
m(0,1) = -1;
m(1,1) = m(1,0) + m(0,1);
h5::fd_t fd = h5::create("some_file.h5", H5F_ACC_TRUNC);
h5::write(fd,"/result", m);
return a.exec();
}
And I got a lot of compilation errors again:
What I have to do?
Compiling with VC++ in Visual Studio 2017, the compiler complains about strdup usage:
Error C4996 'strdup': The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _strdup.
It is referring to line 88 in H5Dwrite.hpp:
try {
for( const auto& reference:ref)
ptr.push_back( strdup( reference.data()) );
} catch( ... ){
throw h5::error::io::dataset::write( h5::error::msg::mem_alloc );
}
Hi,
I am finding compilation or run time issues when attempting to use std::string
and std::vector<std::string>
to read dataset attributes.
Can you confirm that you see these issues too?
Perhaps I am missing something in my code
#include <iostream>
#include <string>
#include <vector>
#include "h5cpp/all"
int main()
{
// Create a file.
h5::fd_t fd = h5::create("a.h5", H5F_ACC_TRUNC, h5::default_fcpl, h5::libver_bounds({ H5F_LIBVER_LATEST, H5F_LIBVER_LATEST }));
// Add a simple dataset.
std::vector<int> data = { 1,2,3,4,5 };
h5::ds_t ds = h5::write(fd, "/test/path", data, h5::create_path | h5::utf8);
// Employ the two methods for adding attributes to dataset to add std::string's.
std::string str1 = "std::string #1";
ds["attr1"] = str1;
std::string str2 = "std::string #2";
h5::awrite(ds, "attr2", str2);
// Employ the two methods for added attributes to dataset to add std::vector<std::string>'s.
std::vector<std::string> vs1 = { "Millicent", "Judith", "Jennifer" };
ds["attr3"] = vs1;
std::vector<std::string> vs2 = { "Duke", "Melvin", "Claude" };
h5::awrite(ds, "attr4", vs2);
// Read and dump the two std::string attributes.
auto attr_1 = h5::aread<std::string>(ds, "attr1");
std::cout << "attr1 : " << attr_1 << std::endl;
auto attr_2 = h5::aread<std::string>(ds, "attr2");
std::cout << "attr2 : " << attr_2 << std::endl;
// Read and dump the two std::vector<std::string> attributes.
auto attr_3 = h5::aread<std::vector<std::string>>(ds, "attr3");
std::cout << "attr3 : ";
for (auto v : attr_3)
std::cout << v << ", ";
std::cout << std::endl;
auto attr_4 = h5::aread<std::vector<std::string>>(ds, "attr4");
std::cout << "attr4 : ";
for (auto v : attr_4)
std::cout << v << ", ";
std::cout << std::endl;
}
I think I see 2 spaces where std::array should be incorporated
The first is in simply writing or reading std::array<T, N> to a dataset/attribute.
The second is writing a collection of std::array<T, N>, like std::vector<std::array<T, N>>
This seems like a really great tool and would like to be a user, however I am wondering what the future of this project will be in terms of vision and timing. It sounds like maybe this is on the development back burner due to other exigencies or funding. If there is a better forum for this I am happy to move this somewhere else. I am mostly curious where it's going or could go.
A recent error report by @mrj10 motivated to check compiler compatibility, and as it turns out the LLVM based Clang toolchain is crashing while parsing the C++ templates.
The workaround is to use gcc 7.4.0, gcc 8.3.0, gcc 9.0.1 or clang 6.0.0 until the problem is located and fixed.
Error report: clang_9.0.1-error.tar.gz
I tried following both the automatic instructions using conan
and cmake
, but neither one is installing correctly. Conan seems to work, but then when I try to import h5cpp.h
in my code I get the error:
CMake Error at CMakeLists.txt:34 (find_package):
By not providing "Findh5cpp.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "h5cpp", but
CMake did not find one.
Could not find a package configuration file provided by "h5cpp" with any of
the following names:
h5cppConfig.cmake
h5cpp-config.cmake
Add the installation prefix of "h5cpp" to CMAKE_PREFIX_PATH or set
"h5cpp_DIR" to a directory containing one of the above files. If "h5cpp"
provides a separate development package or SDK, be sure it has been
installed.
-- Configuring incomplete, errors occurred!
See also "/home/glicka/GEANT/B4a/build/CMakeFiles/CMakeOutput.log".
And when I try to install h5cpp
using cmake
I get the error:
ERROR: boost_filesystem/1.65.1@bincrafters/stable: 'settings.compiler' value not defined
CMake Error at cmake/conan.cmake:322 (message):
Conan install failed='1'
Call Stack (most recent call first):
cmake/conan.cmake:399 (conan_cmake_install)
CMakeLists.txt:21 (conan_cmake_run)
-- Configuring incomplete, errors occurred!
See also "/home/glicka/h5cpp-0.1.0/build/CMakeFiles/CMakeOutput.log".
What do I need to do to get this to work?
I tried to compile ToT of the master
branch on macOS, using the default Xcode clang++-based toolchain, and ran into the errors below.
It could be that the first error is the cause of all the rest, I'm not sure.
A using
directive with the same name as the enclosing class does seem like a legitimate complaint, but I'm not enough of a template guru to figure out how to fix it :)
In file included from /Users/mrj10/h5cpp/examples/basics/basics.cpp:1:
In file included from /Users/mrj10/h5cpp/h5cpp/all:8:
In file included from /Users/mrj10/h5cpp/h5cpp/core:50:
/Users/mrj10/h5cpp/h5cpp/H5Tall.hpp:24:17: error: member 'hid_t' has the same name as its class
using parent::hid_t; // is a must because of ds_t{hid_t} ctor
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::fapl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_fapl, h5::impl::capi_t<long long, H5F_libver_t, H5F_libver_t>, &H5Pset_libver_bounds>,
h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:215:32: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true,
1>, &h5::impl::default_fapl, h5::impl::capi_t<long long, H5F_libver_t, H5F_libver_t>, &H5Pset_libver_bounds>' requested here
const static h5::libver_bounds latest_version({H5F_LIBVER_LATEST, H5F_LIBVER_LATEST});
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::lcpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::lcpl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::lcpl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_lcpl, h5::impl::capi_t<long long, H5T_cset_t>, &H5Pset_char_encoding>, h5::impl::detail::hid_t<h5::impl::lcpl_t,
&H5Pclose, true, true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:242:32: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::lcpl_t, &H5Pclose, true, true,
1>, &h5::impl::default_lcpl, h5::impl::capi_t<long long, H5T_cset_t>, &H5Pset_char_encoding>' requested here
const static h5::char_encoding ascii{H5T_CSET_ASCII};
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::lapl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::lapl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::lapl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_lapl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_elink_acc_flags>, h5::impl::detail::hid_t<h5::impl::lapl_t,
&H5Pclose, true, true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:254:34: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::lapl_t, &H5Pclose, true, true,
1>, &h5::impl::default_lapl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_elink_acc_flags>' requested here
const static h5::elink_acc_flags acc_rdwr{H5F_ACC_RDWR};
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::dcpl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_dcpl, h5::impl::capi_t<long long>, &H5Pset_fletcher32>, h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true,
true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:279:31: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true,
1>, &h5::impl::default_dcpl, h5::impl::capi_t<long long>, &H5Pset_fletcher32>' requested here
const static flag::fletcher32 fletcher32;
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::ocrl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::ocrl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::ocrl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_ocrl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_attr_creation_order>,
h5::impl::detail::hid_t<h5::impl::ocrl_t, &H5Pclose, true, true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:316:38: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::ocrl_t, &H5Pclose, true, true,
1>, &h5::impl::default_ocrl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_attr_creation_order>' requested here
const static h5::attr_creation_order crt_order_tracked{H5P_CRT_ORDER_TRACKED};
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::ocpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:15:28: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::ocpl_t, &H5Pclose, true, true, 1>' requested here
using hidtype = typename phid_t::parent::hidtype;
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:69:18: note: in instantiation of template class 'h5::impl::prop_base<h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::ocpl_t,
&H5Pclose, true, true, 1>, &h5::impl::default_ocpl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_copy_object>, h5::impl::detail::hid_t<h5::impl::ocpl_t,
&H5Pclose, true, true, 1> >' requested here
struct prop_t : prop_base<prop_t<phid_t,init,capi,capi_call>,phid_t> {
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:323:30: note: in instantiation of template class 'h5::impl::prop_t<h5::impl::detail::hid_t<h5::impl::ocpl_t, &H5Pclose, true, true,
1>, &h5::impl::default_ocpl, h5::impl::capi_t<long long, unsigned int>, &H5Pset_copy_object>' requested here
const static h5::copy_object shallow_hierarchy{H5O_COPY_SHALLOW_HIERARCHY_FLAG};
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::acpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:382:33: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::acpl_t, &H5Pclose, true, true, 1>' requested here
const static h5::acpl_t acpl = static_cast<h5::acpl_t>( H5P_DEFAULT );
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:383:33: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::dcpl_t' (aka 'hid_t<h5::impl::dcpl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::dcpl_t dcpl = static_cast<h5::dcpl_t>( H5P_DEFAULT);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::dxpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:384:33: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::dxpl_t, &H5Pclose, true, true, 1>' requested here
const static h5::dxpl_t dxpl = static_cast<h5::dxpl_t>( H5P_DEFAULT );
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:385:66: error: invalid operands to binary expression ('h5::char_encoding' (aka 'prop_t<hid_t<h5::impl::lcpl_t, &H5Pclose, true, true,
detail::hdf5::property>, default_lcpl, h5::impl::capi_t<long long, H5T_cset_t>, &H5Pset_char_encoding>') and 'h5::create_intermediate_group' (aka
'prop_t<hid_t<h5::impl::lcpl_t, &H5Pclose, true, true, detail::hdf5::property>, default_lcpl, h5::impl::capi_t<long long, unsigned int>,
&H5Pset_create_intermediate_group>'))
const static h5::lcpl_t lcpl = h5::char_encoding{H5T_CSET_UTF8} | h5::create_intermediate_group{1};
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:386:33: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::fapl_t' (aka 'hid_t<h5::impl::fapl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::fapl_t fapl = static_cast<h5::fapl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
error: no member named '' in 'h5::impl::detail::hid_t<h5::impl::fcpl_t, &H5Pclose, true, true, 0>'
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:387:33: note: in instantiation of template class 'h5::impl::detail::hid_t<h5::impl::fcpl_t, &H5Pclose, true, true, 1>' requested here
const static h5::fcpl_t fcpl = static_cast<h5::fcpl_t>( H5P_DEFAULT );
^
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:389:41: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::acpl_t' (aka 'hid_t<h5::impl::acpl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::acpl_t default_acpl = static_cast<h5::acpl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::acpl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::acpl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
In file included from /Users/mrj10/h5cpp/examples/basics/basics.cpp:1:
In file included from /Users/mrj10/h5cpp/h5cpp/all:8:
In file included from /Users/mrj10/h5cpp/h5cpp/core:52:
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:390:41: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::dcpl_t' (aka 'hid_t<h5::impl::dcpl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::dcpl_t default_dcpl = static_cast<h5::dcpl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::dcpl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
In file included from /Users/mrj10/h5cpp/examples/basics/basics.cpp:1:
In file included from /Users/mrj10/h5cpp/h5cpp/all:8:
In file included from /Users/mrj10/h5cpp/h5cpp/core:52:
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:391:41: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::dxpl_t' (aka 'hid_t<h5::impl::dxpl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::dxpl_t default_dxpl = static_cast<h5::dxpl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::dxpl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::dxpl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
In file included from /Users/mrj10/h5cpp/examples/basics/basics.cpp:1:
In file included from /Users/mrj10/h5cpp/h5cpp/all:8:
In file included from /Users/mrj10/h5cpp/h5cpp/core:52:
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:392:74: error: invalid operands to binary expression ('h5::char_encoding' (aka 'prop_t<hid_t<h5::impl::lcpl_t, &H5Pclose, true, true,
detail::hdf5::property>, default_lcpl, h5::impl::capi_t<long long, H5T_cset_t>, &H5Pset_char_encoding>') and 'h5::create_intermediate_group' (aka
'prop_t<hid_t<h5::impl::lcpl_t, &H5Pclose, true, true, detail::hdf5::property>, default_lcpl, h5::impl::capi_t<long long, unsigned int>,
&H5Pset_create_intermediate_group>'))
const static h5::lcpl_t default_lcpl = h5::char_encoding{H5T_CSET_UTF8} | h5::create_intermediate_group{1};
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:393:41: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::fapl_t' (aka 'hid_t<h5::impl::fapl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::fapl_t default_fapl = static_cast<h5::fapl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::fapl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
In file included from /Users/mrj10/h5cpp/examples/basics/basics.cpp:1:
In file included from /Users/mrj10/h5cpp/h5cpp/all:8:
In file included from /Users/mrj10/h5cpp/h5cpp/core:52:
/Users/mrj10/h5cpp/h5cpp/H5Pall.hpp:394:41: error: no matching conversion for static_cast from 'hid_t' (aka 'long long') to 'h5::fcpl_t' (aka 'hid_t<h5::impl::fcpl_t,
&H5Pclose, true, true, detail::hdf5::property>')
const static h5::fcpl_t default_fcpl = static_cast<h5::fcpl_t>( H5P_DEFAULT );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'const h5::impl::detail::hid_t<h5::impl::fcpl_t, &H5Pclose, true, true, 1>' for 1st argument
struct hid_t final {
^
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:53:9: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'hid_t' (aka 'long long') to
'h5::impl::detail::hid_t<h5::impl::fcpl_t, &H5Pclose, true, true, 1>' for 1st argument
/Users/mrj10/h5cpp/h5cpp/H5Iall.hpp:112:9: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
struct hid_t<T,capi_close, true,true,hdf5::property> : public hid_t<T,capi_close,true,true,hdf5::any> {
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
When porting h5cpp to Visual C++ in the vs2017-windows
branch, I am only adapting file source code files in the h5cpp
directory. I am not adapting any other files from the master repo.
Perhaps a note to this effect should be added to the main README.md
for the vs2017-windows
branch.
Also, even though I have included MPI and Kita based h5cpp code in the vs2017-windows
branch, perhaps we should note that those feature areas will not work in a Windows environment.
If it is agreed upon, I'll be happy to provide a PR.
Hi,
I created a new repo for the Windows port of H5CPP. It is here:
https://github.com/ChrisDrozdowski/h5cpp-windows
Please modify links to the old repos on this to reflect the new repo.
My rationale is that it puts the h5cpp source and examples in one unified repo. I also included the source for Armadillo and Eigen so that the examples can be compiled out of box with minimal work.
I will be maintaining this repo as the Windows port from now on.
I have Eigen installed in /usr/local/include/eigen3
but still get this error:
h5cpp packettable.cpp -- -std=c++11 -I/usr/local/include -I/usr/local/h5cpp-llvm -Dgenerated.h
H5CPP: Copyright (c) 2018 , VargaConsulting, Toronto,ON Canada
LLVM : Copyright (c) 2003-2010, University of Illinois at Urbana-Champaign.
g++ -o packettable.o -std=c++11 -I/usr/local/include -I/usr/local/h5cpp-llvm -c packettable.cpp
packettable.cpp:7:10: fatal error: Eigen/Dense: No such file or directory
#include <Eigen/Dense> // must include Eigen before <h5cpp/core>
^~~~~~~~~~~~~
compilation terminated.
Not sure what's amiss.
Hello @steven-varga ,
I would like to serialize the following Eigen array, with specific number of columns and memory layout. From a quick read of documentation, I thought the following would work. But compilation log is full of errors. After a careful reading, I see that this king of array is not directly supported. Do I have to go to raw data pointer to serialize my array?
typedef Eigen::Array< double, Eigen::Dynamic, 3, Eigen::RowMajor > ArrayX3D;
ArrayX3D res2 = ArrayX3D::Zero(10, 3);
h5::write(fd, "/res", res2);
Below is my test program
#include <Eigen/Dense>
typedef Eigen::Array< double, Eigen::Dynamic, 1 > ArrayXD;
typedef Eigen::Array< double, Eigen::Dynamic, 3, Eigen::RowMajor > ArrayX3D;
#include <h5cpp/all>
int main()
{
ArrayXD res = ArrayXD::Zero(10);
h5::fd_t fd = h5::create("eigen.h5",H5F_ACC_TRUNC);
h5::write(fd, "/res", res);
ArrayX3D res2 = ArrayX3D::Zero(10, 3);
h5::write(fd, "/res2", res2);
// auto ds_3 = h5::create<double>(fd,"/type/short max_dims", h5::max_dims{res2.rows(),3});
// h5::write(fd, "/res2", res2,h5::current_dims{res2.rows(),3});
}
Hi,
I got one more issue, it's small for any pull request, so decided to describe it here and you will decide how to fix or maybe I'm just using it not correctly...
Ok,
the h5::dt_t has a move assignment operator
Line 25 in 9b9b90c
that not correctly working in the next situation
struct logger {
logger() {
fd = h5::create(...);
pt = h5::create( fd, "first_dataset", ...);
}
void createNextTable() {
pt = h5::create( fd, "name_of_next_dataset", ...);
}
private:
h5::pt_t pt{};
h5::fd_t fd{};
};
so, the line pt = h5::create( fd, "name_of_next_dataset", ...);
will call conversion constructor
Line 21 in 9b9b90c
pt.n
stores number of samples in cache of previous first_dataset
.
the next code
h5::pt_t pt = h5::create( fd, "first_dataset", ...);
std::cout << pt;
h5::append(pt, some_data );
h5::append(pt, some_data );
h5::append(pt, some_data );
pt.flush();
std::cout << pt;
pt = h5::create( fd, "next_dataset", ...);
std::cout << pt;
will generate the next output
packet table:
------------------------------------------
rank: 1 N:1000 n:0
element size: 24 block size: 24000
current dims: [0]
chunk dims: [0]
offset : [0] count : [1]
ds: 360287970189639680 dxpl: 720575940379279425
fill value: 0x55c6dfecc2a0 buffer: 0x55c6dfedff00
packet table:
------------------------------------------
rank: 1 N:3e8 n:3
element size: 18 block size: 5dc0
current dims: [3]
chunk dims: [3]
offset : [0] count : [1]
ds: 500000000000000 dxpl: a00000000000041
fill value: 0x55c6dfecc2a0 buffer: 0x55c6dfedff00
packet table:
------------------------------------------
rank: 1 N:3e8 n:3
element size: 18 block size: 5dc0
current dims: [0]
chunk dims: [0]
offset : [0] count : [1]
ds: 500000000000001 dxpl: a00000000000041
fill value: 0x55c6dfec3980 buffer: 0x55c6dfef7800
I did fixed it in a next way, however not sure that it's correct way...
pt_t& operator=( h5::pt_t&& pt ){
this->n = pt.n;
init(pt.ds);
return pt;
}
Is it correct to reuse h5::pt_t object as I did? Is dataset closed properly when I reuse it?
And one more question, is it really required to call init(pt.ds)
in move assignment, seems like it's done twice...
Regards, Max 73!
Hi! I want to know, is there any methods for me to use fp16 in hdf5-C++?
I intend to use the Half precision floating-point format for storing data that do not need as much precision in order to save space. So does this code support half-precision float storage?
I know that h5py provides an option to store data as fp16, like this:
file.create_dataset(key, data=data, dtype = np.float16)
But no predefined type provided for fp16 storage on the official documentation of hdf5 for C++ , linking here https://support.hdfgroup.org/HDF5/doc1.8/RM/PredefDTypes.html. And I found hdf5 has an C interface named H5Tset_precision, so does this interface have a corresponding implementation in h5cpp?
Currently the HDF5 library is hard to cross compile because of some internals, to ease on the process here are the binaries: hdf5-1.10.6-arm7f.tar.gz
The H5CPP library is currently broken for android armv7f platform due to the size differences between some integral types, however only minor adjustment is needed to make this port happen. Will keep this errata up until the fix is provided.
Attribute operations are not yet handled with h5cpp compiler, because of the difference in templating mechanism.
Expect fix in early November.
Steve,
Can you provide a brief explanation of the half float support code in H5Tall.hpp ? I don't quite see how the data type is registered, etc. Perhaps big naivety on my part though.
Hi Steven, good piece of obstruction; I really like this wrapper for hdf5.
The next code
h5::pt_t pt = h5::create<mc:sometype_t>(fd, getDataSetName(),
h5::max_dims{ H5S_UNLIMITED }, h5::chunk{ 1000 } | h5::gzip{9} );
std::cout << pt;
h5::append( pt, sample );
pt.flush();
std::cout << pt;
will generate output
packet table:
------------------------------------------
rank: 1 N:1000 n:0
element size: 24 block size: 24000
current dims: [0]
chunk dims: [0]
offset : [0] count : [1]
ds: 360287970189639680 dxpl: 720575940379279425
fill value: 0x562efb0632a0 buffer: 0x562efb076f00
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 0:
#000: H5Dio.c line 400 in H5Dwrite_chunk(): failure to copy offset array
major: Dataset
minor: Can't allocate space
#001: H5Dio.c line 120 in H5D__get_offset_copy(): offset doesn't fall on chunks's boundary
major: Dataspace
minor: Inappropriate type
packet table:
------------------------------------------
rank: 1 N:3e8 n:1
element size: 18 block size: 5dc0
current dims: [1]
chunk dims: [1]
offset : [0] count : [1]
ds: 500000000000000 dxpl: a00000000000041
fill value: 0x562efb0632a0 buffer: 0x562efb076f00
I almost didn't work with hdf5 before, is it some kind of limitation of packet tables or a bug.
I can spend some time to fix it if you give me some clues.
This is not common case to flush chunk with only one element, but I have a system where I don't have constant sample rate and flushing and moving to other table on a timeframe base, so quite possible.
Regards,
Max 73!
Check this out.
Below is an example of how one can add attributes to a pack table-generated dataset. This tripped me up for a bit, so an example may be useful to others.
Please adapt as you see fit to work with packet-table example.
I didn't add to the packet-table example myself because it needs to be tested in Linux which I can't currently do.
The important points are creating ds_t
, adding attributes to it, then casting to pt_t
for data appending.
#include <h5cpp/all>
#include <cstddef>
#include <vector>
int main(){
const size_t count = 2048;
const std::size_t chunk_size = 1024;
try
{
h5::fd_t fd = h5::create("example.h5", H5F_ACC_TRUNC);
// Create a dataset.
h5::ds_t ds = h5::create<double>(fd, "some_path", h5::max_dims{ H5S_UNLIMITED },
h5::chunk{ chunk_size } | h5::gzip{ 6 } | h5::fill_value<double>(0));
// Add attributes to dataset.
ds["attr1"] = 1.23;
ds["attr2"] = "string";
// Cast dataset to packet table.
h5::pt_t pt = static_cast<h5::pt_t>(ds);
// Append data to packet table.
for (std::size_t i = 0; i < count; ++i)
h5::append(pt, i * 0.999);
}
catch (const std::exception& e)
{
std::cout << e.what() << std::endl;
}
}
Visual Studio 2017 (v141)
Windows SDK Version: 10.0.17763.0
C++ Language Standard: ISO C++ Latest Draft Standard (/std:c++latest)
These issues is not caused by improvements provided by Commit: 8061fbd or this Pull Request: #20
Compiler reports the following errors when h5cpp/all
is included after windows.h
. If h5cpp/all
is included before windows.h
, these errors are not reported.
In: h5zpipeline.hpp
Error C2589 '(': illegal token on right side of '::' h5cpptest c:\users\c drozdowski\source\repos\h5cpptest\h5cpptest\h5cpp\h5zpipeline.hpp 184
Error C2589 '(': illegal token on right side of '::' h5cpptest c:\users\c drozdowski\source\repos\h5cpptest\h5cpptest\h5cpp\h5zpipeline.hpp 221
In: h5capi.hpp
Error C2589 '(': illegal token on right side of '::' h5cpptest c:\users\c drozdowski\source\repos\h5cpptest\h5cpptest\h5cpp\h5capi.hpp 217
I will attempt to find out why the errors are happening and will report if I can find out the answers.
Hello,
Looking at https://github.com/steven-varga/h5cpp/blob/master/examples/attributes/attributes.cpp it appears that string array attributes (std::aread<std::vector<std::string>>
) is supported. However, I can't find an example for only a single string, i.e. std::aread<std::string>
, and indeed, this fails to compile. Is this not supported? Is there a way to read a single string/char* attribute into a std::string?
Thanks in advance!
Clang 9.0.0 (LLVM) Visual Studio 2019 as IDE.
Using my h5cpp-windows
version of h5cpp, some examples compile fine, some fail (e.g. stl
) with this error:
h5cpp/H5Dwrite.hpp(60): error : call to 'write' is ambiguous
Do you have a suggestion on which direction to go to address this issue? Is it not seen on different platforms?
I want to confirm whether this is expected behavior or not for packet table functionality.
When appending a vector to h5:pt_t
and size of vector != chunk size, an exception is thrown.
(This occurs with my MSVC code- does it happen with Linux???)
Example code:
#include <h5cpp/all>
#include <cstddef>
#include <vector>
int main(){
const size_t count = 2048; // Some arbitrary length not known at compile time.
std::vector<double> data(count);
for (std::size_t i = 0; i < count; ++i) { data[i] = i * 0.123; }
const std::size_t chunk_size = 1024;
try
{
h5::fd_t fd = h5::create("pt1.h5", H5F_ACC_TRUNC);
h5::pt_t pt = h5::create<double>(fd, "some_path", h5::max_dims{ H5S_UNLIMITED },
h5::create_path | h5::utf8, h5::chunk{ chunk_size } | h5::fill_value<double>(0));
h5::append(pt, data); // Exception is thrown in this call.
}
catch (const std::exception& e)
{
std::cout << e.what() << std::endl;
}
}
Steven,
I created a separate repo for the MSVC specific H5CPP examples I've adapted:
https://github.com/ChrisDrozdowski/h5cpp-msvc-examples
I created it because it is essentially different than the structure of the H5CPP repo and I don't want to mix in Visual studio-specific files such a solution or project files.
Please let me know if you have any issues with this.
Since i am currently extending h5cpp in my fork and plan to make a pull request at some point, it would be useful to know the coding conventions for this project.
The best option would be to use a tool for reformatting (astyle, clang-tidy etc.).
This would make it much easier to collaborate and keep the same style throughout the project (this would improve readability and thus maintainability) since collaborators could configure their editor to apply the file.
./before
reading back data previously written:
1 0.5 0.333333 0.25 0.2 0.166667 0.142857 0.125 0.111111 0.1
./after
terminate called after throwing an instance of 'h5::error::io::dataset::create'
what(): /usr/local/include/h5cpp/H5capi.hpp line# 186 : couldn't create dataset...
using stock ubuntu 18.04LTS with minimal dependencies there are compile time errors triggered by the following HDF5 CAPI properties:
H5Pset_file_space_page_size, H5Pset_page_buffer_size, H5Pset_evict_on_close, H5Pset_mdc_image_config
sudo apt-get install -y build-essential libhdf5-serial-dev
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -
sudo sh -c 'echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main" >> /etc/apt/sources.list'
sudo sh -c 'echo "deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main" >> /etc/apt/sources.list'
sudo apt-get update
sudo apt-get -y install clang-tools-7 llvm-7-tools libclang-7-dev
git clone https://github.com/steven-varga/h5cpp.git
git checkout ubuntu-18.04LTS
cd h5cpp && make install
on dev suite: [email protected] the the offending lines are commented out
The recipe for the h5cpp
target in compiler/Makefile
should read
h5cpp: h5cpp.o
$(CXX) $^ $(LDFLAGS) $(LIBS) -o $@
Otherwise we get a linker error because the Clang libs aren't found.
error:
There are if constrexpr ( condition) { } in code base that prevent the code to parse as c++11
workaround: --std=c++17 until conditionally removing the offending constructs
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.