nanocomp / h5utils Goto Github PK

Dear,

The script of h5fromtxt cannot accept the txt file as input, right? It seems that the input here could only be the format of <<EOF

..
..
EOF

If the size of the txt is very large, how do you think to process it?

Thank you!

Sijia

I am using pymeep installed with conda from the conda-forge channel less than a month ago, running on MacOS 10.15.4 (Catalina). The pymeep script runs a meep simulation that produces two h5 files, using the built-in field output functions and at_append for time slices. There is one data set (1000 x 1000 x80) in each h5 file. Assume the names of the files are f1.h5, f2.h5. The utility h5ls appears to work correctly for each file, and h5topng works correctly for each file. But I find a problem with h5math. I have libmatheval installed and the command

h5math -ve 'd1 +d1' out.h5 f1.h5 f2.h5

produces a correct out.h5 file (1000 x 1000 x 80) that can be converted to png files with h5topng. BUT the subtracting expression 'd1 - d2' produces an out.h5 (1000 x 1000 x 80) filled with zeros. The datasets in f1.h5 and f2.h5 are not equal. The command that I use is,

h5math -ve 'd1 - d1' out.h5 f1.h5 f2.h5

I would appreciate some helpful comments or hints as to what might be problem.

John Weiner

Getting this error when I try to use h5topng

(mp) login04% h5topng -v <filename>              
Using colormap "gray" in file "~/miniconda3/envs/mp/share/h5utils/colormaps/".
h5topng error: invalid colormap file

I just cloned the repo and replaced the old files in the directory with new ones but that didn't fix it.

Hello,

I am trying to generate a .vts file from .h5 file using h5tovts command. My .h5 file contains CFD data.

command = h5tovts filename.h5

The command executes and shows me what is inside the hdf5 file but at the end it is not generating the .vts file.
I am getting this message at the end:

raise Exception('Data of dimension {}. Only 3D data supported at the moment.'.format(len(data.shape))) Exception: Data of dimension 2. Only 3D data supported at the moment.

Any help would be appreciated.

The h5utils is very useful for converting hdf5 file to v5d format to make 5d visualization.

The error was raised when I use the command as follows to convert hdf5 file to v5d:

h5tov5d -o radar.v5d -x 400 -y 400 -v -d reflectivity radar.hdf5

Error Information

h5tov5d error: invalid slice of HDF5 data

Test data link

http://tmp.link/f/5e8b4187b6a81

Thanks for your help.

I have a test.h5 file which contains multiple datasets. Each dataset has unique name such as(cod90_componenty_Aodes_odesAll_85,cod10_componentz_Aodes_odesAll_85 etc) and each dataset is under different groups and subgroups.

I have installed Cygwin and I am trying to do this conversion. But I am getting the following error.
Do we have to create the datasets in any particular format so that it gets picked up when I run 'h5tovtk test.h5' command?

Error message:
h5tovtk error: couldn't find data set in HDF5 file

I installed meep on this configuration...
Linux nano02 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 GNU/Linux
I installed it with "sudo apt-get install meep", done "sudo apt-get update" and "dist-upgrade" first.
meep runs ok, but when I run h5topng, it produces the (seemingly) corrent png file, but gives error
/usr/bin/h5topng': double free or corruption (out): 0x00007fff6384c370
I tried the other format conversions, to text and vtk, there is no error message there.
Formerly (five year before) I already used meep, this error did not occur back then.
What can I do?
Thank you, Geza

h5utils configure script is:
[saranya@localhost h5utils-1.13.1]$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sin in -lm... yes
checking for snprintf... yes
checking for inflate in -lz... yes
checking for png_create_write_struct in -lpng... yes
checking for evaluator_get_variables in -lmatheval... no
configure: WARNING: can't find libmatheval: won't be able to compile h5math
checking for h4toh5... no
checking for h5toh4... no
checking for jpeg_start_compress in -ljpeg... yes
checking for DFSDgetdata in -ldf... no
configure: WARNING: can't find libdf (HDF4): won't be able to compile h5fromh4 or h4fromh5
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking hdf.h usability... no
checking hdf.h presence... no
checking for hdf.h... no
checking hdf/hdf.h usability... no
checking hdf/hdf.h presence... no
checking for hdf/hdf.h... no
checking for H5Fopen in -lhdf5... no
configure: error: hdf5 libraries are required for compilation
[saranya@localhost h5utils-1.13.1]$
I already installed HDF5 and set the LDFLAGS and CPPFLAGS but it is showing an error:
[saranya@localhost h5utils-1.13.1]$ export LDFLAGS= "-L/home/saranya/Downloads/hdf5-1.8.20/hdf5/lib"
bash: export: -L/home/saranya/Downloads/hdf5-1.8.20/hdf5/lib': not a valid identifier [saranya@localhost h5utils-1.13.1]$ export CPPFLAGS= "-I/home/saranya/Downloads/hdf5-1.8.20/hdf5/include" bash: export: -I/home/saranya/Downloads/hdf5-1.8.20/hdf5/include': not a valid identifier
[saranya@localhost h5utils-1.13.1]$

Is there any error in setting the ldflags and cppflags?

Bug reported by @JohnWeiner at stevengj/meep#70:

After having installed meep-1.3 on macOS Sierra (see issue stevengj/meep#64), I find that, when invoking the h5topng command with the appropriate options (-S3) operating on the file, test1-eps-000000.00.h5, generated by the first tutorial meep program (straight wave guide), I get a memory error,

h5topng -S3 test1-eps-000000.00.h5
h5topng(28360,0x7fffe584e3c0) malloc: *** mach_vm_map(size=18446744049097375744) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
h5topng error: out of memory

subsequent invocations of h5topng simply results in a segmentation fault. The h5utils package was downloaded from the meep MIT Abinitio site and the appropriate patch, hdf5_h5utils_libpng_patch, was applied with the patch command. The "configure", "make", "sudo make install" without incident.

I have no reason to believe the h5 file is defective although I don't know how to verify this independently. The meep test1.ctl >& test1.out command runs, produces the h5 file, and no errors.

Does anybody know how to diagnose the error allocation problem and fix it?

I find myself rather often trying to convert different sorts of hdf5 files to vtk with h5tovtk.
Despite having access to High-Performance-Computing resources, when converting large files (from about 100 GB up) it is rather frustrating to get the same hopeless response: arrayh5 error: out of memory.
A distributed (MPI?) version of the tool (for multin-node system) or at least a buffered one, capable of copying data in chunks, would be very welcome!
Any hope of seeing something like this in the near future?
Or is this already possible and I'm doing something wrong?
Thank you!

I'm trying to package h5utils for Nix (NixOS/nixpkgs#37092), and would prefer to use the tarball method to install, but the current release does not yet include the fixes for the optional manpages (ae01ea6), so I get build failures when any of those are enabled. Could a new release be made that includes these fixes?

Instead of specifying a single index as an argument to these options, you can also specify a range of indices in a Matlab-like notation: start:step:end or start:end (step defaults to 1). This loops over that slice index, from start to end in steps of step, producing a sequence of output PNG files (with the slice index appended to the filename, before the .png).

Matlab (and python) support shorthand for all indices along a dimension (mainly the :: style syntax) as well as "end" shorthand (end in Matlab and -1 in python).

Supporting this would be helpful, e.g. when scripting h5topng with MEEP, and simulations will run until different times. If this is already implemented, it is not obvious how to use this.

After having installed guile-2.0.11 in /usr/local on my ubuntu box (necessary for meep-1.3 compatibility), I installed all the prerequisite programs and meep-1.3. The guile meep works and generates the expected h5 files from the first tutorial programs (straight waveguide and bent waveguide).

I installed h5utils from the git clone, and using sh autogen to generate the Makefile. Make and "sudo make install" proceed without error. When applyng h5topng -S3 to my output h5 file, tutor1-eps-000000.00.h5, however, I get a message,

"libpng warning: Application built with libpng-1.4.20 but running 1.6.28".

The warning is serious because the supposed png file generated (e.g. tutor1-eps-000000.00.png) is not recognized by any png image reader.

The "Application" in the error message is, I presume, h5utils; but I don't understand why I get the "running 1.6.28", which is a later libpng version, because I explicitly installed libpng-1.4.20 (by downloading the tarball from the sourceforge site) and doing the usual configure, make, make install procedure. The 1.4.20 libraries are really in /usr/local/lib but /usr/local/bin/h5topng seems to be getting its library version from somewhere else. I haven't been able to track down these other 1.6.28 libraries, but I did see that /usr/bin/h5topng exists and is probably a relic from an earlier attempt to install meep from a debian package. I made sure that the $PATH has /usr/local/bin before /usr/bin so it should find the /usr/local/h5topng first. In fact when I explicitly invoke /usr/bin/h5topng, I get a seg. fault.

What needs to be done to get compatibility between h5utils and libpng versions?

Update 1

I found the 1.6.28 libraries. They are in

/usr/lib/x86_64-linux-gnu/libpng16.so.28.0
etc.

For some reason h5topng is going to these libraries at run time.

Update 2
The assumption that, for some reason, it was necessary to install libpng-1.4.20 to be compatible with h5utils, installed in /usr/local, is wrong. Uninstalling libpng-1.4.20 from /usr/local/lib and installing h5utils from the git clone and sh autogen.sh, followed by make and sudo make install is all that is needed. Now with h5utils in /usr/local and the executable commands in /usr/local/bin, and with the PATH variable set so that /usr/local/bin is encountered before /usr/bin, h5topng works as advertised. The whole thing was a wild goose chase. I had version incompatibility on the brain because of the problems with guile versions.