ludwigboess / gadgetio.jl Goto Github PK

We need to check if we can properly re-implement thread-safe parallel IO and if it brings consistent performance improvements.

As per Anna's request we should re-implement IO for Format 1.
Ideally even with a simple function to re-write snapshots from Format 1 to Format 2.

When any mass block is present, GadgetIO will return spurious masses for dark matter due to the check_block_positions returning false.

Suspected line https://github.com/LudwigBoess/GadgetIO.jl/blame/bf51401f73965c4e31d5991a619fbc95fa2d6a97/src/read_snapshot/read_format_2.jl#L142

It would be useful to be able to directly read a Gadget snapshot into a DistributedArray.
However it needs to be checked how the indexing behaves for a Dict of DistributedArrays.

Add SnoopPrecompile functions to pre-compile some typical function calls and speed up first time a file is read.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

We should refactor reading of parttype=-1.
Currently we need to handle a lot of exceptions which makes the code hard to maintain.

A simpler approach could be to write a seperate function at the beginning of read_block that handles read-in of all particles by first checking which particles are present in the simulation and then looping over the individual particles.
At the end we can then stack the Arrays and return them as one large Array, so the user wouldn't feel the difference.

Add IO for log files to simplify analysis and plotting.

Done:

• balance.txt
• cpu.txt
• timings.txt
• timebins.txt

We should add support for Format 3 / HDF5.
This would also allow for some interoperability with Arepo.

Currently when reading all particles with read_block and parttype=-1, individual arrays for each particle type are first created and then copied into the final array. One could improve the performance by directly reading in the full block at once, as the values are saved consecutively.