geodynamics / spatialdata Goto Github PK

• Need to add --with-python-coverage=SCRIPT configure argument. Some os packages install an executable "python2-coverage" (debian-stable), but others install "coverage" and "coverage2".

• Need to pass name of test coverage executable down to run_test_coverage.sh.

Sometimes we want to use output from one PyLith simulation as input in another simulation. The current typical workflow for doing this is to use a Python script to read in data from an HDF5 output file and generate a SimpleDB spatial database.

We should implement a spatial database that provides an efficient way to query data read from the the HDF5 file, so no translation is necessary.

Notes

• We cannot rely on preservation of ordering for efficient queries, because data may be ordered via parallel I/O and a different ordering of the mesh.

• The names of the values in a spatial database do not necessarily match the output. For example, using the stress field as the initial_stress field.

Level of difficulty

Moderate. This is relatively well isolated, but does have some complications and could be difficult to develop an efficient algorithm. Matt suggests we might use the radial basis functions; alternatively we might use a K-D tree to find close points for interpolation.

Add method getDBValues() that passes back an array of names of values in the spatial database.

This is useful when queries return all values but only a few are needed in a particular query.

Extend the spatial database implementation to allow specification of time dependence.

Starting case could be x,y,z,t.

• Need to adjust interface to allow time
• PyLith - Would we need an 4D auxiliary field?

Coordinate system type specified by +proj=tmerc +datum=WGS84 +lon_0=-122.0 +lat_0=45.5 +k=0.996 +units=m is not detected in spatialdata::geocoords::CSGeo::computeSurfaceNormal().

Installation of the Proj.4 datums requires unzip. The spatialdata configure script should check for unzip.

The Python extension modules are currently installed via autotools while the pure Python code is installed via setuptools. This can lead to them being installed under different directories, e.g., lib and lib64. We have attempted to mitigate this by installed the extension modules under $pkgpythondir rather than $pkgpyexecdir.

For consistency of installation, it would probably be better to install both the pure Python code and Python extension modules via setuptools. We use SWIG, so it may not be simple to setup.

SWIG 3.0.2 (the current max supported version) is 5 years old now (the latest release is 4.0). Installing this heirloom is an awkward dependency on modern machines. I tried just editing configure.ac and got an error-free build, but haven't been able to test (cf. #8).

Include instructions for how to correct common errors in error messages for spatial database header and data I/O errors.

Create a new CoordSys object that implements a georeferenced coordinate system with a shift and rotation added on the end.

• Create CSGeoLocal C++ object that derives from CSGeo
• Add localToGeographic() and geographicToLocal() to CSGeo and CSGeoLocal
• Update Converter::_convert to use localToGeographic() and geographicToLocal()
• Add C++ unit tests for CSGeoLocal
• Add C++ unit tests for Converter using CSGeoLocal
• Add SWIG code for CSGeoLocal
• Add Pyre code for CSGeoLocal
• Add documentation for CSGeoLocal, including diagram

Description

CppUnit is not supported in many Integrated Development Environments. There are other C++ unit testing frameworks that are more popular and better supported in IDEs.

Desired Solution

Replace use of CppUnit with Catch2.

Use muParser (https://beltoforion.de/en/muparser/building.php#idUseSource) for compiling

The documentation for SpatialData is currently embedded in the PyLith documentation. We should extract it and fill in missing details.

└── docs
    ├── intro
        ├── preface
        ├── quickstart
        ├── release-notes
        ├── development-plan
    ├── user
        ├── intro
        ├── install
        ├── nondimensionalization
        ├── coordsys
        ├── spatialdb
            ├── overview
            ├── uniformdb
            ├── simpledb
            ├── simplegriddb
            ├── compositedb
            └── gravity-field
        ├── timehistorydb
        ├── troubleshooting
        ├── components
        ├── file-formats
            ├── simpledb
            ├── simplegriddb
            └── timehistorydb
    ├── developer
        ├── contributing
        └── testing
    └── glossary

Interpolation can fail when point cloud contains irregularly spaced points. Position of points relative to grid can matter, because we select the points for interpolation based on the distance from the given point.

Because we don't generate topology information for the point cloud, a good solution in this case would be to use radial basis functions. This would result in a much more robust interpolation algorithm. See issue #1.

Reported by Jeanne Sauber

std::logic_error

• std::invalid_argument
• std::domain_error
• std::out_of_range
• std::length_error

std::runtime_error

std::range_error

If SWIG is not enabled but the source comes from the repository, the error message is that the Makefile doesn't know how to generate the SWIG wrapper files.

We should adjust the Makefile so that the error message is that SWIG was not enabled when it should be.

Update for Proj 6.x API.

Proj parameters (+proj, +datum, +ellipsoid, etc) are no longer the preferred way to specify a projection/CRS. EPSG codes and WTK are the preferred ways to specify a projection and coordinate system.

Reorganize C++ unit test implementations and data.

Data goes in Test{OBJECT}_Cases files.
If only one set of test data, then put all code into Test{Object}.cc file (no header file).

Use C++11 feature std::shared_ptr for better memory management between C++ and Python.

Also use std::unique_ptr when appropriate for holding pointers that are never shared.

Python3 includes the venv module for virtual environments. We should use this to simplify installation.

Python2 EOL is fast approaching and the present codebase is python2-only. futurize is a useful tool for converting a python2 codebase to Python3.

I have some experience using futurize, but I can't currently run the test suite. My system has the default /usr/bin/python as python3. While the autoconf script finds python2 correctly, it seems to be running pytest rather than pytest2, thus tests fail to import the python2 module. I'm not sure what is intended here and haven't dug in to debug yet.

numpy SWIG interface file (numpy.i) is out of date and needs to be updated to a recent version.

Need to update dependencies on Travis-CI to use Proj 6.X. This in turn involves updating the PyLith installer. See geodynamics/pylith_installer#15.

Add option to use radial basis functions for 2-D and 3-D interpolation. This will allow interpolation from any point cloud.

Compute length scale based on RMS distance to N nearest points?
Use user-specified length scale (would not work well for highly variable spacing)?

PyLith needs queries to return all values by default when forming auxiliary fields. This allows optimization of queries when creating auxiliary fields.

Add make dist to automated testing to insure that all files in Makefile.am are included in the repository.

Ideally, we should try to build and run the tests using the result of make dist.

In unit tests and full-scale tests, it would be very helpful to be able to use a spatial database implementation that uses analytical functions. Currently, we usually create a SimpleGridDB spatial database at a high resolution. Obviously this has drawbacks as the resolution of the models increases.

This could be accessible to expert level users that create the low-level function, SWIG interface, and Python object (which would pass the function pointer down).

Transfer of information from Python to C++ should be in _init() not _configure() in Pyre components.

The spatial database for the SCEC CVM-H is for v5.3. We need to update this interface to provide a more robust implementation that is also efficient for querying in parallel. Furthermore, we do not want to hold the entire seismic velocity model in memory.

@baagaard-usgs is working on defining an HDF5-based format for seismic velocity models that would allow parallel I/O and caching. The format is a simpler version of the Etree format used in the USGS Bay Area Seismic Velocity Model 08.3.0. It leverages ideas from the R format used by Arthur Rodgers at LLNL in SW4.

Level of Difficulty

Intermediate to Expert depending on the storage format for the seismic velocity model.

Some seismic velocity models are easily converted/reformatted to a SimpleGridDB. The SimpleGridDB could include an HDF5 reader/writer to store the values as a dataset with appropriate attributes for the metadata that is currently included in the header for an ASCII SimpleGridDB.

Replace use of SWIG with pybind11.