storm-fsv-cvut / smoderp2d Goto Github PK

View Code? Open in Web Editor NEW

9.0 4.0 4.0 56.63 MB

SMODERP2D - Distributed event-based model for surface and subsurface runoff and erosion

Home Page: https://storm-fsv-cvut.github.io/smoderp2d-manual/

License: GNU General Public License v3.0

Python 94.25% Makefile 0.79% HTML 0.22% Shell 1.38% Batchfile 1.10% QML 0.26% R 1.79% Dockerfile 0.21%

surface-runoff soil-erosion gis-based

smoderp2d's Introduction

SMODERP2D

Distributed event-based model for surface and subsurface runoff and erosion.

    @ @ @   @       @     @ @     @ @ @     @ @ @ @  @ @ @    @ @ @
   @        @ @   @ @   @     @   @     @   @        @     @  @     @
   @        @   @   @  @       @  @      @  @        @     @  @     @
     @ @    @       @  @       @  @      @  @ @ @    @ @ @    @ @ @
         @  @       @  @       @  @      @  @        @   @    @
         @  @       @   @     @   @     @   @        @    @   @
    @ @ @   @       @     @ @     @ @ @     @ @ @ @  @     @  @
    \  \  /   / /    \   \  /   \  /    /     /       @ @ @   @ @ @
     \ _\/   /_/      \   \/     \/    /_____/       @     @  @     @
         \__/          \  /      _\___/                    @  @      @
             \____      \/      /                         @   @      @
                  \_____/______/                        @     @      @
                               \                      @       @     @
                                \___________________ @ @ @ @  @ @ @

User documentation: https://storm-fsv-cvut.github.io/smoderp2d-manual/

How to test

Download SMODERP2D source code to your computer.

git clone https://github.com/storm-fsv-cvut/smoderp2d.git

From Docker container

Build an image:

docker build \
 --tag smoderp2d:latest --file docker/Dockerfile .

Run SMODERP2D command line tool from Docker container:

docker run \
 -v `pwd`:/opt/smoderp2d -w /opt/smoderp2d/ --rm --entrypoint \
 ./bin/start-smoderp2d.py smoderp2d \
 --config tests/config_files/quicktest_stream_rill.ini

GRASS GIS

Build an image with GRASS GIS support:

docker build \
 --tag smoderp2d-grass:latest --file docker/grass/Dockerfile .

Run SMODERP2D with GRASS GIS provider from Docker container:

docker run \
 -v `pwd`:/opt/smoderp2d -w /opt/smoderp2d/ --rm --entrypoint \
 ./tests/run_grass_gistest.sh smoderp2d-grass \
 nucice

Run locally

Command line

PYTHONPATH=$PYTHONPATH:`pwd` ./bin/start-smoderp2d.py --config tests/config_files/quicktest_stream_rill.ini

QGIS

Requirements: QGIS 3.28.10 and higher

Define QGIS_PLUGINPATH and PYTHONPATH environmental variables in Settings -> Options -> System and restart QGIS.

Than enable SMODERP2D plugin in Plugins -> Manage and Install Plugins....

Alternatively set up environment variables in command line before starting QGIS:

PYTHONPATH=$PYTHONPATH:`pwd` QGIS_PLUGINPATH=`pwd`/bin/qgis qgis tests/data/nucice/qgis_project.qgz

smoderp2d's People

Contributors

Stargazers

Watchers

Forkers

jerabekjak gaowudao kubsta

smoderp2d's Issues

Compute terrain from clip data

See a76cc72

[CLOSED] define providers

Issue by landam
Thursday Apr 05, 2018 at 07:01 GMT
Originally opened as jerabekjak/smoderp#16

Define providers:

Cmd (arcparse)
ArcGIS (arcpy)
GRASS GIS (pygrass)

arcgis provider testing data error: copyraster_management error (see comment)

two version of arcmap gives an error while running the testing data with the arcgis toolbox.
error message in arcmap 10.6

error message in arcmap 10.5

[CLOSED] nechapu_asi smazat

Issue by kavkapet
Tuesday Jun 13, 2017 at 13:27 GMT
Originally opened as jerabekjak/smoderp#6

None

kavkapet included the following code: https://github.com/jerabekjak/smoderp/pull/6/commits

raise error if .ini file does not exist

if assign --indata file in cmd execution does not exist. Incorrect error is raised

Error raised:
ConfigParser.NoSectionError: No section: 'Other'

should be something line
IncorrectInitFile: file 'somefile ' is does not exist

[CLOSED] test branch

Issue by jerabekjak
Friday Mar 24, 2017 at 17:08 GMT
Originally opened as jerabekjak/smoderp#2

jerabekjak included the following code: https://github.com/jerabekjak/smoderp/pull/2/commits

[CLOSED] Picklepryc

Issue by jerabekjak
Saturday Apr 01, 2017 at 17:02 GMT
Originally opened as jerabekjak/smoderp#5

jerabekjak included the following code: https://github.com/jerabekjak/smoderp/pull/5/commits

[CLOSED] consider pep8

Issue by landam
Tuesday Apr 03, 2018 at 18:42 GMT
Originally opened as jerabekjak/smoderp#14

Please consider following official Style Guide for Python Code, PEP 8, see https://www.python.org/dev/peps/pep-0008/

Richards equation

Issue by jerabekjak
Wednesday Dec 05, 2018 at 15:08 GMT
Originally opened as jerabekjak/smoderp#27

implement richards equation of the infiltration

unable to process txt input in ArcGIS

Input file: 028_m+STD_IC2_nL_Re_IL_SL

Computing intersect of input data...
(u' ', u' ', u' ', 0, 0, 0, u' ', u' ', u' ', u' ', u' ')
Invalid input for data preparation: Values in soilveg tab are not correct (field 'k': empty value found in row 1)

See: https://github.com/storm-fsv-cvut/smoderp2d/blob/master/smoderp2d/providers/arcgis/data_preparation.py#L217

[CLOSED] Test table pre-calculation of the power law values.

Issue by jerabekjak
Monday Nov 26, 2018 at 13:51 GMT
Originally opened as jerabekjak/smoderp#25

[CLOSED] Picklepryc

Issue by jerabekjak
Saturday Apr 01, 2017 at 16:53 GMT
Originally opened as jerabekjak/smoderp#3

jerabekjak included the following code: https://github.com/jerabekjak/smoderp/pull/3/commits

Print logger output also to a text file [*.log] in output directory

[CLOSED] prepare sample dataset

Issue by landam
Thursday Apr 26, 2018 at 08:16 GMT
Originally opened as jerabekjak/smoderp#20

small dataset
larger dataset

API documentation

Issue by landam
Thursday Apr 05, 2018 at 06:54 GMT
Originally opened as jerabekjak/smoderp#15

Sphinx, see eg. http://pywps.readthedocs.io/en/master/api.html#pywps.inout.formats.FORMATS
~~Doxygen (rather for C/C++, not ideal for Python code)~~
... ?

add computation from pickle option into QGIS plugin

add tick box (Run computation from pickle) to Settings, when ticked, option to add pickle file appears (otherwise it's hidden)

warning: check input streams

Stream network routing rule: next edge should be only one.

[CLOSED] study & rewrite original arcpy script

Issue by landam
Thursday Apr 05, 2018 at 07:28 GMT
Originally opened as jerabekjak/smoderp#17

See https://github.com/jerabekjak/smoderp/blob/master/main_src/data_preparation.py

hydrographs are not created with arcgis:data_preparation (see comment)

Globals.points is equal to None in smoderp2d/runoff.py [1] even though points (for hydrographs) are set in toolbox wizard in arc map.
[1]

smoderp2d/smoderp2d/runoff.py

Line 160 in 6485f1d

if Globals.points and Globals.points != "#":

change 3-d object surface.array to multiple 2-d matrices

Currently, we are using a 3-d object surface.array and subsurface.array for position-related variables in branch tf_smoderp. This is a reminiscent from loop-based computations and it is very unreadable. It would be better to rewrite it into x different 2-d matrices, where x is the length of the current third dimension.

Expected positives:

Easier to understand with which variable are we working
It is better to work with smaller arrays than with huge 28-times bigger multi-arrays

fix test data: remove overlapping polygons

soil.shp contains overlapping polygons. see #38

data providers: compare results created by ArcGIS and GRASS GIS

Data preparation

Raster (required)

Vector (required)

soil_b (more areas in grass)
vegetation_b (more areas in grass)

Raster (optional)

Vector (optional)

inter_soil_lu
dem_mask
points_inter (clipped input points)

Stream preparation

Raster (required)

stream_seg

Vector (required)

Raster (optional)

Vector (optional)

sync variable names: underscore vs. CamelCase

Issue by landam
Tuesday Apr 17, 2018 at 12:35 GMT
Originally opened as jerabekjak/smoderp#19

[CLOSED] support python3

Issue by landam
Wednesday Mar 14, 2018 at 11:59 GMT
Originally opened as jerabekjak/smoderp#11

parallelization

Issue by landam
Wednesday Mar 14, 2018 at 11:56 GMT
Originally opened as jerabekjak/smoderp#9

Design computation in parallel:

Options:

one matrix + tiling class
each process with operate with own matrix (problem with overlaps)

Tasks:

numpy parallelization (multiprocess, ...)
gpu - http://stsievert.com/blog/2016/07/01/numpy-gpu/

Sequential sheet and rill flow calculation

Issue by jerabekjak
Wednesday Nov 28, 2018 at 10:35 GMT
Originally opened as jerabekjak/smoderp#26

qgis plugin: initial phase (import input data)

QGIS plugin must create temporal GRASS location and import input data before computation.

Inspiration:

QGIS processing (https://github.com/qgis/QGIS/tree/master/python/plugins/processing/algs/grass7)
Soils plugin: https://github.com/ctu-geoforall-lab/qgis-soil-erosion-plugin/tree/master/pyerosion

Or alternatively:

import grass.script as gs
import sys
print (sys.argv[1])
# gs.run_command('v.import', input=sys.argv[1], output='xxx')
print (gs.gisenv())

grass -c EPSG:5514 /tmp/test2 --exec python test.py vstup1

consider numpy masked arrays

Issue by landam
Wednesday Mar 14, 2018 at 11:48 GMT
Originally opened as jerabekjak/smoderp#7

See https://docs.scipy.org/doc/numpy-1.13.0/reference/maskedarray.generic.html#using-numpy-ma

Could be used for:

region mask (currently nested lists)
... ?

test smoderp on ArcGIS Pro

arcgis: logging works only on first run

Issue by landam
Wednesday Oct 10, 2018 at 13:02 GMT
Originally opened as jerabekjak/smoderp#22

ArcGIS provider has broken logging:

first run: works, messages are printed out
other runs: no messages printed

grass providers fails on each second run

  File "./bin/grass/r.smoderp2d/../../../smoderp2d/providers/base/data_preparation.py", line 321, in _get_array_points_
    self.data['mat_dem'][r][c] != self.data['NoDataValue'] and \
IndexError: index 43 is out of bounds for axis 0 with size 39

check input vector soil layer for overlapping features

Test data (soil.shp) contains overlapping polygons. It should be considered as bug in input data.

rewrite test

Issue by landam
Wednesday Mar 14, 2018 at 12:12 GMT
Originally opened as jerabekjak/smoderp#12

See tests directory.

design datapreparation provider for GRASS GIS

Issue by landam
Thursday Apr 26, 2018 at 08:21 GMT
Originally opened as jerabekjak/smoderp#21

Sort output files in correct directories

Create 3 directories:

core
control
temp

Sort output files into corresponding directory.

qgis plugin prototype

Basic prototype

Data preparation
Interface for smoderp lib (GRASS provider will be probably used, #21 )
Display output data

Tasks

@kavkapet review tabs (how to split parameters)
@pesekon2 remove pickle file option
@pesekon2 new module to define labels (which will be used by QGIS, GRASS module and ArcGIS - see bin/arcgis/SMODERP2D.pyt -> getParameterInfo)

key - `inputSurfaceRaster`
label - _("Input surface raster")
label_cs - ?
section - None | "Settings"
data_type: raster | vector | column | int | float
tooltip: read from ArcGIS file

@kavkapet design soil tables (predefined tables)
@kavkapet which output data display (+ symbology)

Question: how to deal with localization?

Improvements

Graphs
... ?

[CLOSED] Picklepryc

Issue by jerabekjak
Saturday Apr 01, 2017 at 16:55 GMT
Originally opened as jerabekjak/smoderp#4

jerabekjak included the following code: https://github.com/jerabekjak/smoderp/pull/4/commits

review output arrays values

ArcGIS Toolbox run with test data gives strange output data

While test from command line

./bin/start-smoderp2d.py --typecomp roff --indata tests/test.ini

gives different values (some make sense from first POV)

2019-10-05 16:17:20,232 - SMODERP2D - INFO - Array stats: min=0.0 max=0.00284423793801 mean=0.00138464882653 - [__init__:379]
2019-10-05 16:17:20,232 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/cInfil_m3.asc saved - [__init__:99]
2019-10-05 16:17:20,233 - SMODERP2D - INFO - Array stats: min=0.0 max=0.0143333333333 mean=0.00776388888889 - [__init__:379]
2019-10-05 16:17:20,233 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/cRain_m3.asc saved - [__init__:99]
2019-10-05 16:17:20,233 - SMODERP2D - INFO - Array stats: min=0.0 max=0.00534586330829 mean=0.00145814946429 - [__init__:379]
2019-10-05 16:17:20,233 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mVel_m_s.asc saved - [__init__:99]
2019-10-05 16:17:20,234 - SMODERP2D - INFO - Array stats: min=0.0 max=0.0518620497914 mean=0.015633564699 - [__init__:379]
2019-10-05 16:17:20,234 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mrSearStr_Pa.asc saved - [__init__:99]
2019-10-05 16:17:20,234 - SMODERP2D - INFO - Array stats: min=0.0 max=9.33323751905e-05 mean=2.04181664011e-05 - [__init__:379]
2019-10-05 16:17:20,234 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mQsur_m3_s.asc saved - [__init__:99]
2019-10-05 16:17:20,235 - SMODERP2D - INFO - Array stats: min=0.0 max=0.148485353763 mean=0.0326753520574 - [__init__:379]
2019-10-05 16:17:20,235 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/cVsur_m3.asc saved - [__init__:99]
2019-10-05 16:17:20,235 - SMODERP2D - INFO - Array stats: min=0.0 max=1.39237258423e-05 mean=5.44579206604e-06 - [__init__:379]
2019-10-05 16:17:20,235 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mQsheet_m3_s.asc saved - [__init__:99]
2019-10-05 16:17:20,236 - SMODERP2D - INFO - Array stats: min=0.0 max=0.00273906893225 mean=0.000583630068854 - [__init__:379]
2019-10-05 16:17:20,236 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mWLevelRill_m.asc saved - [__init__:99]
2019-10-05 16:17:20,237 - SMODERP2D - INFO - Array stats: min=0.0 max=7.94086531127e-05 mean=1.497236428e-05 - [__init__:379]
2019-10-05 16:17:20,237 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/mQRill_m3_s.asc saved - [__init__:99]
2019-10-05 16:17:20,238 - SMODERP2D - INFO - Array stats: min=0.0 max=0.0442320876233 mean=0.0118387686368 - [__init__:379]
2019-10-05 16:17:20,238 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/widthRill.asc saved - [__init__:99]
2019-10-05 16:17:20,238 - SMODERP2D - INFO - Array stats: min=0.0 max=0.137246258368 mean=0.0280351847523 - [__init__:379]
2019-10-05 16:17:20,238 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/cVIn_M3.asc saved - [__init__:99]
2019-10-05 16:17:20,239 - SMODERP2D - INFO - Array stats: min=0.0 max=0.00025 mean=0.000135416666667 - [__init__:379]
2019-10-05 16:17:20,239 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/surRet_M.asc saved - [__init__:99]
2019-10-05 16:17:20,239 - SMODERP2D - INFO - Array stats: min=0.0 max=0.0 mean=0.0 - [__init__:379]
2019-10-05 16:17:20,239 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/CumVRestL3.asc saved - [__init__:99]
2019-10-05 16:17:20,240 - SMODERP2D - INFO - Array stats: min=0.011662668472 max=3.40282346639e+38 mean=2.02042643317e+38 - [__init__:379]
2019-10-05 16:17:20,240 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/massBalance.asc saved - [__init__:99]
2019-10-05 16:17:20,241 - SMODERP2D - INFO - Array stats: min=-9999 max=1002 mean=-4446.55208333 - [__init__:379]
2019-10-05 16:17:20,241 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/reachFid.asc saved - [__init__:99]
2019-10-05 16:17:20,241 - SMODERP2D - INFO - Array stats: min=-3.40282346639e+38 max=0.0 mean=-2.02042643317e+38 - [__init__:379]
2019-10-05 16:17:20,241 - SMODERP2D - INFO - Raster ASCII output file tests/data/output/volRest_m3.asc saved - [__init__:99]

@jerabekjak Please review correctness of values.

ArcToolbox (@landam)
GRASS module (@landam)
QGIS plugin (@kalator @pikorcik )

argis: documentation for data_preparation module

Issue by kalator
Wednesday Oct 10, 2018 at 13:16 GMT
Originally opened as jerabekjak/smoderp#23

Finishing up documentation in scripts:

data_preparation
stream_preparation

[CLOSED] code refactoring

Issue by landam
Wednesday Mar 14, 2018 at 11:58 GMT
Originally opened as jerabekjak/smoderp#10

Sub-tasks:

define providers (ArcGIS, GRASS GIS, ...), see #16
avoid import-run scenario
...

Green-Ampt

Issue by jerabekjak
Tuesday Mar 27, 2018 at 09:34 GMT
Originally opened as jerabekjak/smoderp#13

Implementation of Green-Ampt infiltration equation for variable rainfall input.

[test ga for unsteady rainfall]
[unify precalc function in other infiltration classes (current precipitation height was added)]

support input of .txt tables to qgis plugin

parse txt file on three dot tool button click
retrieve attribute names

arcgis: store all results in filegdb

Currently only some results are stored in FileGDB. For rest of results, Esri Shapefile format is used.

storm-fsv-cvut / smoderp2d Goto Github PK

smoderp2d's Introduction

SMODERP2D

How to test

From Docker container

GRASS GIS

Run locally

Command line

QGIS

smoderp2d's People

Contributors

Stargazers

Watchers

Forkers

smoderp2d's Issues

Data preparation

Raster (required)

Vector (required)

Raster (optional)

Vector (optional)

Stream preparation

Raster (required)

Vector (required)

Raster (optional)

Vector (optional)

Basic prototype

Tasks

Improvements

Recommend Projects

Recommend Topics

Recommend Org