Code for our paper Hydrological Concept Formation inside Long Short-Term Memory (LSTM) networks.
The results that this code produces can be found on zenodo here.
neuralhydrology: deep learning codebase copied at version for paper that includes the LSTM used in this paper
cell_state: functions used in the notebooks for training linear probes
scripts/cell_state/extract_cell_state.py
-- from a neuralhydrology trained model path, save the cell_states associated with that model for each basinscripts/cell_state/sklearn_models.py
-- fit, predict and evaluate models from sklearn ([init_linear_model
,init_nonlinear_model
])scripts/cell_state/analysis.py
--get_model_weights
from linear regression models,load_probe_components
get probe results from how they're savedscripts/cell_state/timeseries_dataset.py
--TimeSeriesDataset
object for training probesscripts/cell_state/timeseries_model.py
-- train pytorch linear model for soil moisture // snow
Marginalia:
scripts/cell_state/jdb_fitting.py
scripts/cell_state/baseline.py
-- empty :( (but look atnotebooks/07_baseline_cell_state.ipynb
for the random noise baselines)scripts/cell_state/cell_state_dataset.py
-- createCellStateDataset
for cell state and target data (for training pytorch models)scripts/cell_state/normalize.py
-- helper functions for normalizing cell statesscripts/cell_state/utils.py
-- helper functions for training a series of modelstrain_and_evaluate_models
scripts/cell_state/cell_state_model.py
--LinearModel
used bypytorch_probe.py
scripts/cell_state/pytorch_probe.py
--PytorchProbe
probe_data: the data used for training the probes (the probe targets from ERA5Land)
scripts/probe_data/get_era5_sm.py
-- Download the SM and SWE data from ERA5 Land via cds apiscripts/probe_data/era5_hourly_to_daily.py
-- Resample Data to dailyscripts/probe_data/join_daily_to_onefile.py
-- Merge all of the data into one netcdf filescripts/clip_netcdf_to_shapefile.py
-- Convert into basin timeseries (chop out regions from .shp)
extra_features: join_camels_to_era5land
- create one big netcdf file with the original camels data and the era5_land variables (swvl{1,2,3,4}
- snow water volume level {1,2,3,4} & sd
- snow depth)
geospatial: geospatial helper functions for plotting location data from camelsGB plot_spatial_location
plots: plot nice hydrographs with plot_hydrograph
read_model: load model weights from a neuralhydrology.Model with _load_weights
read_nh_results: helper functions for reading result files created by neuralhydrology
static_correlations: plotting functions for getting relationship between statics and nse scores
utils get_data_dir
// move_directory
// corr_df_ready_for_plotting
Marginalia: cwatm_data: functions for getting cwatm data into camels dataset format
scripts/cwatm_data/cwatm_to_camels_dataset.py
-- functions for building TARGET, INPUT, HIDDEN datasetsscripts/cwatm_data/ldd_basins.py
-- river drainage basins from drainage direction (LDD - local drainage direction) for cwatm usingpyflwdir
scripts/cwatm_data/masked_mean_cwatm_data.py
-- build mean data over catchment areas for TARGET, INPUT, HIDDEN
dataset_to_camelsgb_format = functions for creating CAMELSGB dataset for CWatM Data
integrated_gradients: code for calculating integrated gradients (runs slowly)
Pickle files generated using Xarray 0.20.1