Code Monkey home page Code Monkey logo

kge_critical_evaluation's Introduction

Technical note: When best is the enemy of good – critical evaluation of performance criteria in hydrological models

Author ORCIDs

Description

This repository contains the following elements:

  • /script/utils.R Custom functions for KGENP, KGE’’, LME, LCE and NSE
  • /script/run_eval.R Function to calculate performance criteria values:
    • KGE and KGE’: hydroGOF R package (Zambrano-Bigiarini, 2020)
    • KGENP: supplementary material from Pool et al. (2018)
    • DE’: diag-eff Python package (Schwemmle et al., 2021)
    • dr: HydroErr Python package (Roberts et al., 2018)
    • KGE’’, LME, LCE and NSE: custom code
    • KGE_sf: custom code for using KGE with scaling factors
  • /script/cblind_bw_palette.R Color palette adapted for color vision deficiencies and black/white printing
  • /script/Synthetic time series Synthetic time series code
    • syn_flood_event.R Synthetic time series of a flood event (syn1) or two consecutive flood events (syn2)
    • compare_score_param.R R code to generate Figure 3
    • plot_model_distribution.R R code to generate Figure 4
    • plot_omega_vs_omega2.R R code to generate Figure 5
  • /script/ANN model ANN model code
    • Python script to run the ANN model
    • Dummy data file to illustrate the structure of input data
  • /script/Reservoir model Reservoir model code
    • KarstMod .properties file
    • R script to perform the snow routine

Prerequisites

# R packages
library(reticulate)
library(hydroGOF) # for KGE and KGE'
library(dplyr)
library(tidyr)
library(ggplot2)
library(cowplot)

# Python packages
de <- reticulate::import("de") # diaf-eff Python package
hydroErr <- reticulate::import("HydroErr") # HydroEff Python package

# Functions
source("script/utils.R")
source("script/run_eval.R")
source("script/cblind_bw_palette.R")

Synthetic time series

General

# Import synthetic time series of flood events
## syn1: one event
## syn2: two consecutive events
source("script/Synthetic time series/syn_flood_event.R")

# Define omega bounds
omega_log_min <- -0.36
omega_log_max <- 0.36

Figure 1

# Calculate omega bounds (min/max)
omega_min <- 10^(omega_log_min)
omega_max <- 10^(omega_log_max)

# Create min and max transformed time series
model_min <- c(syn2$discharge * omega_min)
model_max <- c(syn2$discharge * omega_max)

# Generate plot
syn2 |> 
  mutate(model_min, model_max) |> 
  ggplot(aes(t, discharge)) +
  geom_ribbon(aes(ymin = model_min, 
                  ymax = model_max, 
                  fill = "Set of transformed time series")) +
  geom_line(aes(color = "Reference time series")) +
  scale_color_manual(name = "", values = "black") +
  scale_fill_manual(name = "", values = "lightgrey") +
  xlab("Time [T]") +
  ylab(expression(paste("Discharge [L"^3~T^-1, "]"))) +
  coord_cartesian(xlim = c(0, 40)) +
  scale_y_continuous(expand = expansion(mult = c(0, 0.1))) +
  theme_bw() +
  theme(legend.position = "bottom",
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.title = element_text(size = 12),
        axis.text = element_text(size = 10),
        strip.text = element_text(size = 10),
        legend.text = element_text(size = 10))

Figure 2

# Define omega values
omega_1 <- 0.75
omega_2 <- 1.2

# Create two transformed time series
model_bad_bad <- c(syn1$discharge * omega_1, syn1$discharge * omega_2)
model_bad_good <- c(syn1$discharge * omega_1, syn1$discharge)

# Generate plot
syn2 |>
  mutate(model_bad_bad, model_bad_good) |> 
  pivot_longer(c("model_bad_bad", "model_bad_good")) |> 
  ggplot() +
  geom_point(aes(x = t, y = discharge, shape = "Reference time series")) +
  geom_line(aes(x = t, y = value, color = "Transformed time series")) +
  facet_wrap(~name, 
             labeller = as_labeller(c("model_bad_bad" = '"Bad-Bad" model (BB model)', 
                                      "model_bad_good" = '"Bad-Good" model (BG model)'))) +
  coord_cartesian(xlim = c(0, 40)) +
  scale_y_continuous(expand = expansion(mult = c(0, 0.1))) +
  scale_shape_manual(name = "", values = 4) +
  scale_color_manual(name = "", values = "#696969") +
  xlab("Time [T]") +
  ylab(expression(paste("Discharge [L"^3~T^-1, "]"))) +
  theme_bw() +
  theme(legend.position = "bottom",
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.title = element_text(size = 12),
        axis.text = element_text(size = 10),
        strip.text = element_text(size = 10),
        legend.text = element_text(size = 10))

Figure 3

source("script/Synthetic time series/compare_score_param.R")

# Define omega values
omega_1 <- 0.75
omega_2 <- 1.2

# Create two transformed time series
model_bad_bad <- c(syn1$discharge * omega_1, syn1$discharge * omega_2)
model_bad_good <- c(syn1$discharge * omega_1, syn1$discharge)

# Generate plot
compare_score_param(syn2$discharge, model_bad_bad, model_bad_good)

Figure 4

source("script/Synthetic time series/plot_model_distribution.R")

# Vector of criteria to study
criteria <- c("kge", "kge_m", "kge_m2", "kgenp", "de", "lme", "nse", "dr", "lce")

# Generate plot
plot_model_distribution(data = syn1,
                        perf_c = criteria,
                        ncol = 3,
                        omega_log_min = omega_log_min,
                        omega_log_max = omega_log_max,
                        step = 0.002)

Figure 5

source("script/Synthetic time series/plot_omega_vs_omega2.R")

# Vector of criteria to study
criteria <- c("kge", "kge_m", "kge_m2", "kgenp", "de", "lme", "nse", "dr", "lce")

# Generate plot
plot_omega_vs_omega2(data = syn1,
                     perf_c = criteria,
                     omega_log_min = omega_log_min,
                     omega_log_max = omega_log_max,
                     step = 0.002)

Hydrological models

KarstMod

Information on the KarstMod platform can be found in Mazzilli et al. (2019b) and the user manual (Mazzilli and Bertin, 2019a). The main workflow is:

  1. Prepare the input data according to the KarstMod format
  2. [Optional] Open a KarstMod .properties file
  3. Import the input data
  4. Define warm-up/calibration/validation periods
  5. Define a path for the output directory
  6. Run calibration

It is possible to modify the model parameters, the objective function, the number of iterations, the maximum time, and other options. The Save button allows to save the modifications into a new .properties file.

ANNs

Information on 1D-Convolutional Neural Networks can be found in ADD DOI of ANN-lumped study.

Dependencies:

References

Chen, Z., Hartmann, A., Wagener, T., and Goldscheider, N.: Dynamics of water fluxes and storages in an Alpine karst catchment under current and potential future climate conditions, Hydrol. Earth Syst. Sci., 22, 3807–3823, https://doi.org/10.5194/hess-22-3807-2018, 2018.

Hock, R.: A distributed temperature-index ice- and snowmelt model including potential direct solar radiation, J. Glaciol., 45, 101–111, https://doi.org/10.3189/S0022143000003087, 1999.

Mazzilli, N. and Bertin, D.: KarstMod User Guide - version 2.2, hal-01832693, 103927, 2019a.

Mazzilli, N., Guinot, V., Jourde, H., Lecoq, N., Labat, D., Arfib, B., Baudement, C., Danquigny, C., Soglio, L. D., and Bertin, D.: KarstMod: A modelling platform for rainfall - discharge analysis and modelling dedicated to karst systems, Environ. Model. Softw., 122, 103927, https://doi.org/10.1016/j.envsoft.2017.03.015, 2019b.

Pianosi, F., Sarrazin, F., and Wagener, T.: A Matlab toolbox for Global Sensitivity Analysis, Environmental Modelling & Software, 70, 80–85, https://doi.org/10.1016/j.envsoft.2015.04.009, 2015.

Pool, S., Vis, M., and Seibert, J.: Evaluating model performance: towards a non-parametric variant of the Kling-Gupta efficiency, Hydrol. Sci. J., 63, 1941–1953, https://doi.org/10.1080/02626667.2018.1552002, 2018.

Roberts, W., Williams, G., Jackson, E., Nelson, E., Ames, D.: Hydrostats: A Python Package for Characterizing Errors between Observed and Predicted Time Series. Hydrology 5(4) 66, doi:10.3390/hydrology5040066, 2018.

Schwemmle, R., Demand, D., and Weiler, M.: Technical note: Diagnostic efficiency – specific evaluation of model performance, Hydrol. Earth Syst. Sci., 25, 2187–2198, https://doi.org/10.5194/hess-25-2187-2021, 2021.

Zambrano-Bigiarini M.: hydroGOF: Goodness-of-fit functions for comparison of simulated and observed hydrological time series. R package version 0.4-0. https://github.com/hzambran/hydroGOF. DOI:10.5281/zenodo.839854, 2020.

kge_critical_evaluation's People

Contributors

busemorose avatar andreaswunsch avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.