Code Monkey home page Code Monkey logo

wxgen's Introduction

Weather generator software

https://travis-ci.org/metno/wxgen.svg?branch=master https://coveralls.io/repos/metno/wxgen/badge.svg?branch=master&service=github

wxgen is a command-line tool for generating arbitrarily long weather time-series. The generator produces gridded output for multiple variables (e.g. temperature, precipitation) and aims to have realistic covariances in space, time, and across variables.

The generator uses a database of past weather model simulations (e.g 15 day forecasts) and combines the segments stochastically. Longer trajectories are created by concatenating the shorter trajectories from the database. This is done by matching the end state of one trajectory with the beginning state of another. The matching is done using a specified metric, such as the sum of the square differences between states (with some kind of normalization strategy as each atmospheric variable has different variances).

Documentation

For information about how to use wxgen, check out the wiki page at https://github.com/metno/wxgen/wiki

Features

Example plot

Installating on Ubuntu

Prerequisites

Install the required pacakges:

sudo apt-get update
sudo apt-get install netcdf-bin libnetcdf-dev libhdf5-serial-dev
sudo apt-get install python-setuptools python-pip
sudo apt-get install python-numpy python-scipy python-matplotlib python-mpltoolkits.basemap

Installing using pip

After this, the easiest is to install the latest version of wxgen using pip:

sudo pip install wxgen

This will create the executable /usr/local/bin/wxgen. If /usr/local/bin is not in your PATH environment variable, then add it (i.e add export PATH=/usr/local/bin/:$PATH to ~/.bashrc). If you do not have sudo privileges do:

pip install wxgen --user

This will create the executable ~/.local/bin/wxgen. Add this to your PATH environment variable if necessary (i.e add export PATH=$PATH:~/.local/bin to ~/.bashrc).

If at a later time you wish to update to a newer release, do:

pip install wxgen --upgrade

(adding --user if appropriate)

Installing from source

Alternatively, to install from source, download the source code of the latest version: https://github.com/metno/wxgen/releases/. Unzip the file and navigate into the extracted folder.

Then install wxgen by executing the following inside the extracted folder:

sudo pip install -r requirements.txt
sudo python setup.py install

If you are working on the code, the -e flag ensures that you do not need to rerun pip install every time you make changes to the code:

sudo pip install -e .

Again, if you do not have sudo privileges, remove sudo and append --user to the previous pip and python commands.

Copyright and license

Copyright (C) 2017 MET Norway. Wxgen is licensed under LGPL version 3 or (at your option) any later version.

wxgen's People

Contributors

fzeiser avatar oysteinidematorget avatar runeyosh avatar tnipen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wxgen's Issues

Implement smarter memory management

Reduce the maximum footprint, i.e. don't load the entire database into file. Compute the wavelet, then dump the variables in memory. When writing the scenarios, load only what is neeeded.

Allow resampling in wxgen truth

Add an option that allows the truth in a database to be resampled, for example to create an ensemble of yearly sequences.

Database for LOM

A few weeks of EC data:

  • Air temperature (500, 700, 1000 hPa)
  • U/V winds (2m)
  • geopotential height (500, 700, 1000 hPa)
  • instantaneous values

CRPS

Comparison of two distributions, for some timescale

Sample data for precipitation downscaler

AROME db:

  • Climatology only
  • Southern Norway
  • Threshold = 0.1 mm/day
  • Need fraction of days with rain
  • Mean and STD for days with rain

EC timeseries:

  • One simulation
  • Keep on lat/lon grid

Timestamps in truth

The timestamps in truth scenarios start at 20160101, however it could be convenient to have the actual timestamps that the scenario is based on. This only works when the user specifies -n 1.

Suggestion: Move map plotting from basemap to cartopy

Hi,

We at Statkraft consider rewriting the code to use cartopy instead of basemap (module) for the following reasons:

  • It's under GPL licensing, which we want to avoid including in our code
  • We understand that the basemap module is getting phased out in favor of cartopy

The way we see it, this can be solved by either moving completely over to cartopy, or having the code check which module is installed and use either one. We will of course implement the cartopy support and make a pull request for this.

Any views or comments on this from you guys at Met? @tnipen

Implement transformations

Examples

  • Number of wet days
  • Number of cold nights

Command-line example:

wxgen verif -tr cold_night
wxgen verif -tr ">=0"

use of wavelet in spatial matching

The spatial matching could probably be done by exploiting a bit more of the wavelet properties. In particular, instead of finding the match by comparing the de-composed coarse fields (i.e. averages over large boxes) -as I firstly suggested, not the smartest thing to do but the easiest- it might be better to compare the wavelet coefficients of the coarser scales.
What we gain is a match that takes into account much more spatial details than before and this might help in the matching of two "rare" situations.

In practice, the command we are using now is (file database.py):
dec = pywt.wavedec2(data, 'haar', level=self.wavelet_levels, axes=(1, 2))[0]
man page is:
http://pywavelets.readthedocs.io/en/latest/ref/2d-dwt-and-idwt.html
search for"pywt.wavedec2":
the function returns a list:
[cAn, (cHn, cVn, cDn), ... (cH1, cV1, cD1)]
and now we are using cAn, instead I'm suggesting to use all the other elements. The quality of the match between two fields is the (scalar) value given by the summation of all the squared differences:
[cHn(field1)-cHn(field2)]^2 + [cVn(field1)-cVn(field2)]^2+[cDn(field1)-cDn(field2)]^2 + ... + [cH1(field1)-cH1(field2)]^2 + [cV1(field1)-cV1(field2)]^2+[cD1(field1)-cD1(field2)]^2
Note that: the dimension of the vectors cHn, cVn, cDn is different from the one of the vectors cH(n-1),cV(n-1),cD(n-1) because they refers to different grids; and cH1, cV1, cD1 should be just 3 scalar values.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.