stalbrec / dwdgribextractor Goto Github PK

dwdGribExtractor is a smart package to easely extract data from numerical weather prediction grib files provided by DWD. The difference to other packages is the location flexibility. So every point on the 2.2km x 2.2km grid (ICON-D2) can be retrieved for the next X forecast hours.

At the moment only ICON_D2 is supported.

For currently available weather variabels see: ICON User Manual Chapter 6.1.4 table 6.4

Install with pip:

$ pip install dwdGrilbExtractor

To read the grib2 files xarray with cfgrib engine is used. The easiest way to install cfgrib and all its binary dependencies is via Conda:

$ conda install -c conda-forge cfgrib

alternatively, if you install the binary dependencies yourself, you can install the Python package from PyPI with:

$ pip install cfgrib

cfgrib depends on the eccodes python package to access the ECMWF ecCodes binary library, when not using conda please follow the System dependencies section there.

It is strongly recommended to use Unix enironment running dwdGrib2location. For windows several issues may appear. At the moment dwdGrib2location is tested with Windows10 and works if following requirements are satiesfied. To build eccodes on windows by your own should be avoided. The easiest way to use eccodes on windows is to install it in an MSYS environment.

1. Install MSYS

2. Install eccodes. Depending on your system run in the MSYS cli:

   $ pacman -S mingw-w64-ucrt-x86_64-eccodes
   

3. Set environment variables:

   ECCODES_DIR = <path_to_ecccodes_install_dir> e.g (C:\msys64\ucrt64)
   ECCODES_DEFINITION_PATH = <path_to_eccodes_definitions> (e.g C:\msys64\ucrt64\share\eccodes\definitions)
   

4. Add eccodes to path. This is the folder inside the MSYS environment where the .exe files are located:

   e.g. C:\msys64\ucrt64\bin
   

5. Install ecCodes:

   $ pip install eccodes
   

6. Install cfgrib:

   $ pip install cfgrib
   

7. Check if cfgrib is working:

   $ python -m cfgrib selfcheck
   Found: ecCodes v2.20.0.
   Your system is ready.
   

1. Install eccodes with apt or build it by your own TODO

locationList = {
    "Vienna": {
        "lat": 48.20,
        "lon": 16.37
    },
    "Graz": {
        "lat": 47.07,
        "lon": 15.43
    }
}

variables = ["aswdir_s", "aswdifd_s", "t_2m"]

forecast = ICON_D2(locations = locationList, forecastHours = 3)
data = forecast.collectData(varList = variables, cores = None) # Disable multiprocessing
#data = forecast.collectData(varList = variables, cores = 4)


#### Indexing one location
loc = "Graz"
result = data.loc[loc]
result = data.loc[loc, "2021-09-06 06:15:00"]

#### Indexing multiple locations
loc = ["Graz", "Vienna"]
result = data.loc[loc]

#### Indexing one location with datetime condition
loc = "Graz"
mask = data.loc[loc].index.get_level_values(0) > np.datetime64('2021-09-06T06:15:00')
mask = data.loc[loc].index.get_level_values(0).hour == 8
result = data.loc[loc][mask]

#### Indexing multiple locations with datetime condition
loc = ["Graz", "Vienna"]
mask = data.loc[loc].index.get_level_values(1) > np.datetime64('2021-09-06 06:15:00')
mask = data.loc[loc].index.get_level_values(1).hour == 8
result = data.loc[loc][mask]

• Multiprocessing on some windows machines may not work. Disable it by setting forecast.collectData(varList = variables, cores = None)
• Spyder IDE does not produce print outputs if multiprocessing is enabled.
• Dont run the code in Spyder with F5 or debug mode. This calls runfile() and sometimes crashes memory.

Manuel Strohmaier

Licensed under the MIT license. See LICENSE for details.

The DWD has information about their terms of use policy in German and English.