Summary

With psyplot 1.1.0, the formatoption widget framework has been introduced
that makes it possible to implement a fmt specific widget into the
formatoptions widget of the GUI. However, this is still missing for most of the
formatoptions

Reason

It makes it faster and more intuitive to handle and update the plots through
the GUI. See also https://github.com/Chilipp/psy-simple/issues/1

Detailed explanation

For mapplot, here is the current state:

seems like, installing psy-maps via conda install psy-maps does not work out-of-the-box anymore, has cartopy has made several dependencies optional (see SciTools/cartopy#1670). This is why we need to add scipy as a dependency for psy-maps

The datagrid is not displayed correctly, in this specific case for the curvilinear grid of MPIOM when trying the datagrid option at psy.plot.mapplot:

maps = psy.plot.mapplot(file_lr,
                        name='area',
                        cmap='Blues',
                        maskleq=0,
                        stock_img=True,
                        lonlatbox=[-180, 180, 60, 90],
                        extend='max',
                        load=True,
                        projection='northpole',
                        xgrid=False, ygrid=False,
                        clabel='{desc}',title='LR',
                        bounds=np.arange(0,10000000000,1000000000),
                        datagrid=dict(color='k', linewidth=0.1),
                        tight=True)

A full reproducible example is available at Github, with all the auxiliary files needed: https://github.com/davidmnielsen/gridcell_area

This is not a bug but instead a request of clarifications and additional infos.

I have a suite of scripts which download, process and plot data from the operational ICON model run at DWD and available on opendata.dwd.de. This data contains a LOT of points so optimizing the processing and plotting of data is paramount in order to maintain the script operational every day.

I'm trying to understand whether a transition from my combination of basemap and matplotlib to psyplot would be possible. In order to do that I'm trying to benchmark execution time of psyplot when doing some of my standard plots. In theory, as far as I understand, for filled contour plots psyplot is calling pyplot.tricontourf under the hood, so the difference should not be that big. The only difference, I guess, is an additional layer of preprocessing on the data, which I'm doing anyway in my script.

However I'm having some problems with psy-maps.

First of all, as the files downloaded from DWD opendata server do not have grid information inside (presumably to save space) I need to do a cdo setgrid before passing the file to psyplot.project.plot.mapplot function otherwise the plotting does not work. As far as I understand psyplot does not only need clat and clon from the grid but also the edges. Is it possible to use one of the more primitive methods before mapplot to pass only three 1-dimensional arrays (e.g. clat, clon and data) as done in pyplot.tricontourf? This way I can use an external file to store only the grid information as I currently do.

Second, I tried to reproduce my basic plot that you see here

which I simply do like this:

dset= read_dataset(variables=['T'])
lat, lon = get_coordinates()
temp_850 = dset['t'].metpy.sel(vertical=850 * units.hPa).load()
temp_850.metpy.convert_units('degC')
fig = plt.figure(figsize=(15, 10))
ax  = plt.gca()
_, x, y = get_projection(lon, lat, "world")
cs = ax.tricontourf(x, y, temp_850[0], extend='both', cmap=get_colormap("temp"),
                                    levels=np.arange(-35., 30., 1.))

read_dataset, get_projection and get_coordinates are functions that I wrote to simplify the code. They only wrap some xarray and basemap calls. This takes about 34 seconds of wall time; the input array has dimensions time: 93, ncells: 2949120. The coordinates lat and lon are saved in a different file.

Trying to do the same with psy-maps I first created a file where I also store the entire grid information inside (not only clat and clon otherwise it doesn't work as said before)

psy.plot.mapplot('~/Downloads/temp/test_psy/T_2020093006_global_grid.nc',
                 name='t', projection='robin',
                 levels=np.arange(238, 303, 2.), cmap=get_colormap("temp"),
                 )

I get this result

which takes about 50 seconds.
It seems that the levels parameter is completely ignored and somehow I cannot pass the pyplot axis to mapplot to control the figure size.

Do you have any idea on how to create a plotting script that I could use to reproduce my plotting setup so that I could benchmark it and see whether it is really faster?

Right now in the documentation of psy-maps there are only examples with really small grids so I'm not sure how the packages would behave for larger grids.

Code Sample

import psyplot.project as psy
from matplotlib.colors import Normalize

combined = psy.plot.mapcombined(
    'les_toscana_test_icon_oper_DOM03_ML.nc',
    time=1,
    name=[['t_2m', ['u_10m', 'v_10m']]],
    bounds=Normalize(vmin=280, vmax=310),
    cmap='gist_ncar',
    vplot='stream',
    )

Problem description

When running the code I get the following error

Plotter.__init__(self, data, ax, auto_update, project, draw, make_plot, clear, enable_post, **kwargs)
   1258 for key, value in six.iteritems(kwargs):  # then the user values
   1259     self[key] = value
-> 1260 self.initialize_plot(data, ax=ax, draw=draw, clear=clear,
...
--> 469     data = data.to_series()
    470 mask = np.asarray(data.notnull())
    471 return data.values[mask]
AttributeError: 'InteractiveList' object has no attribute 'to_series'

The same code works when using mapplot without the vectors, so there must be some problems when passing the normalization istance of matplotlib in mapcombined.

Output of psyplot -aV

As identified by @Casthardi, grid cells at the pole are not warped correctly. The screenshot below is a northpole projection (i.e. the center of the plot is at 90 °N) of an ICON file, and we can see that some grid cells are displayed correctly

This repository needs to be moved to the psyplot github organization. The issue will be closed when everything is setup

• Repo moved
• CI fixed
• Docs fixed
• Reference figures and CI scripts updated

Links to other psyplot projects fixed

• badges
• other files

Using ICON netCDF files where some variables have no time dimension, these variables are greyed out and cannot be displayed.
See screenshot and attached nbetCDF file

extpar_EU-R02B06_DOM01.nc.zip

Code Sample, a copy-pastable example if possible

take psy-views rotated-pole-test.nc file for instance, and plot it via

psy.plot.mapplot("rotated-pole-test.nc", plot="poly", name="t2m")

Problem description

The grid cell boundaries are not transformed correctly, what you get is

Expected Output

What you should get is

Output of psyplot -aV

psy_maps.plugin:
  requirements:
    cartopy: 0.17.0
  version: 1.3.0+5.g0fc5e8c
psy_reg.plugin:
  requirements:
    scipy: 1.4.1
    statsmodels: 0.11.1
  version: 1.3.0+2.gd87aa82
psy_simple.plugin:
  version: 1.3.0+4.g9388e25
psy_strat.plugin:
  version: 0.1.0
psyplot:
  requirements:
    matplotlib: 3.2.1
    numpy: 1.18.4
    pandas: 1.0.3
    python: 3.7.6 | packaged by conda-forge | (default, Mar 23 2020, 23:03:20)  [GCC
      7.3.0]
    xarray: 0.15.1
  version: 1.3.0+10.g867092b
psyplot_gui:
  requirements:
    pyqt: 5.12.3
    qt: 5.12.5
    qtconsole: 4.7.4
  version: 1.3.0+3.g7df7d45
psyplot_test.plugin:
  version: 1.0.0

Code Sample, a copy-pastable example if possible

p = psy.plot.mapplot(in_file,
                     name=var_name,
                     projection=projection,
                     map_extent=[lonmin,lonmax,latmin,latmax])

plt.savefig('plot.png')

Problem description

The plotted area extends over the whole figure when I specify a sub-domain with map_extent. The issue does not occur when the script is employed interactively within ipython, only when the figure is saved as .png. Additionally specifying lonlatbox=[lonmin,lonmax,latmin,latmax] does not solve the issue.

Expected Output

I'd like to have a clean cut at the edges.

Output of psyplot -aV

NOTE

hi!

I tested psy-view with icon data generated by

cdo -f nc topo,icoR02B05 ti.nc

initial plot with psy-view works fine, but when I zoom in the whole canvas is painted with contour colors:

when I click on the canvas once the map gets re-drawn and everything looks fine again

This does not happen with regular gridded input file

cheers
ralf

I use psy-view 0.1.0

Output of psyplot -aV

psy_maps.plugin:
  requirements:
    cartopy: 0.18.0
  version: 1.3.2
psy_simple.plugin:
  version: 1.3.1
psyplot:
  requirements:
    matplotlib: 3.3.4
    numpy: 1.20.1
    pandas: 1.2.3
    python: 3.9.2 (default, Feb 20 2021, 18:40:11)  [GCC 10.2.0]
    xarray: 0.16.2
  version: 1.3.1
psyplot_gui:
  requirements:
    pyqt: 5.15.4
    qt: 5.15.2
    qtconsole: 5.0.1
  version: 1.3.0

Summary

Within the UGRID conventions it is possible to defined variables to live on the nodes, rather than the faces. Within psyplot, we do visualize them at the the moment, but silently this provides wrong results.

Reason

It should be possible to visualize these elements, too.

Detailed explanation

Within the get_cell_node_coord method of the UGRID decoder, we generate triangles using a delauney triangulation.

https://github.com/psyplot/psyplot/blob/36a23ce964d58e41468fd1f97188b75454c14b7d/psyplot/data.py#L1807

I think this is fine for now as we do not have generic methods to generate grids from the nodes. What grid is the best is a very scientific question and should rather be answered by a custom decoder class for the data.

None the less, the current implementation in psy-maps (and psy-simple) is wrong. In the _polycolor method, we say transformed, array=arr.ravel(). For variables on a node however, this does not generate correct results because the length of the array (which is the same as the number of nodes) is not the same as for transformed, which has the length of the number of triangles.

So array should actually be the mean of the nodes for each generated face element (i.e. for each generated triangle).

Examples

matplotlibs tripcolor method is doing exactly this: https://github.com/matplotlib/matplotlib/blob/e097bf4baf8f275fda91f224b537076caf17dd91/lib/matplotlib/tri/tripcolor.py#L108

ping @platipodium

On macOS with the latest Psy-View version displaying the parameter HSURF in the file
cdo001_c.nc.zip
I get the following figures (upper right wit option "mapplot" lower right with option "plot2d"

Consider the following (zipped) netCDF file

regional-icon-test.zip

psy.plot.mapplot('regional-icon-test.nc', name='t2m', projection='ortho', cmap='viridis')

produces the following plot

and clon and clat are both 0. Instead, it should be

Clon and clat are probably not converted to degree when calculating clon and clat.

In these cases the whole earth (in longitudinal direction) is displayed.


and the related data files
CORDEX-data.zip
It should be like e.g. the European region, i.e. only the necessary part should be displayed:

• CircleCI-config added
• Redirects configured in RTD

Code Sample, a copy-pastable example if possible

sp = psy.plot.mapplot('test.nc')
sp.update(projection='ortho')

Problem description

Updating the projection removes the plot (or generates wrong results). This is not the case for cartopy 0.17

Before update

After update

Code Sample

mapvectors = ds.psy.plot.mapvector(time=0,
                                lonlatbox=[0.,17.,42.,50.],
                                name=[['U', 'V']],
                                density=10, 
                                arrowsize=400, 
                                title='Vector Plot on Regular Grid')

Problem description

I dont seem to be able to get lonlatbox and density to work simultaneously on a vector plot. map_extent does work with density, but not lonlatbox.

Expected Output

Expect lonlatbox to work as usual when specifying arrow density in vectorplot.

Output of psyplot -aV

psy_maps.plugin:
  requirements:
    cartopy: 0.20.2
  version: 1.4.1
psy_reg.plugin:
  requirements:
    scipy: 1.8.0
    statsmodels: 0.13.2
  version: 1.4.0
psy_simple.plugin:
  version: 1.4.1
psy_transect.plugin:
  version: 0.0.1.dev0
psyplot:
  requirements:
    matplotlib: 3.4.3
    numpy: 1.22.3
    pandas: 1.4.1
    python: 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:20:46)  [GCC
      9.4.0]
    xarray: 2022.3.0
  version: 1.4.2
psyplot_gui:
  requirements:
    pyqt: 5.12.3
    qt: 5.12.9
    qtconsole: 5.3.0
  version: 1.4.0

NOTE

This is a bug report.