ncas-cms / cf-plot Goto Github PK

In #34 it was noticed the current docs source material contains numerous issues that throw warnings in the Sphinx build, which should be worked through and remedied. Some of these are indicative of issues in the code itself and not just documentation-related, which are the priority to fix, though it is probably simplest to aim to fix them all at once and if any emerge as more difficult to fix, raise a separate Issue to address later.

Specific issues, copied from console build process resulting in update from #34

$ make html                                                                   ─╯
python3 -m sphinx -b html -d ../build/doctrees   . ../build
Running Sphinx v7.2.6
[autosummary] generating autosummary for: add_cyclic.rst, advanced.rst, axes.rst, axes_plot.rst, bfill.rst, calculate_levels.rst, cbar.rst, cf_data_assign.rst, cf_var_name.rst, check_data.rst, ..., version_2.2.rst, version_2.3.rst, version_2.4.rst, version_3.0.rst, version_3.1.rst, version_3.2.rst, version_3.3.rst, versions.rst, vloc.rst, wrf.rst
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 88 source files that are out of date
updating environment: [new config] 88 added, 0 changed, 0 removed
reading sources... [100%] wrf
/home/slb93/git-repos/cf-plot/docs/source/advanced.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/cbar.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.cf_var_name:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.cf_var_name:17: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/colour_scales.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/con.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.con:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/docs/source/cscale.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.cscale:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.cscale_get_map:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.cscale_get_map:8: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/cylindrical.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/download.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.find_pos_in_array:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.find_pos_in_array:13: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/gclose.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/gopen.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/gpos.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/graphs.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/gset.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/hovmuller.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/hovmuller.rst:6: WARNING: Title underline too short.

Example 10 - latitude-time
-------------------------
/home/slb93/git-repos/cf-plot/docs/source/internal_routines.rst:4: WARNING: toctree contains reference to nonexisting document 'timeaxis'
/home/slb93/git-repos/cf-plot/docs/source/levs.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/lineplot.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.mapaxis:11: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/mapset.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/multiple_plots.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.ndecs:7: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/polar.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.polar_regular_grid:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.polar_regular_grid:14: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/pressure.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.process_color_scales:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.process_color_scales:15: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/projections.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.regrid:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.regrid:12: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/reset.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.reset:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.reset:11: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.rgaxes:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.rgaxes:20: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/rotated_pole.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/rotated_pole.rst:32: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.set_map:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.set_map:12: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/setvars.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.setvars:32: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.setvars:67: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/stipple.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stipple:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stipple:21: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/stipple_plots.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/stipple_plots.rst:3: WARNING: duplicate label stipple, other instance in /home/slb93/git-repos/cf-plot/docs/source/stipple.rst
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stipple_points:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stipple_points:16: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stream:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.stream:31: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.supscr:8: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/training.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/traj.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/trajectories.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/trajectories.rst:140: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/docs/source/unstructured.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/user_defined.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst:476: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst:485: WARNING: Inline substitution_reference start-string without end-string.
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst:1146: ERROR: Unknown target name: "colorbar".
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst:585: WARNING: duplicate label trajectories, other instance in /home/slb93/git-repos/cf-plot/docs/source/trajectories.rst
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst:664: WARNING: duplicate label graphs, other instance in /home/slb93/git-repos/cf-plot/docs/source/graphs.rst
/home/slb93/git-repos/cf-plot/docs/source/vect.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.vect:2: WARNING: Line block ends without a blank line.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.vect:62: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/vectors.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_1.5.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_1.6.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_1.7.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_1.8.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_1.9.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_2.0.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_2.1.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_2.1.rst:49: ERROR: Error in "image" directive:
no content permitted.

.. image::  images/scale1_original.png
   :scale: 52%




 ::

   Done
/home/slb93/git-repos/cf-plot/docs/source/version_2.1.rst:94: ERROR: Error in "image" directive:
no content permitted.

.. image::  images/cm.png
   :scale: 52%



 ::

   Done
/home/slb93/git-repos/cf-plot/docs/source/version_2.1.rst:160: ERROR: Error in "image" directive:
no content permitted.

.. image::  images/colourscales/BrBG.png
   :scale: 52%


 ::

   Done
/home/slb93/git-repos/cf-plot/docs/source/version_2.2.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_2.3.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_2.4.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_3.0.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst:92: WARNING: Title underline too short.

10. cfp.gpos(1) causes stray box lines
=====================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst:92: WARNING: Title underline too short.

10. cfp.gpos(1) causes stray box lines
=====================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst:218: WARNING: Title underline too short.

19. cfp.con - improved Z axis detection
======================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst:218: WARNING: Title underline too short.

19. cfp.con - improved Z axis detection
======================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.2.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/version_3.2.rst:182: WARNING: Title underline too short.

14. cfp.con - ugrid keyword changed to irregular
===============================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.2.rst:182: WARNING: Title underline too short.

14. cfp.con - ugrid keyword changed to irregular
===============================================
/home/slb93/git-repos/cf-plot/docs/source/version_3.3.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/cfplot/cfplot.py:docstring of cfplot.vloc:12: WARNING: Field list ends without a blank line; unexpected unindent.
/home/slb93/git-repos/cf-plot/docs/source/wrf.rst:2: WARNING: Explicit markup ends without a blank line; unexpected unindent.
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/slb93/git-repos/cf-plot/docs/source/advanced.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/colour_scales.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/cylindrical.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/download.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/gallery.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/graphs.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/hovmuller.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/internal_routines.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/issues.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/license.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/multiple_plots.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/older_documentation.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/polar.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/pressure.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/projections.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/rotated_pole.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/routines.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/stipple_plots.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/training.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/trajectories.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/unstructured.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/user_defined.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/user_guide.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/vectors.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version3_changes.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.3.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.5.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.6.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.7.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.8.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_1.9.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_2.0.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_2.1.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_2.2.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_2.3.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_2.4.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_3.0.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_3.1.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_3.2.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/version_3.3.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/versions.rst: WARNING: document isn't included in any toctree
/home/slb93/git-repos/cf-plot/docs/source/wrf.rst: WARNING: document isn't included in any toctree
done
...

During #34, it was realised it would be useful and pertinent to add a link to Andy's built documentation (for v3.3.0) to the 'Older documentation' page, as a link to the final generated copy under his user space, for persistency and to preserve the history, now that we have moved the docs to live under the NCAS-CMS GitHub user/org space.

Some distribution-related files have been committed previously to the top level of the repository, but these aren't useful for anything and could quite actively confuse package managers if the repo is cloned. At present they just bloat the repo, so should be removed. These are:

• the whole cf_plot.egg-info directory;
• the MANIFEST file, but not the MANIFEST.in file which is important to identify what gets included in the distribution.

After #45, where auto-formatting was applied to abide by PEP8, I notice we are still left with numerous cases of strings, often longer and/or inclusive of variable string-representation reporting, which are formatted in a mixture of ways:

which can be replaced by wrapping in parentheses and stating each component on a new line, where they will automatically be merged into one.

We should convert these so they use a consistent formatting, for all but cases where there is a clear reason to use a different approach e.g. clarity when { is needed in the string. Since it is used throughout cf-python and is what am most familiar with, I will use f-string formatting, where we can prefer .format as a second choice if there's a reason f-string aren't clear for a given case. We should therefore remove all cases of old-style/% formatting.

Are there any plans to support Python 3 in the near future?

And with the upcoming API changes in CF-Python 3, will these be supported as well?

At the moment, I've forked the CF-Plot repository and converted the code as good as possible to Python 3 and the new CF-Python API, but as I'm not familiar with the CF-Plot code base, this attempt is likely lacking, and just a poor-man's attempt at providing support for some colleagues that want to try and use CF-Plot with Python 3.

In addition to standard unit tests for functionality, we should set up testing which checks that images produced by routines with visual output are a precise match to a reference image known to be correct, for a wide coverage of applicable inputs.

In the codebase there are currently some limited tests of this description, which use ImageMagick to compare outputs with reference images. The coverage of these tests should be increased, but first I propose to change the means for image comparison.

The README is very bare at the moment. We should populate it with plenty of information, for example quoting from the GitHub documentation page on READMEs:

A README is often the first item a visitor will see when visiting your repository. README files typically include information on:

What the project does
Why the project is useful
How users can get started with the project
Where users can get help with your project
Who maintains and contributes to the project

In particular I would like to include some images and links, notably showcasing some nice plots that cf-plot can make, and linking and advertising cf-python as the underlying data analysis tool.

Currently the codebase does not conform to PEP8 (and aligned) style conventions which are standard for Python code, for example it makes heavy use of old-style string formatting and has a large amount of trailing whitespace.

To avoid spending unnecessary time and effort to conform by such formatting, we should use a tool like black which will auto-format in a precise and absolute way as an extension of PEP8.

We can then use linters including pre-commit hooks to ensure that the style continues to abide by those standards (amongst other verifications).

Transfer the full cf-plot documentation, as-is, and importantly wit the full capability to re-generate from source updates, from http://ajheaps.github.io/cf-plot/ to a new home of https://ncas-cms.github.io/cf-plot, consistent with the other NCAS CF Data Tools libraries. This change is already promised under the current homepage:

Warning - these pages are static and will be removed at some stage in the future. The new location will be https://ncas-cms.github.io/cf-plot

We should remove any commented-out 'dead' code in the codebase (and going over it lately I have found a significant amount peppered about), which will live forever in the version control system so doesn't need to clog up the line count of the present code, even if those lines are important for some contextual reason.

Hi Andy,

Just a minor issue I've had with plotting ocean salinity timeseries. These typically have a mean of ~34 but very small (<0.001) variations about that mean. lineplot fails when I attempt to plot this (see attached example).
example.tar.gz

[dlrhodso@sci1 monitoring]$ ./lineplot_example.py 
Traceback (most recent call last):
  File "./lineplot_example.py", line 10, in <module>
    cfp.lineplot(data)
  File "/apps/jasmin/jaspy/miniconda_envs/jaspy3.8/m3-4.9.2/envs/jaspy3.8-m3-4.9.2-r20211105/lib/python3.8/site-packages/cfplot/cfplot.py", line 6690, in lineplot
    yticks, ymult = gvals(dmin=miny, dmax=maxy, mod=mod)
TypeError: cannot unpack non-iterable NoneType object

The problem seems to be around line 3227 in cf_plot ie gvals()

    # Modify if requested or if out of range 0.001 to 2000000
    if data_range < 0.001:
        while dmax1 <= 3:
            dmin1 = dmin1 * 10.0
            dmax1 = dmax1 * 10.0
            data_range = dmax1 - dmin1
            mult = mult - 1

for the salinity data data_range is 0.0004 but the line while dmax1 <= 3: is skipped over, because dmax1 is ~34.

If I replace while dmax1 <= 3: with while data_range < 0.001: this bug is fixed!

It's a minor issue that will probably only affect a few ocean folk, but thought I'd mention it!
Dan

A pair of users have reported, for the same dataset, that the stipple method is erroring as follows:

cfp.stipple(g, min=0.2, max=0.9, size=100, color='#00ff00')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cfplot/cfplot.py", line 4227, in stipple
    plotvars.mymap.scatter(xnew[valid_points], ynew[valid_points],
AttributeError: 'NoneType' object has no attribute 'scatter'

I have asked for further details and will note anything significant on this thread; the specific case in which this emerges is still under investigation.

The current documentation has an 'Issues' page which states that user support queries should be sent as a email, directed previously to Andy and now to me at our institutional addresses, but we should instead advise these be raised on this Issue Tracker instead as a more open and systematic means for issue reporting and management.

In practice, this means replacing the text there, namely this and anything similar:

If you find a problem with cf-plot please email Sadie Bartholomew ([email protected]) with the following:

with a link to the tracker here.

There are numerous functions marked, in their docstring, with the statement "This is an internal routine and is not used by the user." e.g:

It would be ideal, for clarity to developers, to follow tradition and rename these all so they have leading underscores in their names.

At present, cf-plot only supports two-dimensional plotting, by means of interfacing with the Cartopy and matplotlib libraries.

However, as a long-term goal, a feature that would be really useful to users would be to also support 3D plotting. This would be very time-consuming and complex to develop from scratch, but if we interface with libraries which support 3D cartographic plots, this can do the heavy lifting, and as with the 2D case we then provide an interface on top of that to handle metadata considerations and such and consolidate the set-up using our own API.

Such a library that looks very promising in this respect is GeoVista. It looks like it is still under-development (pre-v1.0 and placeholder docs) but once this looks more mature we should get in touch with the developers to investigate ways forward towards the goal of supporting 3D plotting. There may also be other libraries that would work, so we should explore the ecosystem for anything relevant.

colorbar_position is broken (for some use cases at least) in 3.2.0

Test case:

data file test.nc attached (<1MB, but in a zip file because GitHub doesn't allow attaching .nc directly): test.nc.zip

Broken example (paths redacted in error message):

>>> import cf
>>> cf.__version__
'3.14.1'
>>> import cfplot as cfp
>>> cfp.__version__
'3.2.0'
>>> f=cf.read('test.nc')[1]
>>> f
<CF Field: eastward_wind(time(1), pressure(1), latitude(160), longitude(320)) m s**-1>
>>> cfp.con(f, colorbar_position=[0.1, 0.1, 0.8, 0.02])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/cfplot/cfplot.py", line 1029, in con
    cbar(labels=cbar_labels, orientation=cb_orient, position=colorbar_position,
  File "/path/to/cfplot/cfplot.py", line 8256, in cbar
    colorbar = matplotlib.colorbar.ColorbarBase(ax1, cmap=cmap,
UnboundLocalError: local variable 'ax1' referenced before assignment

Working example (same code, earlier cfplot version - also a different cf version but guessing that this is unimportant):

>>> import cf
>>> cf.__version__
'3.13.0'
>>> import cfplot as cfp
>>> cfp.__version__
'3.1.23'
>>> f=cf.read('test.nc')[1]
>>> f
<CF Field: eastward_wind(time(1), pressure(1), latitude(160), longitude(320)) m s**-1>
>>> cfp.con(f, colorbar_position=[0.1, 0.1, 0.8, 0.02])
>>> 

produces:

There are basic tests covering a limited subset of routines contained in a method regression_tests in the main module, at present, though I am told these haven't been run for many years:

These should be moved out to a dedicated testing module and the coverage increasing greatly to ensure all functions are checked across the cf-plot API.

Naturally, many functions produce visual output, where image-comparison checks against correct intended outputs are the robust means to test the core functionality. This is slightly more complicated and different with respect to setup, and therefore will be covered in another issue, however in these cases under this scope we can at least test that relevant calls to functions run without error to produce some image which opens successfully in a matplotlib window.

As well as linting to ensure proper agreed code style (see #16), we should also use some tool to lint for detectable errors such a syntax errors and undefined variables that may not be picked up by testing. Tools that do this include flake8 and ruff.

When this is included, we should run these linter(s) as (part of) regular CI jobs.

Change the Sphinx docs browser-based (HTML-format build) theme from the 'classic' theme to a more modern one which allows the user to switch between light and dark modes.

This is for accessibility and readability purposes: for various reasons, some people will be able to engage much better with content on a dark(er) background. But this is also a personal feature request since I can get migraines and through experience know that lots of time spent looking at (dark text on) light backgrounds can in some cases trigger a migraine and since I'll be the (main) one developing the docs, it seems important to provide a means for me to view in a dark mode.

From the link above showcasing different themes, it looks like there are at least a few which support user configurability between light and dark modes, in this way.

cf-plot has an 'extend' keyword option to cbar, levs and to specific functions to enable or disable extension markers on each end of the displayed colour bar, which is set by default to be enabled at both ends:

which produces colour bar ends like this (see red annotations):

However, in practice I have seen people show plots in presentations where the values plotted start at zero where the extension marker is still at the lower end. This made me think that we should update our default behaviour.

Namely, given very often when data values begin at 0(.0) that means the data in question doesn't make physical sense as a negative value, e.g. for a ratio or fraction such as albedo, for a vector magnitude such as wind magnitude, or for a scalar quantity such as mass which has no meaning in negative terms.

In these cases by default we still show the extension markers on the lower end, which is misleading data visualisation (and as much as the user should change this on the plot to correct it, often I have seen this hasn't been done - maybe they didn't have the time, or weren't aware that cf-plot allows one to configure this). In these cases, the colour bar should have extend='max' as the default behaviour, so that only the colour bar upper end gets the extension marker end:

(Note this would only be default behaviour, and can still be overridden by the user in case the range of possible values does extend below zero and they just happen to want to show the levels beginning with zero.)

It would be nice to have a Cheat Sheet document to summarise important function calls and outputs and in general the utility of cf-plot.

We currently have a fantastic one for cf-python, and that has the benefit of working both in online format and as a printed document. I suggest we design our Cheat sheet' so that it extends the cf-python one to showcase the plotting capability with cf-plot after set up with cf-python as per the cf-python Cheat Sheet. If we can create an identical page format, then the cf-plot cheat sheet can also be printed out and sit alongside the cf-python one to form an overall cf* NCAS Data Tools Cheat Sheet that is twice the size but the individual documents can also work standalone.

Use the Python built-in logging module to manage messaging to the user, in particular to add finer configurable granularity of detail and/or severity of said messages, and consolidate the code so that it doesn't have to wrap anything to print with if verbose statements, like so:

As with cf-python, I think it would be useful also here to add another level as a custom one to extend the five provided, which indicates extra detail as opposed to just increased severity (see https://github.com/NCAS-CMS/cf-python/blob/de6d0b5cf9d52aa0d0302d76e68fdfab94a62004/cf/__init__.py#L359-L370).

Presently the docstrings in the codebase are in some format I do not recognise, so we should convert them into a standardised format that is recognised by the Sphinx docs builder.

Since cf-python uses a format closely aligned with the Google standard, it makes sense to be consistent with that and to use the same, so that is my choice unless anyone has any objections.

There are currently two files with the filename README in the root of the repository, one with .txt and one with .md as extensions. We should remove one, as having two in the repository root is confusing and only one gets rendered on the library homepage.

I prefer keeping the markdown file, since it means we can have rich text formatting unlike with pure text and also that file is older (9 years according to the commit history).

Follow-on work noticed during #47:

there are lots of new-line characters peppered about the strings especially for user messages, which I will remove (they are not useful since users will be viewing these on terminals or IDEs or similar of variable width where standard line wrapping will apply for readability, and these custom new lines could cause weird gaps and wrapping issues)

We should set up some form of Continuous integration to run tests and linting on different operating systems and with different Python versions, for example, to streamline and corroborate our workflows.

I advise we use GitHub Actions since it is free to use, very powerful and mature with great documentation, and I have plenty of experience in using it.

We should add some GitHub community health files to, in particular, indicate that development contributions are welcome and demonstrate the basic workflow contributors can use to raise issues and suggest code changes.

We have a working setup.py file with basic metadata covering the essentials, but I would like to add much more (optional) metadata to cover such important information as:

• keywords and classifiers to tag the context of cf-plot;
• constraints on the Python version required to run;
• compatible OS platforms.

The example in the cf-plot docs assume that the user has access to the example file on the Reading systems. It might be worth producing a small example netCDF file (with some heavy subsetting, just big enough to produce a plot), and including it in the distribution, with the docs changed to refer to it. Thanks.

As alluded to in #37, the .gitignore is presently very bare and many files need to be manually ignored when staging for a commit. The simplest way to get a more useful .gitignore without having to think much about what needs to be added, is to copy one from a pre-populated Python template, for example as provided by GitHub itself (now an option when you create a repo to add it, but it wasn't back in the day...) i.e. the file here.

Notably though, from looking at this template I would also like to customise it a little, e.g. to:

• add Sphinx artefacts to be ignored, such as the .rst.txt under /build/_sources/*,
• add data files (mostly netCDF), which we shouldn't generally touch, but might want to play around with for testing and end up writing over with edits, therefore would be good to include to ignore.

Hi @sadielbartholomew

It has been seen that sometimes cfplot seg faults for some versions of matplotlib when esmpy is also installed.

E.g., for me, esmpy==8.6.0 and matplotlib==3.8.0

I have found, however, that matplotlib==3.8.3 works with various versions of esmpy. So, could we make that the new minimum version requirement?

Thanks,
David

Dear Andrew,

first of all many thanks for this nice module!

I am trying to use it in IPython notebook, and experience problems with plotting polar projections. It seems like proj='npstere' in mapset does not have any effect...

Here is an example (I use NCEP reanalysis data)
http://nbviewer.ipython.org/gist/koldunovn/6eeaaf5b4741fefc14f1

Currently the vast majority of the code is contained in one file, cfplot.py, which as a consequence is over 10,000 lines long and difficult to follow, manage and navigate. It would be much better to split this into separate files in a modular hierarchy based on functionality and natural groupings of objects, etc.

I will come up with a plan for what I see is the most appropriate modularisation for cf-plot shortly and post it here as a proposal, before undertaking the work.

Quite self-explanatory, though see NCAS-CMS/cf-python#190 (comment) for some context if required.

There are pages that document the per-release change history of cf-plot, namely from a landing page entitled 'Planned releases', but in my opinion they are too hidden away from anyone that might to see these from the repository homepage and also the title is misleading since these releases have already been made.

Therefore we should translate the information about changes on these pages onto one single stand-alone file which documents the full change history, with the standard name of 'change log'.

Note there is a file 'CHANGES.txt' in the root of the repository, which fits the requirements here, though it only has one entry with the last commit from over 12 years ago.

I propose we rename and update this as the location of the proposed lone change log, purely to preserve the timestamp of 12 years' worth of development (even if some early detail will be missing!).

Similarly to #49, but more specifically, there is a long code block of logically-similar conditional setting which can and should be reduced down to three or so lines using setattr(), since the statements all have the same structure:

As well as extending the coverage of image comparison tests I propose to change the approach to doing these comparisons (see #13 for details and the current method), namely to instead use a framework for the underlying image comparison infrastructure already set up and utilised by Matplotlib for the comparison, namely the matplotlib.testing.compare module with methods such as compare_images

Reasons this would be a useful change include that:

• the matplotlib means for image comparison is maintained externally and known to be (and stay) robust;
• it provides syntactic sugar in the form of dedicated decorators so that unit tests can merely be decorated to configure for image comparison.

Further to #22, it appears to me that some warnings raised would be more appropriate as being fatal, hence should be upgraded to errors. Therefore, when we systematically go through all the Warning cases, we should also consider if an error is more appropriate and if so, convert to some specific Exception.

There are ~20 cases where a Warning is raised in the code:

$ pwd
/home/slb93/git-repos/cf-plot/cfplot
$ git grep "raise Warning" | wc -l
21

These should be made more specific to categorise them, notably in most cases it seems from a quick look that they should each be upgraded to a UserWarning to make it clear it isn't something dodgy about the environment or code, but rather a message to the user about the context of cf-plot and the underlying matplotlib usage, etc.

See TODO from #31 (comment). To add, an example of notes to lift and shift in this manner is those under the corresponding page for version 1.3.

There are lots of locations in the code I have noticed where the conditional if/else logic can be streamlined or improved, so it flows more intuitively and reduces the need for extra levels of indentation, etc.

This is both in the sense of direct small-scale conversions, for example, there is a whole similar block that can be reduced with elif statements here:

as well as larger-scale changes to the conditional structure. All of which should not change the behaviour of the code in any way, only its implementation, to make it easier to work with and develop on top of.