scitools / iris Goto Github PK

The current merge process will choose the first candidate dimension or first candidate dimension group that satisfies a functional relationship.

This may not align with the expected or desired cube from a user perspective.

Investigate the hint mechanism and factory querying in order to determine the best default ordering for candidate dimensions.

i.e.

• time over forecast_reference_time over forecast_period,
• and level_height over model_level_number over sigma (or atmosphere_hybrid_height_coordinate) for hybrid-height.

Update: Refer to #25 for an update to the preferred order (model_level_number is now primary)

I have now seen several codefiles from users where "sample_data_path" has been used inappropriately.
For example, like --

fname = iris.sample_data_path('my', 'file', 'that', 'isnt', 'part', 'of', 'the', 'sample', 'data')

( Spot the pointless-ness. )
I'm sure this just comes from copying example code.
So, I think it could be better managed, either in documentation or code structure.

I have a couple of related queries with this --

1. Although "sample_data_path" appears all over the examples, there is no documentation entry for it, because of the visibility restriction built into iris.init.py ...

# Restrict the names imported when using "from iris import *"
__all__ = ['load', 'load_strict', 'save', 'Constraint', 'AttributeConstraint']

2. Possibly, it should live in iris_tests anyway ??

qplt.pcolormesh(iris.load(iris.sample_data_path('uk_hires.pp'))[0][0, 0, ...])

Generates the exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lib/iris/quickplot.py", line 158, in pcolormesh
    result = iplt.pcolormesh(cube, *args, **kwargs)
  File "lib/iris/palette.py", line 138, in wrapper_func
    return func(*args, **kwargs)
  File "lib/iris/plot.py", line 705, in pcolormesh
    return _draw_2d_from_bounds('pcolormesh', cube, *args, **kwargs)
  File "lib/iris/plot.py", line 214, in _draw_2d_from_bounds
    result = _map_common(draw_method_name, None, iris.coords.BOUND_MODE, cube, data, *args, **kwargs)
  File "lib/iris/plot.py", line 349, in _map_common
    lats, lons = iris.analysis.cartography.get_lat_lon_contiguous_bounded_grids(cube)
  File "lib/iris/analysis/cartography.py", line 196, in get_lat_lon_contiguous_bounded_grids
    lons = lon_coord.contiguous_bounds()
  File "lib/iris/coords.py", line 548, in contiguous_bounds
    self._sanity_check_contiguous()
  File "lib/iris/coords.py", line 533, in _sanity_check_contiguous

Install on Windows fails with package directory 'lib\iris_pyke_rules' does not exist. This is due to find_package_tree not handling the fact that \lib\iris\fileformats directory contains a sub-directory.

Tests carried out on Windows 7.

Following through from #1...
Implement hybrid pressure loading and saving for GRIB.

Currently the user must perform an additional check to see whether the coordinate is scalar or dimensioned before attempting a collapse.

cubecol = cube.collapsed('time',iris.analysis.MEAN)

"... iris/cube.py",
line 1839, in collapsed
raise iris.exceptions.CoordinateCollapseError('Cannot
collapse a dimension which does not describe any data.')

The use of standard name and long name is sometimes sporadic in netCDF files. It appears that netCDF files sometimes contain coordinate variables where the variable name has significant meaning and may be more suitable than the long name as the coordinate name on the resulting cube. For example, consider a file with a coordinate variable called 'level', with no standard name, but a descriptive long name with lots of spaces. Constrained loading then becomes troublesome as:

iris.load(filename, my long descriptive long name coordinate=10)

clearly won't work. Perhaps this is an indication that this syntax is a little strange, and yes there is the option of the coord_values keyword. However, what the user wants to do is:

iris.load(filename, level=10)

where level is the CF variable name, but as things currently stand, this psuedo-metadata is lost on loading.

Consider:

1. Should we retain the CF variable name?
2. If so, how?
3. How does this fit with the CF data model?
4. Should it be used on save to netCDF?
5. What about saving to other formats?

Prepare and make available the data for the gallery examples.

Remove TEST_COMPAT code and legacy XML methods.

Review the iris._merge.ProtoCube._guess_axis() method.

Align this with the iris.util.guess_coord_axis() function.

In particular, ensure the correct ordering for forecast_time.

NB. This change will result in wide-spread .cml changes due to cube dimension reordering.

Also see #7.

Convert the coordinate systems in iris.coord_systems to CF-style structures/names/etc.

Conventionally latitude precedes longitude.

analysis.cartography.get_lat_lon_grids(cube) returns lats, lons
analysis.cartography.unrotate_pole(...) returns lons, lats

More generally:
grep -r 'lat._lon' * | wc -l
1668
grep -r 'lon._lat' * | wc -l
206

Does this need to be consistent, or return a dictionary {lat: y, lon: x}?

Currently the Iris documentation does not state which order the coords will be returned for rotate_pole and unrotate_pole.

The gitwash point to the old Iris.git repository rather than the iris.git repository. Re-build the gitwash docs with the appropriate command.

Note: See docs/iris/src/developers_guide and specifically, gitwash_command.txt

As a hangover from the transition to the CF data model, the CF loader is creating a default GeogCS when no coordinate system is supplied in the file. This behaviour should be removed.

#54 avoided numrod plotting, pending #67.

Once #67 is merged, add a test to plot the loaded nimrod data.

Convert the tests and results symlinks back to real, in-line content.

Remove the legacy heuristics from iris.util.guess_coord_axis().

if coord.name().upper() == "X":
        axis = "X"
    if coord.name().upper() == "Y":
        axis = "Y"
    if coord.name().upper() == "Z":
        axis = "Z"

Note that these cases are used by the calculus code.

Cube.aggregated_by() appears to be broken.
The aggregated data dimension is not being collapsed.

https://github.com/bblay/iris_issue_files/blob/master/test_agg.py

$  python test_agg.py 
------------------
original_cube                       (w: 3; z: 3; y: 3; x: 3)
     Dimension coordinates:
          w                           x     -     -     -
          z                           -     x     -     -
          y                           -     -     x     -
          x                           -     -     -     x
shape (3, 3, 3, 3)
------------------
aggregated_cube                     (*ANONYMOUS*: 3; z: 3; y: 3; x: 3)
     Dimension coordinates:
          z                                     -     x     -     -
          y                                     -     -     x     -
          x                                     -     -     -     x
     Auxiliary coordinates:
          w                                     x     -     -     -
     Attributes:
          history: Mean of original_cube aggregated over w
     Cell methods:
          mean: w
shape (3, 3, 3, 3)

Scenario:

• load data
• setup current map projection as osgb (with region taken from cube)
• plot OS map image (with region taken from current map)
• cell-plot data

Hypothetical example code:

cube = iris.load("my_file")
iris.plot.map_setup("osgb", cube)
iris.plot.draw_wms(wms_url)
iris.quickplot.contourf(cube)

Currently iris cubes regridded method ignores the fact that masked data is present in the cube.

The culprit is within the analysis method interpolate.py where the numpy data array of the resulting cube is defined by the following:
new_data = numpy.empty(new_shape, dtype=source_cube.data.dtype)

A simple example of current behaviour is here:
https://gist.github.com/3512677

i.e.

• Switch XML output to new format.
• Remove legacy XML support.
• Update XML_NAMESPACE_URI to urn:x-iris:cubeml-0.2.
• Validate test results.

docs/iris/src/userguide/cube_maths.rst lines 28 and 29 refer to a cube called air_press. This should be air_temp.

Convert the source and history coordinates into attributes.

Use standardised Met Office attribute names and values to store STASH codes (as STASH() instances) and UM version numbers.

• Ensure the STASH instances are written out in netCDF as strings, not a triple of numbers - even when the STASH attribute exists on a coordinate variable.
• Ensure UM version number is expressed in standard form.

iris pp loading (metadata translation) is rather slow.
There seems to be some room for code optimisation here within the rule, though the amount of saving is as of yet are not known

Library changes:

• Add colour-deficient colour schemes from http://colorbrewer2.org/.
• Each colour scheme file should include the licence as shown below.
• Convenience function for adding a citation to a plot.

Documentation changes:

• An acknowledgement: "This product includes color specifications and designs developed by Cynthia Brewer (http://colorbrewer.org/)."
• The licence as shown below, and a mention of the "as is" nature of the colour schemes.
• Usage/courtesy guidelines:
  • Summary: Add a citation in the image (link to the convenience function); add a reference in a journal/article.
  • Link to http://www.personal.psu.edu/cab38/ColorBrewer/ColorBrewer_updates.html - section 5.

Licence text:

Apache-Style Software License for ColorBrewer software and ColorBrewer Color Schemes

Copyright (c) 2002 Cynthia Brewer, Mark Harrower, and The Pennsylvania State University.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.

There is no interpolation user guide, only reference documentation.
A few examples might be helpful. Perhaps a new section 4.4?
Or, as there's quite a bit in there, perhaps a new section after 4?

Refactor the _Groupby.group() method while loop in iris.analysis.__init__.py in order to remove the dependency of None elements in the items list variable.

This will involve binding the content of items and groups variables.

User feedback from Mac installation:

Mac OSX dynamic libraries end .dylib
rather than .so

I cheated by soft-linking libc.dylib to libc.so.6 and
libudunits2.dylib to libudunits2.so.

Extend iris.unit to support Mac-compatible references to libc and libudunits2 - i.e. the dylib suffix.

Enhance Iris so that you can write out packed PP files.

This is a user driven requirement to collapse a cube which has multidimensional coordinates with bounds and perform some analysis:
....
cube.collapsed(['latitude','longitude'], iris.analysis.SUM)
....

see gist for example:
https://gist.github.com/3706991

Making a cube iterable causes more issues than it is worth (@munslowa found this to his surprise).

import iris
fname = iris.sample_data_path('air_temp.pp')
temperature = iris.load_strict(fname)
for thing in temperature:
    print thing

Remove this capability from the cube.

This ticket proposes an enhancement to allow a region to be extracted by specifying a bounding box.

Currently, users may attempt to extract a region from a global dataset using constraints:

latc = iris.Constraint(latitude=lambda cell: minlat <= cell <= maxlat)
lonc = iris.Constraint(longitude=lambda cell: minlon <= cell <= maxlon)
region = cube.extract(latc & lonc)

However, the output will retain the circular property, which they would have to change manually:
region.coord('longitude').circular = False

The lib/iris/plot.py docstrings for iris.plot.contour(), iris.plot.contourf(), iris.plot.outline(), iris.plot.pcolormesh(), iris.plot.points() and iris.plot.plot() all list Coords as an argument rather than coords. As a result the reference documentation (http://scitools.github.com/iris/iris/iris.html) contains the same errors.

The cube summary markup can be misaligned.

See the output for dimension coordinates and auxiliary coordinates for the following netCDF file:

>>> import iris
>>> filename = '/data/local/dataZoo/NetCDF/label_and_climate/FC_167_mon_19601101.nc'
>>> cube = iris.load_strict(filename)
>>> print cube
air_temperature                     (*ANONYMOUS*: 120; *ANONYMOUS*: 21; latitude: 73; longitude: 144)
     Dimension coordinates:
          latitude                              -                 -             x              -
          longitude                             -                 -             -              x
     Auxiliary coordinates:
          forecast_period                                     x                 -             -              -
          forecast_reference_time                             x                 -             -              -
          Experiment identifier                               -                 x             -              -
          Institution responsible for the forecast system     -                 x             -              -
          Method of production of the data                    -                 x             -              -
          realization                                         -                 x             -              -
     Scalar coordinates:
          height: 2.0 m
     Attributes:
          Comment: Data interpolated from original model grid into a regular grid. Data restrictions:...
          Generator: SeasPy v1.1
          unit_long: kelvin
          Title: ENSEMBLES project
          Created: Thu Apr 16 14:36:57 2009
          Conventions: CF-1.0
          data_type: float
          References: http://www.ecmwf.int/research/EU_projects/ENSEMBLES/index.html, http:/...
     Cell methods:
          mean: leadtime (interval 6 h)
>>> 

The cube summary should also clip the returned name of a coordinate, particularly those that do not have a standard_name and a long long_name.

lib/iris/fileformats/_pyke_rules/fc_rules_cf.krb line 700 issues a warning when encountering an invalid unit. The use of %r means that the user can get warnings like:

 UserWarning: Ignoring netCDF variable u'time' invalid units u'wibble'

When working with global fields it is occasionally useful to extract data across the longitude boundary, for example when attempting to extract the region 30W-30E. At the moment it is not straightforward to extract such a region in iris (for plotting or analysis), but Carwyn (cpelley) has provided a solution using the following code

import iris
import numpy as np

extb = iris.load_strict(fname)

longco = extb.coord('longitude')
if longco.is_monotonic():
    extb.data = np.roll(extb.data, len(longco.points)//2)
    longco.points -= 180.
    if longco.has_bounds():
        longco.bounds = longco.bounds - 180.
else:
    print 'ERROR: the points are not monotonic, this workaround will not work'
    sys.exit(0)

after which a suitably phrased constraint can be used to extract the data from the cube extb.

The numpy method to "roll" an array along an index could be directly implemented within the cube object allowing this task to be performed more easily. Such a method could also be very useful in preparing global fields to be plotted with the most important region centred.

The type of a scalar coordinate is not preserved for scalars:

>>> ml_10 = iris.Constraint(model_level_number=10)
>>> print type(iris.load_strict('/data/local/dataZoo/iris_resources/tests/PP/globClim1/dec_subset.pp', \
...                                                  'air_potential_temperature' & ml_10).coord('sigma'))
<class 'iris.coords.DimCoord'>

>>> print type(iris.load_strict('/data/local/dataZoo/iris_resources/tests/PP/globClim1/dec_subset.pp',  \
...                                                 'air_potential_temperature').extract(ml_10).coord('sigma'))
<class 'iris.coords.AuxCoord'>

Investigate what PR change caused the userguide/cube_statistics.rst doctest to fail in upstream/master and resolve.

git bisect may help here.

The work on CF-style coordinate systems (#65) highlighted a possible issue with spherical curl.

As a result of #65, the unit of the radius changed from "km" to "m".

1. Should the results in the test_contrived_spherical_curl1 and test_contrived_sphrical_curl2 have been divided by 1000?
2. Are the values of the decimal keyword sufficiently strict?

As far as I can tell, writing NetCDF files with Iris won’t let me specify that, for example, time is a “record dimension”. It seems that time in this case has to be a coordinate, just like latitude, longitude, etc. Is it possible to make time (or anything) a record dimension?

The record dimension of a NetCDF file has size unlimited. It also means that the files can be simply concatenated with ncrcat. If there are no record dimensions then ncrcat doesn’t know which dimension of the data to append to.

Iris automatically making the outermost dimension unlimited would be a good and useful first step. Ultimately though it would be useful to have some sort of override to specifiy which, if any, dimension is unlimited

Fix setup.py to install COPYING, COPYING.LESSER, CHANGES instead of LICENCE.

When setting a CoordSystem into a Coord, ensure the Coord's name and units are as expected by the CoordSystem.

Wait for #52 #65.

Remove Coord.sin(), Coord.cos(), and Coord._trig_method() from the Coord classes. This will require updating iris.analysis.calculus and various tests.