Code Monkey home page Code Monkey logo

netcdf4-python's Introduction

Python/numpy interface to the netCDF C library.

Build status PyPI package Anaconda-Server Badge DOI

News

For details on the latest updates, see the Changelog.

06/17/2024: Version 1.7.1 released. Fixes for wheels, no code changes.

06/13/2024: Version 1.7.0 released. Add support for complex numbers via auto_complex keyword to Dataset (PR #1295)

10/20/2023: Version 1.6.5 released. Fix for issue #1271 (mask ignored if bool MA assinged to uint8 var), support for python 3.12 (removal of python 3.7 support), more informative error messages.

6/4/2023: Version 1.6.4 released. Now requires certifi to locate SSL certificates - this allows OpenDAP https URLs to work with linux wheels (issue #1246).

3/3/2023: Version 1.6.3 released.

11/15/2022: Version 1.6.2 released. Fix for compilation with netcdf-c < 4.9.0 (issue #1209).
Slicing multi-dimensional variables with an all False boolean index array now returns an empty numpy array (instead of raising an exception - issue #1197).

09/18/2022: Version 1.6.1 released. GIL now released for all C lib calls, set_alignment and get_alignment module functions added to modify/retrieve HDF5 data alignment properties. Added Dataset methods to query availability of optional compression filters.

06/24/2022: Version 1.6.0 released. Support for quantization (bit-grooming and bit-rounding) functionality in netcdf-c 4.9.0 which can dramatically improve compression. Dataset.createVariable now accepts dimension instances (instead of just dimension names). 'compression' kwarg added to Dataset.createVariable to support szip as well as new compression algorithms available in netcdf-c 4.9.0 through compression plugins (such as zstd, bzip2 and blosc). Working arm64 wheels for Apple M1 Silicon now available on pypi.

10/31/2021: Version 1.5.8 released. Fix Enum bug, add binary wheels for aarch64 and python 3.10.

6/22/2021: Version 1.5.7 released. Fixed OverflowError on Windows when reading data with dimension sizes greater than 2**32-1. Masked arrays no longer returned for vlens.

2/15/2021: Version 1.5.6 released. Added Dataset.fromcdl and Dataset.tocdl, which require ncdump and ncgen utilities to be in $PATH. Removed python 2.7 support.

12/20/2020: Version 1.5.5.1 released. Updated binary wheels for OSX and linux that link latest netcdf-c and hdf5 libs.

12/01/2020: Version 1.5.5 released. Update license wording to be consistent with MIT license.

07/23/2020: Version 1.5.4 released. Now requires numpy >= 1.9.

10/27/2019: Version 1.5.3 released. Fix for issue #972, plus binary wheels for python 3.8.

09/03/2019: Version 1.5.2 released. Bugfixes, no new features.

05/06/2019: Version 1.5.1.2 released. Fixes another slicing regression (issue #922)) introduced in the 1.5.1 release.

05/02/2019: Version 1.5.1.1 released. Fixes incorrect __version__ module variable in 1.5.1 release, plus a slicing bug (issue #919)).

04/30/2019: Version 1.5.1 released. Bugfixes, no new features.

04/02/2019: Version 1.5.0.1 released. Binary wheels for macos x and linux rebuilt with netcdf-c 4.6.3 (instead of 4.4.1.1). Added read-shared capability for faster reads of NETCDF3 files (mode='rs').

03/24/2019: Version 1.5.0 released. Parallel IO support for classic file formats added using the pnetcdf library (contribution from Lars Pastewka, pull request #897).

03/08/2019: Version 1.4.3.2 released. Include missing membuf.pyx file in source tarball. No need to update if you installed 1.4.3.1 from a binary wheel.

03/07/2019: Version 1.4.3.1 released. Fixes bug in implementation of NETCDF4_CLASSIC parallel IO support in 1.4.3.

03/05/2019: Version 1.4.3 released. Issues with netcdf-c 4.6.2 fixed (including broken parallel IO). set_ncstring_attrs() method added, memoryview buffer now returned when an in-memory Dataset is closed.

10/26/2018: Version 1.4.2 released. Minor bugfixes, added Variable.get_dims() method and master_file kwarg for MFDataset.__init__.

08/10/2018: Version 1.4.1 released. The old slicing behavior (numpy array returned unless missing values are present, otherwise masked array returned) is re-enabled via set_always_mask(False).

05/11/2018: Version 1.4.0 released. The netcdftime package is no longer included, it is now a separate package dependency. In addition to several bug fixes, there are a few important changes to the default behaviour to note:

  • Slicing a netCDF variable will now always return masked array by default, even if there are no masked values. The result depended on the slice before, which was too surprising. If auto-masking is turned off (with set_auto_mask(False)) a numpy array will always be returned.
  • _FillValue is no longer treated as a valid_min/valid_max. This was too surprising, despite the fact the thet netcdf docs attribute best practices suggests that clients should to this if valid_min, valid_max and valid_range are not set.
  • Changed behavior of string attributes so that nc.stringatt = ['foo','bar'] produces an vlen string array attribute in NETCDF4, instead of concatenating into a single string (foobar). In NETCDF3/NETCDF4_CLASSIC, an IOError is now raised, instead of writing foobar.
  • Retrieved compound-type variable data now returned with character array elements converted to numpy strings (issue #773). Works for assignment also. Can be disabled using set_auto_chartostring(False). Numpy structured array dtypes with 'SN' string subtypes can now be used to define netcdf compound types in createCompoundType (they get converted to ('S1',N) character array types automatically).
  • valid_min, valid_max, _FillValue and missing_value are now treated as unsigned integers if _Unsigned variable attribute is set (to mimic behaviour of netcdf-java). Conversion to unsigned type now occurs before masking and scale/offset operation (issue #794)

11/01/2017: Version 1.3.1 released. Parallel IO support with MPI! Requires that netcdf-c and hdf5 be built with MPI support, and mpi4py. To open a file for parallel access in a program running in an MPI environment using mpi4py, just use parallel=True when creating the Dataset instance. See examples/mpi_example.py for a demonstration. For more info, see the tutorial section.

9/25/2017: Version 1.3.0 released. Bug fixes for netcdftime and optimizations for reading strided slices. encoding kwarg added to Dataset.__init__ and Dataset.filepath to deal with oddball encodings in filename paths (sys.getfilesystemencoding() is used by default to determine encoding). Make sure numpy datatypes used to define CompoundTypes have isalignedstruct flag set to avoid segfaults - which required bumping the minimum required numpy from 1.7.0 to 1.9.0. In cases where missing_value/valid_min/valid_max/_FillValue cannot be safely cast to the variable's dtype, they are no longer be used to automatically mask the data and a warning message is issued.

6/10/2017: Version 1.2.9 released. Fixes for auto-scaling and masking when _Unsigned and/or valid_min, valid_max attributes present. setup.py updated so that pip install works if cython not installed. Now requires setuptools version 18.0 or greater.

6/1/2017: Version 1.2.8 released. From Changelog:

  • recognize _Unsigned attribute used by netcdf-java to designate unsigned integer data stored with a signed integer type in netcdf-3 issue #656.
  • add Dataset init memory parameter to allow loading a file from memory pull request #652, issue #406 and issue #295.
  • fix for negative times in num2date issue #659.
  • fix for failing tests in numpy 1.13 due to changes in numpy.ma issue #662.
  • Checking for _Encoding attribute for NC_STRING variables, otherwise use 'utf-8'. 'utf-8' is used everywhere else, 'default_encoding' global module variable is no longer used. getncattr method now takes optional kwarg 'encoding' (default 'utf-8') so encoding of attributes can be specified if desired. If _Encoding is specified for an NC_CHAR ('S1') variable, the chartostring utility function is used to convert the array of characters to an array of strings with one less dimension (the last dimension is interpreted as the length of each string) when reading the data. When writing the data, stringtochar is used to convert a numpy array of fixed length strings to an array of characters with one more dimension. chartostring and stringtochar now also have an 'encoding' kwarg. Automatic conversion to/from character to string arrays can be turned off via a new set_auto_chartostring Dataset and Variable method (default is True). Addresses issue #654
  • Cython >= 0.19 now required, _netCDF4.c and _netcdftime.c removed from repository.

1/8/2017: Version 1.2.7 released. Python 3.6 compatibility, and fix for vector missing_values.

12/10/2016: Version 1.2.6 released. Bug fixes for Enum data type, and _FillValue/missing_value usage when data is stored in non-native endian format. Add get_variables_by_attributes to MFDataset. Support for python 2.6 removed.

12/1/2016: Version 1.2.5 released. See the Changelog for changes.

4/15/2016: Version 1.2.4 released. Bugs in handling of variables with specified non-native "endian-ness" (byte-order) fixed ([issue #554] (#554)). Build instructions updated and warning issued to deal with potential backwards incompatibility introduced when using HDF5 1.10.x (see Unidata/netcdf-c/issue#250).

3/10/2016: Version 1.2.3 released. Various bug fixes. All text attributes in NETCDF4 formatted files are now written as type NC_CHAR, unless they contain unicode characters that cannot be encoded in ascii, in which case they are written as NC_STRING. Previously, all unicode strings were written as NC_STRING. This change preserves compatibility with clients, like Matlab, that can't deal with NC_STRING attributes. A setncattr_string method was added to force attributes to be written as NC_STRING.

1/1/2016: Version 1.2.2 released. Mostly bugfixes, but with two new features.

  • support for the new NETCDF3_64BIT_DATA format introduced in netcdf-c 4.4.0. Similar to NETCDF3_64BIT (now NETCDF3_64BIT_OFFSET), but includes 64 bit dimension sizes (> 2 billion), plus unsigned and 64 bit integer data types. Uses the classic (netcdf-3) data model, and does not use HDF5 as the underlying storage format.

  • Dimension objects now have a size attribute, which is the current length of the dimension (same as invoking len on the Dimension instance).

The minimum required python version has now been increased from 2.5 to 2.6.

10/15/2015: Version 1.2.1 released. Adds the ability to slice Variables with unsorted integer sequences, and integer sequences with duplicates.

9/23/2015: Version 1.2.0 released. New features:

7/28/2015: Version 1.1.9 bugfix release.

5/14/2015: Version 1.1.8 released. Unix-like paths can now be used in createVariable and createGroup.

    v = nc.createVariable('/path/to/var1', ('xdim', 'ydim'), float)

will create a variable named 'var1', while also creating the groups 'path' and 'path/to' if they do not already exist.

Similarly,

    g = nc.createGroup('/path/to') 

now acts like mkdir -p in unix, creating groups 'path' and '/path/to', if they don't already exist. Users who relied on nc.createGroup(groupname) failing when the group already exists will have to modify their code, since nc.createGroup will now return the existing group instance. Dataset.__getitem__ was also added. nc['/path/to'] now returns a group instance, and nc['/path/to/var1'] now returns a variable instance.

3/19/2015: Version 1.1.7 released. Global Interpreter Lock (GIL) now released when extension module calls C library for read operations. This speeds up concurrent reads when using threads. Users who wish to use netcdf4-python inside threads should read http://www.hdfgroup.org/hdf5-quest.html#gconc regarding thread-safety in the HDF5 C library. Fixes to setup.py now ensure that pip install netCDF4 with export USE_NCCONFIG=0 will use environment variables to find paths to libraries and include files, instead of relying exclusively on the nc-config utility.

Installation

The easiest way to install is through pip:

pip install netCDF4

or, if you are a user of the Conda package manager,

conda install -c conda-forge netCDF4

Development installation

  • Clone GitHub repository (git clone https://github.com/Unidata/netcdf4-python.git)

  • Make sure numpy and Cython are installed and you have Python 3.8 or newer.

  • Make sure HDF5 and netcdf-4 are installed, and the nc-config utility is in your Unix PATH.

  • Run python setup.py build, then pip install -e ..

  • To run all the tests, execute cd test && python run_all.py.

Documentation

See the online docs for more details.

Usage

Sample iPython notebooks available in the examples directory on reading and writing netCDF data with Python.

netcdf4-python's People

Contributors

barronh avatar bekozi avatar ckhroulev avatar cmarquardt avatar cpaulik avatar davidhassell avatar dependabot[bot] avatar docotak avatar fredrik-1 avatar headtr1ck avatar hetland avatar hmaarrfk avatar huard avatar jjhelmus avatar jswhit avatar jswhit2 avatar matthew-brett avatar maxbo avatar mcuntz avatar milos-korenciak avatar mwtoews avatar ocefpaf avatar pastewka avatar sebastic avatar shoyer avatar sroet avatar takluyver avatar thehesiod avatar timoroth avatar zedthree avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

netcdf4-python's Issues

python run_all.py segfaults Mac OS X 10.5.6

From [email protected] on March 03, 2009 07:25:07

What steps will reproduce the problem? 1. Follow documentation to install netCDF4-0.7.7 and all required packages (zlib, hdf5, netcdf-
4, all of which test correctly)

  1. Do rudimentray test:

hank:test tjs$ python
Python 2.5.1 ( r251 :54863, Jan 13 2009, 10:26:13)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import netCDF4

  1. cd to test directory of netCDF4 src distribution and python run_all.py What is the expected output? What do you see instead? Expected: Not a seg fault, I presume.

hank:test tjs$ python run_all.py
..E.Segmentation fault

This happens in about 2 seconds

hank:test tjs$ python --version
Python 2.5.1
hank:test tjs$ python
Python 2.5.1 ( r251 :54863, Jan 13 2009, 10:26:13)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

What version of the product are you using? On what operating system? netCDF4-0.7.7

MacOS X 10.5.6 Please provide any additional information below. quad-core mac pro

latest Xcode

hdf5-1.82

netcdf-4.0

zlib-1.2.3

ifort 11.059

gcc: i686-apple-darwin9-gcc-4.0.1

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=13

setup problem in Windows XP (netcdf3)

From [email protected] on September 26, 2008 20:31:16

What steps will reproduce the problem? - python setup-nc3.py install What is the expected output? What do you see instead? - Did not finish install, lengthy error output instead:
...
...
Found executable c:\mingw\bin\gcc.exe
Traceback (most recent call last):
File "setup-nc3.py", line 60, in
ext_modules = extensions)
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\core.py", line 176, in setup
return old_setup(**new_attr)
File "C:\Python25\lib\distutils\core.py", line 151, in set
up
dist.run_commands()
File "C:\Python25\lib\distutils\dist.py", line 974, in run
_commands
self.run_command(cmd)
File "C:\Python25\lib\distutils\dist.py", line 994, in run
command
cmd_obj.run()
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\command\install.py", line 16, in
run
r = old_install.run(self)
File "C:\Python25\lib\distutils\command\install.py", line
506, in run
self.run_command('build')
File "C:\Python25\lib\distutils\cmd.py", line 333, in run

command
self.distribution.run_command(command)
File "C:\Python25\lib\distutils\dist.py", line 994, in run
command
cmd_obj.run()
File "C:\Python25\lib\distutils\command\build.py", line 11
2, in run
self.run_command(cmd_name)
File "C:\Python25\lib\distutils\cmd.py", line 333, in run

command
self.distribution.run_command(command)
File "C:\Python25\lib\distutils\dist.py", line 994, in run
_command
cmd_obj.run()
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\command\build_ext.py", line 217,
in run
self.build_extensions()
File "C:\Python25\lib\distutils\command\build_ext.py", lin
e 416, in build_extensions
self.build_extension(ext)
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\command\build_ext.py", line 400,
in build_extension
build_temp=self.build_temp,**kws)
File "C:\Python25\lib\distutils\ccompiler.py", line 845, i
n link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\mingw32ccompiler.py", line 153, i
n link
func(*args[:func.im_func.func_code.co_argcount])
File "C:\Python25\lib\distutils\unixccompiler.py", line 21
4, in link
libraries)
File "C:\Python25\lib\site-packages\numpy-1.0.4.0002-py2.5
-win32.egg\numpy\distutils\ccompiler.py", line 377, in gen_l
ib_options
runtime_library_dirs, libraries)
File "C:\Python25\lib\distutils\ccompiler.py", line 1242,
in gen_lib_options
opt = compiler.runtime_library_dir_option (dir)
File "C:\Python25\lib\distutils\unixccompiler.py", line 27
8, in runtime_library_dir_option
compiler = os.path.basename(sysconfig.get_config_var("CC
"))
File "C:\Python25\lib\ntpath.py", line 200, in basename
return split(p)[1]
File "C:\Python25\lib\ntpath.py", line 164, in split
d, p = splitdrive(p)
File "C:\Python25\lib\ntpath.py", line 119, in splitdrive
if p[1:2] == ':':
TypeError: 'NoneType' object is unsubscriptable What version of the product are you using? On what operating system? - version 0.7.6

  • Windows XP
  • Python 2.5 (numpy1.0.4 installed and working)
  • NetCDF 3.6.2 Please provide any additional information below. - NETCDF3_DIR environment variable is set
  • import scientific.IO.NetCDF does work
  • Cygwin (but not Cygwin Python) installed. It is not being used here, but
    it has created issues in the past.
  • Using this GCC binary: http://www.develer.com/oss/GccWinBinaries - The first line of output is a warning:
    C:\Python25\lib\distutils\dist.py:263: UserWarning: Unknown
    distribution option: 'summary'

Not sure where the problem might lie. Any suggestions would be appreciated!

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=8

File saved as netcdf-4 despite choosing format='NETCDF3_CLASSIC'

From [email protected] on February 19, 2010 07:40:14

What steps will reproduce the problem? 1. Store the attached file AL20_ROBS_1979 and the script Convert.py in the
same directory.
2. From ipython shell, '%run Convert'
3. Check the format of the input and output files using 'ncdump -k
AL20_ROBS_1979' and 'ncdump -k AL20_ROBS_1979-processed.nc' What is the expected output? What do you see instead? I expect the input and output files to have the same format, namely
NETCDF3_CLASSIC, since the output file was created with an explicit
'format=' parameter to match the input file format. However, the output
file is a netcdf-4 file. The same thing happens if on line 8 of Convert.py
I explicitly specify format='NETCDF3_CLASSIC'. The output file is still
netcdf-4. What version of the product are you using? On what operating system? netcdf-4.1-rc1
netCDF4-0.9
64-bit ubuntu linux

Attachment: AL20_ROBS_1979 Convert.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=40

Import Error using python2.4 - easy fix provided

From [email protected] on March 17, 2010 16:34:54

What steps will reproduce the problem? 1. Install netCDF4-0.9-py2.4-linux-i686.egg using easy_install
2. Try importing netCDF4 (in python2.4) What is the expected output? What do you see instead? File
"/var/www/django/dgw-wsgi/python_modules/lib/python2.4/site-packages/netCDF4-0.9-py2.4-linux-i686.egg/netCDF4_utils.py",
line 424
key = reversed(self).next() if last else iter(self).next()
^
SyntaxError: invalid syntax What version of the product are you using? On what operating system? RedHat Linux, python2.4, netCDF4-0.9-py2.4-linux-i686 Please provide any additional information below. Inline conditional statements aren't supported in python 2.4.

Simply changing line 424 to:
if last: key = reversed(self).next()
else: key = iter(self).next()
fixed the problem.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=42

unexpected truncation of data ?

From [email protected] on March 29, 2010 15:35:59

What steps will reproduce the problem? 1. Execute the attached test.py ; a foo.nc netCDF is created
2. ncdump foo.nc What is the expected output? What do you see instead? v is a scalar variable of type short with scale_factor=0.1 and add_offset=0.
The attribute maskandscale is set to True (by default since netCDF4 0.9)

Documentation states that :

If maskandscale is set to True, and the variable has a scale_factor and an
add_offset attribute, then data read from that variable is unpacked using:
data = self.scale_factor*data + self.add_offset

When data is written to a variable it is packed using:
data = (data - self.add_offset)/self.scale_factor

I was expecting that assigning 1.1 to the v data results in a short value
of 11 in netCDF , not 10.
Since v data is printed as 1.0 and not 1.1 after assignment, it looks like
data value is truncated before being packed. What version of the product are you using? On what operating system? netcdf4 0.9
netcdf-4.1-beta2
hdf5-1.8.4-patch1
numpy-1.3.0
Python-2.6.4

Linux Centos 5.2 Please provide any additional information below. I'm new at Python and netcdf4-python. Am I missing something.. ? My
apologies if I am.

Attachment: test.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=46

Missing dimension in NetCDF4 file written using netCDF4-0.9

From [email protected] on February 13, 2010 10:18:52

What steps will reproduce the problem? 1. From the python prompt, run the attached script. Then issue
x = TestNCDF4()
x.WriteFile()

  1. After that, 'ncdump test.nc4' does not show the 'time' dimension.
    Instead, it shows 'latitude' twice, and the arrays 'conc' and 'surface
    pressure' show dimensions of latitude x latitude x longitude instead of
    time x latitude x longitude
  2. The resulting file can also be viewed using netCDF4 as follows:

from netCDF4 import Dataset
fid = Dataset('test.nc4','r')
fid.dimensions

This will show only two dimensions ('latitude' and 'longitude') and not three. What is the expected output? What do you see instead? I expect to see three dimensions, 'time', 'latitude' and 'longitude' in the
written file. Instead, I only see two. What version of the product are you using? On what operating system? netcdf-4.1
netCDF4-0.9
Ubuntu linux x86_64

Thanks,

-Sourish

Attachment: TestNCDF4.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=36

Order of variables contained in a netcdf file

From [email protected] on February 27, 2009 08:45:29

To operate on the entire data in huge netcdf-files efficiently I'd need to
know the order in which the variables are stored in the files I read. Is
there any way to obtain this informtaion within the netcdf4-python module?
Didn't find anything in the documentation...

(I hope this is the right place to raise this issue, I didn't find a user
mailing list or s.thing similar...)

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=12

File with unlimited dimension much bigger than file with fixed dimension of same length

From [email protected] on April 06, 2010 09:40:16

What steps will reproduce the problem? 1. Run the attached script ("%run TestNCDF" within ipython)
2. Do "T = TestNCDF4(); T.Test()"
3. Outside python, do "ls -lh Test_*.nc4" What is the expected output? What do you see instead? Since the two files contain the same data, I would expect them to be
roughly the same size. Instead, one is 7.7M and the other is 44M. This is
just a test case. The difference is more dramatic for my real data, where
the file with unlimited dimensions is 403M and the one with fixed
dimensions is 37M. Also, the larger file (403M) takes a very, very long
time (> 5 min) to read with

fid = Dataset('Real_data_unlimited.nc4','r')
pressure = fid.variables['Pressure'][:]

whereas the file with fixed dimensions is read in less than a second. What version of the product are you using? On what operating system? netcdf library 4.0.1
netCDF4 0.9
linux amd64
python 2.6.4

Attachment: TestNCDF.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=48

nc4tonc3 chokes on least_significant_digit attribute

From david.huard on April 06, 2009 08:15:20

I just needed an utility to convert netcdf4 to netcdf3 and noticed you
added it a couple of days ago !

I tried it and noticed it got

Traceback (most recent call last):
File "/usr/local/bin/nc4tonc3", line 114, in
nc3tonc4(filename4,filename3,clobber=overwritefile,quiet=quiet)
File "/usr/local/bin/nc4tonc3", line 37, in nc3tonc4
setattr(var,attname,getattr(ncvar,attname))
File "netCDF4.pyx", line 1804, in netCDF4.Variable.setattr
(netCDF4.c:12598)
File "netCDF4.pyx", line 714, in netCDF4._set_att (netCDF4.c:3020)
AttributeError: NetCDF: Not a valid data type or _FillValue type mismatch

So I added those lines to nc4tonc3.py the script see what happened

    for attname in ncvar.ncattrs():
        try:            
            setattr(var,attname,getattr(ncvar,attname))
        except:
            print 'Could not set attribute', attname, ':',

getattr(ncvar, attname)

and I got

huardda@angus:~/forcing/wind/80$ nc4tonc3 geowinds.nc geowinds.nc3
copying global attributes ..
copying dimensions ..
copying variable lat
copying variable uwnd
Could not set attribute least_significant_digit : 6

The uwnd variable is created using

That is, my netcdf variable is generated with

uwnd = root.createVariable('uwnd','f4',('time','y','x',),
zlib=True,least_significant_digit=6)

If you can't guess what the problem is right away, I'll hunt it later on.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=15

MFDataset docstring

From david.huard on July 31, 2008 14:20:20

Here is a patch adding the 'exclude' keyword to the MFDataset docstring. I
needed this feature and almost started implementing it when I noticed it
was already there !

Index: utils.pyx

--- utils.pyx ( revision 547 )
+++ utils.pyx (working copy)
@@ -130,9 +130,9 @@

class MFDataset(Dataset):
"""
-MFDataset(self, files, check=False)
+MFDataset(self, files, check=False, exclude=[])

-class for reading multi-file netCDF Datasets, making variables
+Class for reading multi-file netCDF Datasets, making variables
spanning multiple files appear as if they were in one file.

Datasets must be in C{NETCDF4_CLASSIC, NETCDF3_CLASSIC or NETCDF3_64BIT}
@@ -170,7 +170,7 @@

Usage:

-nc = MFDataset(files, check=False)
+nc = MFDataset(files, check=False, exclude=[])

@param files: either a sequence of netCDF files or a string with a
wildcard (converted to a sorted list of files using glob) The first file

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=6

NetCDF4 interface compile error

From [email protected] on February 02, 2008 10:44:30

What steps will reproduce the problem? 1. Installing netcdf-4.0-2008013102 on 32- and -64-bit Linux platforms with
the recommended gfortran 4.1.2 flags and libraries, e.g. the following HDF
library.

configure --prefix=/opt/gfortran/netcdf-4.0-2008013102 --enable-netcdf-4
--with-hdf5=/opt/gfortran/hdf5-1.8.0-rc2 --with-zlib=/usr

  1. Installing hdf5-1.8.0-rc2 on 32- and 64-bit Linux platforms with the
    recommended gfortran 4.1.2 flags.

configure --prefix=/opt/gfortran/hdf5-1.8.0-rc2 --enable-fortran
--enable-cxx --with-zlib=/usr

  1. Attempting to build netCDF4-0.7.3.1 What is the expected output? What do you see instead? The error message, consistent across 4 platforms thus far:

compile options: '-I/opt/gfortran/netcdf-4.0-2008013102/include
-I/opt/gfortran/hdf5-1.8.0-rc2/include
-I/usr/lib/python2.5/site-packages/numpy/core/include
-I/usr/include/python2.5 -c'
gcc: netCDF4.c
netCDF4.c: In function โ€˜pyx_pf_7netCDF4_8Variable___initโ€™:
netCDF4.c:7882: error: โ€˜NC_CHUNK_SUBโ€™ undeclared (first use in this function)
netCDF4.c:7882: error: (Each undeclared identifier is reported only once
netCDF4.c:7882: error: for each function it appears in.)
netCDF4.c:7905: error: โ€˜NC_CHUNK_SEQโ€™ undeclared (first use in this function)
netCDF4.c:8088: warning: passing argument 3 of โ€˜nc_def_var_chunkingโ€™ makes
integer from pointer without a cast
netCDF4.c:8088: error: too many arguments to function โ€˜nc_def_var_chunkingโ€™
netCDF4.c: In function โ€˜pyx_pf_7netCDF4_8Variable_chunkingโ€™:
netCDF4.c:9836: error: too many arguments to function โ€˜nc_inq_var_chunkingโ€™
netCDF4.c:9963: error: โ€˜NC_CHUNK_SEQโ€™ undeclared (first use in this function)
netCDF4.c:9986: error: โ€˜NC_CHUNK_SUBโ€™ undeclared (first use in this function)
netCDF4.c:10009: error: โ€˜NC_CHUNK_SIZESโ€™ undeclared (first use in this
function)
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_getlibversionโ€™:
netCDF4.c:14529: warning: assignment discards qualifiers from pointer
target type
netCDF4.c: In function โ€˜initnetCDF4โ€™:
netCDF4.c:20679: warning: integer constant is too large for โ€˜longโ€™ type
netCDF4.c:20690:62: warning: integer constant is so large that it is unsigned
netCDF4.c:20690: warning: this decimal constant is unsigned only in ISO C90
netCDF4.c:20690: warning: integer constant is too large for โ€˜longโ€™ type
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_8Variable___init
โ€™:
netCDF4.c:7882: error: โ€˜NC_CHUNK_SUBโ€™ undeclared (first use in this function)
netCDF4.c:7882: error: (Each undeclared identifier is reported only once
netCDF4.c:7882: error: for each function it appears in.)
netCDF4.c:7905: error: โ€˜NC_CHUNK_SEQโ€™ undeclared (first use in this function)
netCDF4.c:8088: warning: passing argument 3 of โ€˜nc_def_var_chunkingโ€™ makes
integer from pointer without a cast
netCDF4.c:8088: error: too many arguments to function โ€˜nc_def_var_chunkingโ€™
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_8Variable_chunkingโ€™:
netCDF4.c:9836: error: too many arguments to function โ€˜nc_inq_var_chunkingโ€™
netCDF4.c:9963: error: โ€˜NC_CHUNK_SEQโ€™ undeclared (first use in this function)
netCDF4.c:9986: error: โ€˜NC_CHUNK_SUBโ€™ undeclared (first use in this function)
netCDF4.c:10009: error: โ€˜NC_CHUNK_SIZESโ€™ undeclared (first use in this
function)
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_getlibversionโ€™:
netCDF4.c:14529: warning: assignment discards qualifiers from pointer
target type
netCDF4.c: In function โ€˜initnetCDF4โ€™:
netCDF4.c:20679: warning: integer constant is too large for โ€˜longโ€™ type
netCDF4.c:20690:62: warning: integer constant is so large that it is unsigned
netCDF4.c:20690: warning: this decimal constant is unsigned only in ISO C90
netCDF4.c:20690: warning: integer constant is too large for โ€˜longโ€™ type
error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC
-I/opt/gfortran/netcdf-4.0-2008013102/include
-I/opt/gfortran/hdf5-1.8.0-rc2/include
-I/usr/lib/python2.5/site-packages/numpy/core/include
-I/usr/include/python2.5 -c netCDF4.c -o
build/temp.linux-i686-2.5/netCDF4.o" failed with exit status 1 What version of the product are you using? On what operating system? Please provide any additional information below. This version of NetCDF4 will not compile successfully with the
--enable-netcdf-4 flag against versions of HDF5 older than this one. Russ
Rew tells me that's due to the final change in the HDF5 API before the
release of HDF5 1.8.0.

On the other hand, the NetCDF3 version works fine with NetCDF 3.6.2.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=1

Missing NC_SHARE capability

From [email protected] on May 21, 2009 08:30:46

Dataset mode accepts "r", "r+", "a", and "w", but does not allow for the
shared modifier (e.g. "ws"). The NetCDF buffering is optimized for
sequentially adding along the unlimited dimension, but this is not always
practical. When not writing along the unlimited dimension, buffered
writing is much slower than unbuffered writing.

I have attached an example usage that would dramatically improve
performance by the "ws" capability. In the meantime, I recompile with
NC_WRITE|NC_SHARE as necessary.

--Example: duplicate a file

d1 = SomeOtherNetCDFLikeObject(path1,'r')
d2 = Dataset(path2,'w')
for dn, dv in d1.dimensions.iteritems():
d2.createDimension(dn, dv)

for vn, vv in d1.variables.iteritems():
nv = d2.createVariable(vn, vv.typecode(), vv.dimensions)
for pn in ncattrs(vv):
setattr(nv, pn, getattr(vv, pn))

nv[:] = vv[:]

d2.sync()

-- end example

Thanks for an awesome product.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=18

AttributeError: 'module' object has no attribute '_StartCountStride' (Snow Leopard)

From [email protected] on December 10, 2009 09:42:28

What steps will reproduce the problem? 1. Install netcdf-4.0 from source with the following configure options

./configure --prefix=/usr/local --enable-shared --disable-f77
--enable-static --with-hdf5=/usr/local --with-zlib=/opt/local
--with-szlib=/opt/local --enable-netcdf-4 CC=gcc CFLAGS='-arch i386'
LDFLAGS='-arch i386' FCFLAGS='-arch i386' CXXFLAGS='-arch i386'

HDF5 was installed from source, zlib and szlib were installed from Macports.

  1. Install netCDF4-0.8.2 (python module for netcdf4) after setting proper
    SZIP_DIR, HDF5_DIR and NETCDF4_DIR, using the command

python setup.py build
sudo python setup.py install

  1. Try to import netCDF4 from python (ipython shell):

[1]: import netCDF4 What is the expected output? What do you see instead? I expect the module to be imported without any error(s). Instead, I see the
following:

[1]: import netCDF4

AttributeError Traceback (most recent call last)

/Users/basu/ResearchCode/Test IO/ in ()

/Users/basu/ResearchCode/Test IO/netCDF4.pyx in netCDF4 (netCDF4.c:30879)()

AttributeError: 'module' object has no attribute '_StartCountStride' What version of the product are you using? On what operating system? netCDF4-0.8.2 on Mac OSX 10.6 (snow leopard) Please provide any additional information below. $ gcc -v
Using built-in specs.
Target: i686-apple-darwin10
Configured with: /var/tmp/gcc/gcc-5646.1~2/src/configure --disable-checking
--enable-werror --prefix=/usr --mandir=/share/man
--enable-languages=c,objc,c++,obj-c++
--program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib
--build=i686-apple-darwin10 --with-gxx-include-dir=/include/c++/4.2.1
--program-prefix=i686-apple-darwin10- --host=x86_64-apple-darwin10
--target=i686-apple-darwin10
Thread model: posix
gcc version 4.2.1 (Apple Inc. build 5646) (dot 1)

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=32

Attributes clashing with reserved words are impossible to write

From [email protected] on October 09, 2009 09:36:50

This is more a feature request than a bug as it's a problem all
Python->NetCDF bindings suffer. Apologies if this isn't the correct forum.

Attributes clashing with netCDF4 object properties and methods are
unsettable and must be read with getattr (as far as I can see). This
makes writing generic code on top of netCDF4 difficult. I'm sure it's only
a matter of time before I am asked to set a global attribute of
"file_format", "parent", "groups" or the like. What steps will reproduce the problem? $ ncdump test2.nc
netcdf test2 {

// global attributes:
:file_format = "my_format" ;
}
$ python

from netCDF4 import Dataset
d = Dataset('test2.nc', 'a')

The attribute "file_format" is hidden from traditional attribute access

d.file_format
'NETCDF3_CLASSIC'

You can get at it with getattr

d.getattr('file_format')
'my_format'

but not set it

d.setattr('file_format', 'foo')
Traceback (most recent call last):
File "", line 1, in
File "netCDF4.pyx", line 1635, in netCDF4.Dataset.setattr
(netCDF4.c:8163)
AttributeError: 'file_format' is one of the reserved attributes ('_grpid',
'_grp', '_varid', 'groups', 'dimensions', 'variables', 'dtype',
'file_format', '_nunlimdim', 'path', 'parent', 'ndim', 'maskandscale',
'cmptypes', 'vltypes', '_isprimitive', '_isvlen', '_iscompound'), cannot rebind What version of the product are you using? On what operating system? Tested on v0.8.1, linux with netcdf-4.0.1-beta3. Please provide any additional information below. Mirroring NetCDF attributes as Python object attributes is clearly a big
win but I would request an unambiguous alternative method of
getting/setting attributes along the line of the Dataset.ncattrs() function
for listing attribute names. I can help with the coding if you wish.

Stephen.Pascoe at stfc ac uk

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=26

Dict might not be the right datatype for dimensions etc

From [email protected] on November 11, 2009 09:44:43

What steps will reproduce the problem? 1. Load a netCDF file.
2. move the data into an object of your own for processing etc
3. make a new netCDF file and fill it with above data

See attached files: run dict_bug.py, it produces dict_bug_new.nc from
dict_bug.nc. But order in dict_bug.nc and dict_bug_out.nc is not the same. What is the expected output? What do you see instead? The order of dimensions, variables and attributes are changed, due to the
use of dict data structure. It would be nice to be able to keep the order
or choose it. For example, ncview displays coordinates in the way they
appear, thus having first X then Y would be good. What version of the product are you using? On what operating system? netCDF 0.7.3.1 on linux, Python 2.5 Please provide any additional information below. There are a number of ordered dict types floating around but not in the
standard library, sadly.

Attachment: dict_bug.py dict_bug.nc dict_bug_out.nc

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=28

ubuntu install

From [email protected] on July 07, 2008 15:09:51

I'm trying to install under Ubuntu 8.04. I can't find where the HDF5
directories are. I have non-python netCDF installed as well.

HDF5_DIR environment variable not set, checking some standard locations ..,
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
checking /home/drew ...
Traceback (most recent call last):
File "setup.py", line 42, in
raise ValueError('did not find HDF5 headers and libraries')
ValueError: did not find HDF5 headers and libraries

thanks

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=4

MFDataset Variable with lenth 1 dimension cannot be sliced

From [email protected] on January 04, 2010 11:04:45

To reproduce the problem follow the directions in the Documentation section
7 (Reading data from a multi-file netCDF dataset) with these modifications.

  1. add the creation of a y dimensions with a length of 1
  2. create the x variable with both the x and y dimensions

What is the expected output?
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]

What do you see instead?
Traceback (most recent call last):
...
File "utils.pyx", line 503, in netCDF4._Variable.getitem (netCDF4.c:28767)
ValueError: array dimensions are not compatible for copy

What version of the product are you using?
v 0.8.2

On what operating system?
Mac and Linux Please provide any additional information below. I think the source of the problem is the squeeze in the comments below.
/* "/Users/jsw/python/netcdf4-python/utils.pyx":503

  •         lstArr = numpy.concatenate(lstArr)
    
  •         if dat.dtype != data.dtype: data = data.astype(dat.dtype)
    
  •         data[tuple(ind)] = lstArr.squeeze()             # <<<<<<<<<<<<<<
    
  •     # Remove extra singleton dimensions.
    
    */

There are cases where there are intentional singleton dimensions (e.g. CMAQ
single layer files).

Thanks a ton!

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=34

date2index is buggy when used with dates outside the support.

From david.huard on December 02, 2009 14:11:05

Using a date before the first date in the time variable returns index -1.
Using a date after the last date in the time variable raises an error.

I've fixed this by replacing indices == -1 by 0, and those equal to N by
N-1, where N is the length of the time variable.

If this is OK with you, I'll commit those changes and the accompanying tests.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=30

mistake in doc of num2date and date2num

From [email protected] on April 19, 2010 07:32:00

The documentation for num2date says:

'Like the matplotlib num2date function, except that it allows for different
units and calendars. Behaves the same if units = 'days since 001-01-01
00:00:00' and calendar = 'proleptic_gregorian'.'
(same goes for date2num)

But matplotlib's behaviour is in fact reproduced by setting:
units = 'days since 001-12-31 00:00:00' calendar = 'proleptic_gregorian'

Note that in matplotlibs doc, the behaviour of their date conversion tool
is wrongly documented (bug report issiued).

CODE:

netCDF3.date2num(datetime.datetime(1,1,1,0,0) , units = 'days since
0001-01-01 00:00:00', calendar = 'proleptic_gregorian')
0.0
pylab.date2num(datetime.datetime(1,1,1,0,0))
1.0

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=49

os-x install failure

From [email protected] on November 13, 2008 14:25:07

What steps will reproduce the problem? 1. Trying to install from source fails. What is the expected output? What do you see instead? sudo python2.5 setup.py install

Traceback (most recent call last):
File "setup.py", line 72, in
raise ValueError('did not find netCDF version 4 headers and libs in %s'
% netCDF4_dir)
ValueError: did not find netCDF version 4 headers and libs in /opt/local/

I see that this fails when searching netcdf.h for 'nc_inq_compound', which
I can't find myself in the file. What version of the product are you using? On what operating system? os-x 10.4.x (mac pro, xeon)
Successfully installed hdf5-1.6.8 and netCDF4-0 from source.

I am using netcdf 4.0:
cdump -version
...
netcdf library version "4.0" of Nov 13 2008 14:05:21 Please provide any additional information below. Everything worked fine on my other machine running ubuntu 8.04; indeed I
used the same source for both.

Mike

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=9

problems with snow leopard install

From [email protected] on September 14, 2009 15:24:39

What steps will reproduce the problem? 1. Installed netcdf 4.0.1 both from source and using macports (same issue).
2. downloaded netCDF4-0.8.2
3. basic install What is the expected output? What do you see instead? some install warnings.

mbauer@boreas:~/Downloads/netCDF4-0.8.2/ > sudo python setup.py install
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/Pyrex/Compiler/Errors.py:17:
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
self.message = message

HDF5_DIR environment variable not set, checking some standard locations ..,
checking /Users/mbauer ...
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
HDF5 found in /opt/local

NETCDF4_DIR environment variable not set, checking some standard locations ..,
checking /Users/mbauer ...
checking /usr/local ...
netCDF4 found in /usr/local
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:267:
UserWarning: Unknown distribution option: 'summary'
warnings.warn(msg)
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler
options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands
--fcompiler options
running build_src
building py_modules sources
building extension "netCDF4" sources
running build_py
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
running scons
running build_scripts
running install_lib
running install_scripts
changing mode of
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/grib2nc4 to 755
changing mode of
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/nc3tonc4 to 755
changing mode of
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/nc4tonc3 to 755
running install_egg_info
Removing
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4-0.8.2-py2.6.egg-info
Writing
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4-0.8.2-py2.6.egg-info What version of the product are you using? On what operating system? python 2.6.2 (via macports)
netcdf 4.0.1 (tried both macports and direct install)
hdf5 1.8.3 (via macports) Please provide any additional information below. Attend to call netcdf4 in python give the following:

import netCDF4
ImportError:
dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4.so,
2): Symbol not found: _nc_close
Referenced from:
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4.so
Expected in: flat namespace
in
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4.so

I get the same for netcdf3 only as well.

Mike

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=24

python ./setup.py bdist_rpm fails

From [email protected] on May 19, 2010 09:16:46

What steps will reproduce the problem? 1. python ./setup.py bdist_rpm on redhat system fails What is the expected output? What do you see instead? rpms in dist directory What version of the product are you using? On what operating system? python 2.4 on SL5.4 Please provide any additional information below. There are some errors in the setup.py script. Furthermore on redhat type
systems pyo have to be distributed as well. The attached patch fixes the
problems

Attachment: netCDF4.patch

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=50

tst_compoundvar.py segfaults

From [email protected] on April 20, 2009 01:24:03

What steps will reproduce the problem? Hardware: Thinkpad T60

  1. Install ubuntu 9.04 rc via upgrade from 8.10
  2. Install hdf5 1.8.2 from tarball to /usr/local
  3. Install netcdf 4.0.1 from tarball to /usr/local
    "make test" runs all tests successfully.
  4. Install netCDF4 0.8 from tarball (standard setup.py build; sudo python
    setup.py install)

In directory "test", run_all.py hung the first time with an error that I
unfortunately lost by blowing away the terminal window too soon, but it was
something like "glibc detected: python: corrupted doubly-linked list". The
second time it segfaulted. Then I tried running the tests separately, and
found that the problem is in tst_compoundvar.py (segfault again).

(There is no urgency in solving this problem, as far as I am concerned.)

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=16

stop responding when opening netcdf

From [email protected] on February 20, 2010 18:58:29

What steps will reproduce the problem? 1. run the basic netcdf open tutorial
2. open for read only
3. print file_format What is the expected output? What do you see instead? print of the file_format but it causes python to stop responding What version of the product are you using? On what operating system? netCDF4-0.8.2.win32-py2.5.exe, Windows 7 Please provide any additional information below. The included netcdf file is from segmotion. Its been known to have problems
opening in other apps.

Attachment: 20060402-201025.netcdf

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=41

netCDF4.c(230) : error C2026: string too big, trailing characters truncated

From [email protected] on November 21, 2009 17:31:06

When building netCDF4-python, version 0.9 from svn, with Microsoft Visual
Studio 2008, the compiler throws error C2026 at line 230 of netCDF4.c.
After breaking up the string into smaller ones as follows, the build succeeds:

static char __pyx_mdoc[] = "\n"
"Introduction\n"
"============\n"
"\n"

...

"OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR\n"
"PERFORMANCE OF THIS SOFTWARE.";

32- and 64-bit binaries for Python 2.6 for Windows are available at
< http://www.lfd.uci.edu/~gohlke/pythonlibs/#netcdf4 >

Christoph

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=29

netCDF4 snapshot and HDF4 support

From [email protected] on September 11, 2009 14:38:24

Thought that this could be useful information.
I am pretty happy about it!

Compile netcdf-4.1-beta2-snapshot2009081900 with HDF4.

./configure --with-hdf5=/software/supports/ --enable-netcdf-4
--disable-f90 -enable-hdf4 --enable-hdf4-file-tests --enable-c-only
--with-hdf4=/software/supports/

Change setup.py to add some libraries and that's it!!!!!

--- setup.py 2009-08-18 14:52:22.000000000 -0400
+++ setupHDF4.py 2009-09-11 15:52:13.844746239 -0400
@@ -80,6 +80,14 @@
lib_dirs.append(os.path.join(szip_dir,'lib'))
inc_dirs.append(os.path.join(szip_dir,'include'))
libs.append('sz')
+# add hdf4 to link if desired.
+hdf4_dir = os.environ.get('HDF4_DIR')
+if hdf4_dir is not None:

  • lib_dirs.append(os.path.join(hdf4_dir,'lib'))
  • inc_dirs.append(os.path.join(hdf4_dir,'include'))
  • libs.append('mfhdf')
  • libs.append('df')
  • libs.append('jpeg')
    extensions =
    [Extension("netCDF4",["netCDF4.c"],libraries=libs,library_dirs=lib_dirs,include_dirs=inc_dirs,runtime_library_dirs=lib_dirs)]

Denis

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=23

dynamic module does not define init function

From [email protected] on July 29, 2009 08:25:26

What steps will reproduce the problem? 1. I have just installed python version 2.6.2 on a mac 10.5.7 and I have
successfully installed netcdf4 and hdf5 as 64 bit libraries. I have also
successfully installed netCDF4-python, but when i do:
from netCDF4 import Dataset What is the expected output? What do you see instead? I expect the module to load functionally with no print to stdout, but what
i get is this:
Python 2.6.2 ( r262 :71600, Jul 27 2009, 13:32:33)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import numpy
from netCDF4 import Dataset
Traceback (most recent call last):
File "", line 1, in
ImportError: dynamic module does not define init function (initnetCDF4) What version of the product are you using? On what operating system? netcdf4-python version 0.8.1 (downloaded today) Please provide any additional information below. I am sure I am doing something wrong in the compilation but i cannot figure
out what. My CFLAGS for netcdf4 and hdf5 are: -O3 -Wall -arch x86_64
-Df2cFortran -fno-common.

Thanks for your help.
Cheers, Trond

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=21

Enhancement: Prevent accidental deletion of existing datasets when opening with 'w' pramater.

From [email protected] on March 18, 2010 14:54:51

What steps will reproduce the problem? 1. Open an existing netcdf dataset, and without thinking type 'w' instead
of 'a' for the access mode.
2. Scream, and yell, and cry as you realize you just wiped out the whole
dataset. What is the expected output? What do you see instead? Expected: If I was so careless as to type 'w' instead of 'a' or 'r', then
I'd expect the system to warn me once it sees that a file with the same
name already exists. Please provide any additional information below. I understand that python's "open" method has the same behavior, and I'm not
particularly fond of that either. But IMO, netCDF4's behavior should be
more advanced so as to protect the data from accidental deletion.

I think a simple "file exists?" check, warning the user if true, would be
very helpful. The user could also provide an additional parameter
specifying what do do, like "wipeoutExisting", or "backupExisting", if data
already exists at the specified location.

Has this issue already been discussed somewhere? I'd be happy to
contribute a patch if the community agrees it's a good idea.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=43

Can't setup 0.7.4 on Windows XP machine

From [email protected] on June 20, 2008 07:52:09

What steps will reproduce the problem? 1. Install HDF 1.8.1
2. Install NETCDF-4.0 (stable release version)
3. Set HDF5_DIR, SZIP_DIR and NETCDF4_DIR env variables
4. Run setup.py install What is the expected output? What do you see instead? Expected to install, instead, errored:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\netCDF4-0.7.4\setup.py", line 46, in ?
raise ValueError('did not find HDF5 headers and libraries in %s' %
HDF5_dir)

ValueError: did not find HDF5 headers and libraries in C:\hdf5\hdf5lib\release What version of the product are you using? On what operating system? NETCDF4-0.7.4, Windows XP SP2 Please provide any additional information below. I checked the setup.py file and it seems to fail on the finding the version
number in the header file (include/H5pubconf.h). I checked the header file
located in the dir as specified above and did not find this line of code:

define H5_VERSION as needed by setup.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=2

installs fine (OS X) but fails on import

From [email protected] on January 20, 2009 14:06:03

I installed HDF5-1.8.2 and netcdf-4.0.1 from binaries (build to OS X on Intel). I installed
netCDF4-0.7.6 using setup.py install.

When I try to import netCDF4, I get this:

import netCDF4
Traceback (most recent call last):
File "", line 1, in
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-
packages/netCDF4.so, 2): Symbol not found: _H5P_CLS_FILE_ACCESS_g
Referenced from: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-
packages/netCDF4.so
Expected in: dynamic lookup

I'm a total noob. Any suggestions would be welcome.
Thanks.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=10

Run-time errors after (maybe-not-so) sucessful netcdf4-python install

From [email protected] on December 21, 2009 17:14:19

What steps will reproduce the problem? Various run-time errors occur, for many different cases -- for example, in
trying to write NETCDF3_CLASSIC format (the following code works without
the format='NETCDF3_CLASSIC' flag):

from netCDF4 import Dataset

rootgrp = Dataset('test.nc','w',format='NETCDF3_CLASSIC')
rootgrp.createDimension('time',None)
rootgrp.createDimension('lat', 73)
rootgrp.createDimension('lon', 144)

latitudes = rootgrp.createVariable('lat','f4',('lat',))
longitudes = rootgrp.createVariable('lon','f4',('lon',))
somevar = rootgrp.createVariable('somevar','f4',('time','lat','lon'))

import numpy
lats = numpy.arange(-90,91,2.5)
lons = numpy.arange(-180,180,2.5)
latitudes[:] = lats[:]
longitudes[:] = lons[:]
f = numpy.zeros((1,len(lats),len(lons)), dtype='f4')
somevar[:] = f[:]

rootgrp.close() What is the expected output? What do you see instead? Expected output, a netcdf file with variables lat,lon,somevar
Instead: Runtime error
Traceback (most recent call last):
File "test.py", line 18, in
somevar[:] = f[:]
File "netCDF4.pyx", line 2453, in netCDF4.Variable.setitem
(netCDF4.c:16793)
File "netCDF4.pyx", line 2577, in netCDF4.Variable._put (netCDF4.c:17771)
RuntimeError: Protocol error What version of the product are you using? On what operating system? Ubuntu Karmic
netcdf4 0.8.2 Please provide any additional information below. I built the HDF5 (1.8.4) and netCDF (4.0.1) following the instructions on
this site (ie with --enable-shared, etc)
Neither seemed to have any problems.

I don't get any errors on the import statements in Python and I have some
functionality -- ie the error in the above code comes about when trying to
assign values to the 3-d variable, everything else works okay.

However, I get all sorts of run-time errors when I do various things, so
clearly I have done something wrong :(

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=33

problems installing in snow leopard

From [email protected] on February 18, 2010 23:33:55

What steps will reproduce the problem? 1. Install hdf5 v1.8.4
2. Install netcdf4 v4.0 --
3. Then install netCDF4 basic install

Then I get something like this:

/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:266:
UserWarning: Unknown distribution option: 'summary'
warnings.warn(msg)
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler
options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands
--fcompiler options
running build_src
building py_modules sources
building extension "netCDF4" sources
running build_py
creating build/lib.macosx-10.3-fat-2.6
copying netCDF4_utils.py -> build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/netcdftime
copying netcdftime/init.py -> build/lib.macosx-10.3-fat-2.6/netcdftime
copying netcdftime/netcdftime.py -> build/lib.macosx-10.3-fat-2.6/netcdftime
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'netCDF4' extension
compiling C sources
C compiler: gcc-4.0 -arch ppc -arch i386 -isysroot
/Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -fno-common -dynamic
-DNDEBUG -g -O3

creating build/temp.macosx-10.3-fat-2.6
compile options: '-I/usr/local/netcdf4/include -I/usr/local/hdf5/include
-I/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include
-I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
gcc-4.0: netCDF4.c
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_getlibversionโ€™:
netCDF4.c:23991: warning: assignment discards qualifiers from pointer
target type
netCDF4.c: In function โ€˜__pyx_pf_7netCDF4_getlibversionโ€™:
netCDF4.c:23991: warning: assignment discards qualifiers from pointer
target type
gcc-4.0 -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g
-bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.6/netCDF4.o
-L/usr/local/netcdf4/lib -L/usr/local/hdf5/lib -L/usr/local/netcdf4/lib
-L/usr/local/hdf5/lib -lnetcdf -lhdf5_hl -lhdf5 -lz -o
build/lib.macosx-10.3-fat-2.6/netCDF4.so
ld: warning: in /usr/local/netcdf4/lib/libnetcdf.dylib, file is not of
required architecture
ld: warning: in /usr/local/hdf5/lib/libhdf5_hl.dylib, file is not of
required architecture
ld: warning: in /usr/local/hdf5/lib/libhdf5.dylib, file is not of required
architecture
ld: warning: in /usr/local/netcdf4/lib/libnetcdf.dylib, file is not of
required architecture
ld: warning: in /usr/local/hdf5/lib/libhdf5_hl.dylib, file is not of
required architecture
ld: warning: in /usr/local/hdf5/lib/libhdf5.dylib, file is not of required
architecture
running scons
running build_scripts
creating build/scripts.macosx-10.3-fat-2.6
copying and adjusting utils/nc3tonc4 -> build/scripts.macosx-10.3-fat-2.6
copying and adjusting utils/nc4tonc3 -> build/scripts.macosx-10.3-fat-2.6
copying and adjusting utils/grib2nc4 -> build/scripts.macosx-10.3-fat-2.6
changing mode of build/scripts.macosx-10.3-fat-2.6/nc3tonc4 from 644 to 755
changing mode of build/scripts.macosx-10.3-fat-2.6/nc4tonc3 from 644 to 755
changing mode of build/scripts.macosx-10.3-fat-2.6/grib2nc4 from 644 to 755
running install_lib
copying build/lib.macosx-10.3-fat-2.6/netCDF4.so ->
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
running install_scripts
copying build/scripts.macosx-10.3-fat-2.6/grib2nc4 ->
/Library/Frameworks/Python.framework/Versions/2.6/bin
copying build/scripts.macosx-10.3-fat-2.6/nc3tonc4 ->
/Library/Frameworks/Python.framework/Versions/2.6/bin
copying build/scripts.macosx-10.3-fat-2.6/nc4tonc3 ->
/Library/Frameworks/Python.framework/Versions/2.6/bin
changing mode of
/Library/Frameworks/Python.framework/Versions/2.6/bin/grib2nc4 to 755
changing mode of
/Library/Frameworks/Python.framework/Versions/2.6/bin/nc3tonc4 to 755
changing mode of
/Library/Frameworks/Python.framework/Versions/2.6/bin/nc4tonc3 to 755
running install_egg_info
Writing
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4-0.9-py2.6.egg-info

Then when I import netCDF4 in python i get

import netCDF4
Traceback (most recent call last):
File "", line 1, in
ImportError:
dlopen(/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4.so,
2): Symbol not found: _nc_close
Referenced from:
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/netCDF4.so
Expected in: dynamic lookup

I have seen this issue in the list, but seems like my error is something
different, because i get a warning about that hdf5 and netcdf4 is not in
the required architecture..

What should i change here, build different hdf5 and netcdf4, or there is
some tweak to do in netCDF4 setup.. ?

Thanks!

  • Favio Medrano

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=39

Problems subclassing netCDF3.Dataset

From [email protected] on February 18, 2010 11:15:16

Hi,

I'm trying to generate a subclass of netCDF3.Dataset, basically by doing

import netCDF3

class Mynetcdf(netCDF3.Dataset):
def init(self,somelist):
netCDF3.Dataset.ini(self,'tmp.nc','w')
self.somelist=somelist

alist=['a','b','c']

mncdf=Mynetcdf(alist)

Now, mncdf.somelist is in fact a string!

I'm using python 2.6 on ubuntu 9.04 and version netCDF 0.9

Any idea what goes wrong here?

Thanks a lot!

Lucas Merckelbach

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=38

possible performance issue with netCDF3 files

From [email protected] on October 05, 2009 10:45:10

I believe I may have discovered a performance issue, although it may well
be a case of me doing something unusual. I have created a minimal test
case, attached.

In the attached script, I create a netCDF3 classic file, with 3 dimensions,
and create 100 variables, each with a single attribute. As you should be
able to see from the output of the script, the time taken to insert each
attribute grows rapidly - at the end, it is taking about 10 seconds to
insert a single attribute. However, if I set the output format to NETCDF4,
the problem goes away.

My software versions:
netCDF 4.0.1
netcdf4-python 0.8.2
hdf5 1.8.1

Attachment: test.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=25

glibc error in tst_compoundvar

From david.huard on February 08, 2010 09:08:06

tst_compoundvar crashes on Ubuntu 9.10 with

netcdf 4.0.1
hdf5 1.8.4
numpy 1.5.0.dev8079

Here is the backtrace, I can't make sense of any of it.
#0 0x00e2c5a6 in ?? () from /lib/tls/i686/cmov/libc.so.6
#1 0x00e2f7cd in free () from /lib/tls/i686/cmov/libc.so.6
#2 0x003498b1 in array_dealloc (self=0x867e670) at

numpy/core/src/multiarray/arrayobject.c:209
#3 0x003498fc in array_dealloc (self=0x8632010) at

numpy/core/src/multiarray/arrayobject.c:196
#4 0x0815eaba in frame_dealloc (f=0x85df994) at ../Objects/frameobject.c:420
#5 0x08101384 in tb_dealloc (tb=0x858293c) at ../Python/traceback.c:28
#6 0x08101394 in tb_dealloc (tb=0x85826bc) at ../Python/traceback.c:27
#7 0x0808e821 in PyDict_DelItem (op=0xb7fbbb54, key=0x85828b8) at

../Objects/dictobject.c:755
#8 0x0808e88c in PyDict_DelItemString (v=0xb7fbbb54, key=0x8215bdd

"exc_traceback")
at ../Objects/dictobject.c:2314
#9 0x080d8554 in reset_exc_info (f=0x85c4854, throwflag=0) at

../Python/ceval.c:3123
#10 PyEval_EvalFrameEx (f=0x85c4854, throwflag=0) at ../Python/ceval.c:2718
#11 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7f94cc8, globals=0xb7f9157c,

locals=0x0, args=0x8584238,
argcount=2, kws=0x84c9390, kwcount=0, defs=0xb7ee1c18, defcount=1,
closure=0x0)
at ../Python/ceval.c:2968
#12 0x0816022f in function_call (func=0xb7ee472c, arg=0x858422c, kw=0x8581a44)

at ../Objects/funcobject.c:524

#13 0x0806120a in PyObject_Call (func=0xb7ee472c, arg=0x858422c, kw=0x8581a44)

at ../Objects/abstract.c:2492

#14 0x080daf19 in ext_do_call (f=0x85c46ec, throwflag=0) at

../Python/ceval.c:4019
#15 PyEval_EvalFrameEx (f=0x85c46ec, throwflag=0) at ../Python/ceval.c:2429
#16 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7f94d10, globals=0xb7f9157c,

locals=0x0, args=0x85841f8,
argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at
../Python/ceval.c:2968
#17 0x0816014c in function_call (func=0xb7ee4764, arg=0x85841ec, kw=0x0) at

../Objects/funcobject.c:524
#18 0x0806120a in PyObject_Call (func=0xb7ee4764, arg=0x85841ec, kw=0x0) at

../Objects/abstract.c:2492
#19 0x080684ac in instancemethod_call (func=0x858234c, arg=0x85841ec, kw=0x0)

at ../Objects/classobject.c:2579

#20 0x0806120a in PyObject_Call (func=0x858234c, arg=0x85841cc, kw=0x0) at

../Objects/abstract.c:2492
#21 0x080aedd7 in slot_tp_call (self=0x858410c, args=0x85841cc, kwds=0x0)

at ../Objects/typeobject.c:5378

#22 0x0806120a in PyObject_Call (func=0x858410c, arg=0x85841cc, kw=0x0) at

../Objects/abstract.c:2492
#23 0x080dbc3c in do_call (f=0x85c4584, throwflag=0) at ../Python/ceval.c:3924
#24 call_function (f=0x85c4584, throwflag=0) at ../Python/ceval.c:3729
#25 PyEval_EvalFrameEx (f=0x85c4584, throwflag=0) at ../Python/ceval.c:2389
#26 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7fa04e8, globals=0xb7f9157c,

locals=0x0, args=0x857e758,
argcount=2, kws=0x84c9380, kwcount=0, defs=0x0, defcount=0,
closure=0x0) at ../Python/ceval.c:2968
#27 0x0816022f in function_call (func=0xb7ee4b8c, arg=0x857e74c, kw=0x85819bc)

at ../Objects/funcobject.c:524

#28 0x0806120a in PyObject_Call (func=0xb7ee4b8c, arg=0x857e74c, kw=0x85819bc)

at ../Objects/abstract.c:2492

#29 0x080daf19 in ext_do_call (f=0x85c441c, throwflag=0) at

../Python/ceval.c:4019
#30 PyEval_EvalFrameEx (f=0x85c441c, throwflag=0) at ../Python/ceval.c:2429
#31 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7fa0530, globals=0xb7f9157c,

locals=0x0, args=0x8584138,
argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at
../Python/ceval.c:2968
#32 0x0816014c in function_call (func=0xb7ee4bc4, arg=0x858412c, kw=0x0) at

../Objects/funcobject.c:524
#33 0x0806120a in PyObject_Call (func=0xb7ee4bc4, arg=0x858412c, kw=0x0) at

../Objects/abstract.c:2492
---Type to continue, or q to quit---
#34 0x080684ac in instancemethod_call (func=0x857df7c, arg=0x858412c, kw=0x0)

at ../Objects/classobject.c:2579

#35 0x0806120a in PyObject_Call (func=0x857df7c, arg=0x858408c, kw=0x0) at

../Objects/abstract.c:2492
#36 0x080aedd7 in slot_tp_call (self=0x85840ec, args=0x858408c, kwds=0x0)

at ../Objects/typeobject.c:5378

#37 0x0806120a in PyObject_Call (func=0x85840ec, arg=0x858408c, kw=0x0) at

../Objects/abstract.c:2492
#38 0x080dbc3c in do_call (f=0x85c42b4, throwflag=0) at ../Python/ceval.c:3924
#39 call_function (f=0x85c42b4, throwflag=0) at ../Python/ceval.c:3729
#40 PyEval_EvalFrameEx (f=0x85c42b4, throwflag=0) at ../Python/ceval.c:2389
#41 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7fa04e8, globals=0xb7f9157c,

locals=0x0, args=0x857e878,
argcount=2, kws=0x82f12f0, kwcount=0, defs=0x0, defcount=0,
closure=0x0) at ../Python/ceval.c:2968
#42 0x0816022f in function_call (func=0xb7ee4b8c, arg=0x857e86c, kw=0x8581934)

at ../Objects/funcobject.c:524

#43 0x0806120a in PyObject_Call (func=0xb7ee4b8c, arg=0x857e86c, kw=0x8581934)

at ../Objects/abstract.c:2492

#44 0x080daf19 in ext_do_call (f=0x85c414c, throwflag=0) at

../Python/ceval.c:4019
#45 PyEval_EvalFrameEx (f=0x85c414c, throwflag=0) at ../Python/ceval.c:2429
#46 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7fa0530, globals=0xb7f9157c,

locals=0x0, args=0x857e958,
argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at
../Python/ceval.c:2968
#47 0x0816014c in function_call (func=0xb7ee4bc4, arg=0x857e94c, kw=0x0) at

../Objects/funcobject.c:524
#48 0x0806120a in PyObject_Call (func=0xb7ee4bc4, arg=0x857e94c, kw=0x0) at

../Objects/abstract.c:2492
#49 0x080684ac in instancemethod_call (func=0x857df54, arg=0x857e94c, kw=0x0)

at ../Objects/classobject.c:2579

#50 0x0806120a in PyObject_Call (func=0x857df54, arg=0x857edcc, kw=0x0) at

../Objects/abstract.c:2492
#51 0x080aedd7 in slot_tp_call (self=0x857efec, args=0x857edcc, kwds=0x0)

at ../Objects/typeobject.c:5378

#52 0x0806120a in PyObject_Call (func=0x857efec, arg=0x857edcc, kw=0x0) at

../Objects/abstract.c:2492
#53 0x080dbc3c in do_call (f=0x85c3a1c, throwflag=0) at ../Python/ceval.c:3924
#54 call_function (f=0x85c3a1c, throwflag=0) at ../Python/ceval.c:3729
#55 PyEval_EvalFrameEx (f=0x85c3a1c, throwflag=0) at ../Python/ceval.c:2389
#56 0x080dd384 in fast_function (f=0x85c38ac, throwflag=0) at

../Python/ceval.c:3792
#57 call_function (f=0x85c38ac, throwflag=0) at ../Python/ceval.c:3727
#58 PyEval_EvalFrameEx (f=0x85c38ac, throwflag=0) at ../Python/ceval.c:2389
#59 0x080dd384 in fast_function (f=0x85c0a6c, throwflag=0) at

../Python/ceval.c:3792
#60 call_function (f=0x85c0a6c, throwflag=0) at ../Python/ceval.c:3727
#61 PyEval_EvalFrameEx (f=0x85c0a6c, throwflag=0) at ../Python/ceval.c:2389
#62 0x080dddf2 in PyEval_EvalCodeEx (co=0xb7ee03c8, globals=0xb7f9157c,

locals=0x0, args=0x857ec98,
argcount=1, kws=0x0, kwcount=0, defs=0xb7ee3158, defcount=5,
closure=0x0) at ../Python/ceval.c:2968
#63 0x0816014c in function_call (func=0xb7ee53e4, arg=0x857ec8c, kw=0x0) at

../Objects/funcobject.c:524
#64 0x0806120a in Py...

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=35

RuntimeError: HDF error - large nc file / max groups?

From [email protected] on April 05, 2010 23:52:47

What steps will reproduce the problem? 1. looping through function to create group / group variables via input
file set
2. will produce error when nc file size is approx 3.42 GB
3. at 3.42 GB function has created approx 3558 groups What is the expected output? What do you see instead? Expected to continue appending nc file. See error. What version of the product are you using? On what operating system? python-2.6.5.msi
netCDF4-0.9.win32-py2.6.exe
numpy-1.3.0-win32-superpack-python2.6.exe

Wing IDE Professional
3.2.5-1 (rev 21674)

Windows 7 64-bit, 8GB RAM Please provide any additional information below. rootgrp is opened and closed with each iteration:

rootgrp = Dataset(NC_FIL, 'a', format='NETCDF4')

code.....

rootgrp.close()

The error is not input file dependent; can go back and create group /
group variables with input file sets inclusive of the input file where
the error occurred.

Produced same error twice, after lengthy program execution time.
Appending to nc files seems to slow dramatically with continued execution,
although I believe this is to be expected.

No problem creating 1.44 GB nc file containing 1624 groups / group
variables with same function and same formatted input files.

Exceptions:
File "C:\Python26\Lib\site-packages\netCDF4.pyd", line 2518, in
netCDF4.Variable.setitem (netCDF4.c:17207)
File "C:\Python26\Lib\site-packages\netCDF4.pyd", line 2642, in
netCDF4.Variable._put (netCDF4.c:18195)

  • traceback to function statements;
    lat = grp.createVariable('lat','f4',('lat',))
    lat[:] = lats

Wing IDE Professional Messages;
Debugger: Open failed: C:/Python26/Lib/site-packages/netCDF4.pyd is
probably a binary file
-- the Messages contain many instances of the above message, but up to the
point of RuntimeError: HDF error, always reconnects with the debugger.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=47

64 bit linux installation fails

From [email protected] on August 19, 2009 02:46:47

I've installed netcdf-4.0.1 on a Samsung NC20 running Ubuntu.

root@nc20:~# uname -a
Linux nc20 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 19:25:34 UTC 2009
x86_64 GNU/Linux

I've done 'make test' under both NetCDF and hdf-1.8.3 and both passes OK.

Now when I try and install netcdf-python I get:

kb@nc20:~/python/netCDF4-0.8.2$ python setup.py build
:
:
compile options: '-I/opt/netcdf4/include -I/opt/hdf5-1.8.3/include
-I/usr/lib/python2.6/dist-packages/numpy/core/include
-I/usr/include/python2.6 -c'
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/netCDF4.o -L/opt/netcdf4/lib
-L/opt/hdf5-1.8.3/lib -Wl,-R/opt/netcdf4/lib -Wl,-R/opt/hdf5-1.8.3/lib
-lnetcdf -lhdf5 -lhdf5_hl -lz -o build/lib.linux-x86_64-2.6/netCDF4.so
/usr/bin/ld: /opt/netcdf4/lib/libnetcdf.a(nc4attr.o): relocation
R_X86_64_32 against `a local symbol' can not be used when making a shared
object; recompile with -fPIC
/opt/netcdf4/lib/libnetcdf.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

kb@nc20:~/python/netCDF4-0.8.2$ file /usr/bin/ld
/usr/bin/ld: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped

What do I need to do to fix it?

Karsten

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=22

MemoryError

From [email protected] on March 05, 2009 09:52:44

What steps will reproduce the problem? 1. Run a loop where you open one large (158MB) netCDF file for each iteration
2. Extract the data from the netCDF loop and do something to the data
3. close the file, delete the stored arrays and go to next iteration. After 11-16 iterations I run
out of memory. I delete the arrays by doing: del array1, array2... What is the expected output? What do you see instead? MemoryError (error code=12) What version of the product are you using? On what operating system? 0.7.7 + python 2.5.1 on Mac OS 10.5 (Intel) 32 bit Please provide any additional information below. I seem to run out of memory when i loop over netcdf files that are opened, extracted, and then
closed. when I run top as the program is running, I can see memory consumption increases for
each iteration until it crashes. I enabled garbage collection and printed out what is in the
memory (that is not fred between iterations). It looks like this:
...
gc: collectable <netCDF4.Dimension 0x3579288>
gc: collectable <netCDF4.Dimension 0x3579260>
gc: collectable <netCDF4.Dimension 0x3579210>
gc: collectable <netCDF4.Dimension 0x35792b0>
gc: collectable <netCDF4.Dimension 0x35792d8>
gc: collectable <netCDF4.Dimension 0x3579300>
gc: collectable <netCDF4.Dimension 0x3579328>
gc: collectable <netCDF4.Dimension 0x3579350>
gc: collectable <netCDF4.Dimension 0x3579378>
gc: collectable <dict 0x33838a0>
gc: collectable <netCDF4.Variable 0x355af48>
gc: collectable <netCDF4.Variable 0x355af80>
gc: collectable <netCDF4.Variable 0x355afb8>
gc: collectable <netCDF4.Variable 0x3540030>
gc: collectable <netCDF4.Variable 0x3540068>
gc: collectable <netCDF4.Variable 0x35400a0>
gc: collectable <netCDF4.Variable 0x35400d8>
gc: collectable <netCDF4.Variable 0x3540110>
gc: collectable <netCDF4.Variable 0x3540148>
gc: collectable <netCDF4.Variable 0x3540180>

...

Thanks for helping me.
Cheers, Trond Kristiansen

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=14

dimensions of size 0 gives no error

From [email protected] on July 28, 2009 04:49:55

Just a slight thing:

import netCDF3
ncfile = netCDF3.Dataset('name', 'w')
ncfile.createDimension('ndim1', size=0)

Creates a unlimited dimension

import netCDF3
ncfile = netCDF3.Dataset('name', 'w')
ncfile.createDimension('ndim1', size=-1)

gives an error.

Shouldn't the first also give an error as opposed to create an unlimited
dimension? or else it should be documented.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=19

Size one variables are not represented as numpy array

From [email protected] on July 28, 2009 05:14:02

The type of a variable corresponding to a netCDF variable of length >1 is a
'numpy.ndarray', but for a netCDF variable of length 1 a python 'float' is
returned. This causes problems when using things like len(var[:]).
Couldn't a numpy.ndarray of shape (1,) be returned?

A netCDF file with following CDL representation:

netcdf bug_size_array {
dimensions:
time = 1 ;
X = 2;
variables:
double time(time) ;
double X(X) ;
data:

time = 99.0 ;
X = 1., 2.;
}

run with

import netCDF3
ncfile = netCDF3.Dataset('bug_netcdf3.nc', 'r')
for var in ncfile.variables.values():
print var[:], type(var[:]), var[:].shape

produces output:

[ 1. 2.] <type 'numpy.ndarray'> (2,)
99.0 <type 'float'>
Traceback (most recent call last):
File "tmp.py", line 4, in
print var[:], type(var[:]), var[:].shape
AttributeError: 'float' object has no attribute 'shape'

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=20

Dataset.renameVariable method doesn't appear to work properly

From [email protected] on May 18, 2009 10:34:16

What steps will reproduce the problem? 1. root = Dataset('a netcdf file', 'r+')
2. root.renameVariable('old name', 'new name') 3. What is the expected output? What do you see instead? I expected the statement in (2) to rename the variable in the netcdf file.
Instead I get an error:

Traceback (most recent call last):
File "", line 1, in
File "netCDF4.pyx", line 1172, in netCDF4.Dataset.renameVariable
(netCDF4.c:5213)
RuntimeError: NetCDF4: Operation not allowed in data mode What version of the product are you using? On what operating system? I'm using version 0.8 of the product on a Mac OS X 10.4.11 operating system. Please provide any additional information below. I made sure the netcdf file I was opening had its permission settings set
to read/write. I also tried using the 'a' flag instead of the 'r+' flag to
open the data file.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=17

MFDataset weird behavior

From david.huard on July 31, 2008 10:06:18

I am working with NCEP reanalysis netCDF files, and here is what I tried to do:

D = nc.MFDataset('uwnd.sig995.199*.nc', True)

<13: array([ 17435256., 17435262., 17435268., 17435274.])

[14]: D.variables['time'][4]

ValueError Traceback (most recent call last)

/storage/data/NCEP1/ in ()

/usr/local/lib64/python2.5/site-packages/netCDF4.so in
netCDF4._Variable.getitem (netCDF4.c:16019)()

ValueError: 0-d arrays can't be concatenated

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.