Hello all,

I've been trying out some diagnostic code in which I loop through all of the radar names listed in the network() list. The documentation in radDataRead notes that if channel isn't set, it will read data from all channels, but this doesn't seem to be working in the develop branch. I have this functionality in the master branch. Is there something different going on here?

Specifically, I'm having trouble reading in both kod.c and kod.d data when asking for kod and channel=None.

-Kevin

Now that radDataRead.py and radDataTypes.py have been generalized beyone institution specific code, sdDataRead.py and sdDataTypes should be made consistent with their rad* counterparts.

In other words, the sdDataPtr needs to be made more pythonic in the same way as #47 made radDataPtr more pythonic. After this, fetchUtils.py functions need to be implemented in sdDataTypes.py in the same way as #59 generalized radDataTypes.py.

Once I have some more free time (~one week, maybe less), I will start working on this.

RTI plotting fails with an Assertion error if you try to use the 'mag' or 'mlt' coords. Here's the code I tried running:

pydarn.plotting.rti.plotRti(datetime(2009,6,24,0,30),'bks',eTime=datetime(2009,6,24,1,30),coords='mag')

and here's the error message:

Found cached file: /tmp/sd/20090624.000000.20090624.020000.bks.fitex

reached end of data
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-13-9f34eb4a61a6> in <module>()
----> 1 pydarn.plotting.rti.plotRti(datetime(2009,6,24,0,30),'bks',eTime=datetime(2009,6,24,1,30),coords='mag')

/home/ashtonsethreimer/.local/lib/python2.7/site-packages/davitpy-0.2-py2.7-linux-x86_64.egg/pydarn/plotting/rti.pyc in plotRti(sTime, rad, eTime, bmnum, fileType, params, scales, channel, coords, colors, yrng, gsct, lowGray, pdf, png, dpi, show, retfig, filtered, fileName, custType, tFreqBands, myFile, figure, xtick_size, ytick_size, xticks, axvlines, plotTerminator)
    256       #draw the axis
    257       ax = drawAxes(rtiFig,times[fplot],rad,cpid[fplot],bmnum,nrang[fplot],frang[fplot],rsep[fplot],p==len(params)-1,yrng=yrng,coords=coords,\
--> 258                     pos=pos,xtick_size=xtick_size,ytick_size=ytick_size,xticks=xticks,axvlines=axvlines)
    259 
    260 

/home/ashtonsethreimer/.local/lib/python2.7/site-packages/davitpy-0.2-py2.7-linux-x86_64.egg/pydarn/plotting/rti.pyc in drawAxes(myFig, times, rad, cpid, bmnum, nrang, frang, rsep, bottom, yrng, coords, pos, xtick_size, ytick_size, xticks, axvlines)
    437         if(coords == 'geo' or coords == 'mag'):
    438           site = pydarn.radar.network().getRadarByCode(rad).getSiteByDate(times[i])
--> 439           myFov = pydarn.radar.radFov.fov(site=site, ngates=nrang[i],nbeams=site.maxbeam,rsep=rsep[i],coords=coords)
    440           if(myFov.latFull[bmnum].max() > ymax): ymax = myFov.latFull[bmnum].max()
    441           if(myFov.latFull[bmnum].min() < ymin): ymin = myFov.latFull[bmnum].min()

/home/ashtonsethreimer/.local/lib/python2.7/site-packages/davitpy-0.2-py2.7-linux-x86_64.egg/pydarn/radar/radFov.pyc in __init__(self, frang, rsep, site, nbeams, ngates, bmsep, recrise, siteLat, siteLon, siteBore, siteAlt, siteYear, elevation, altitude, model, coords, date_time, coord_alt)
    237                       lonC, latC = coord_conv(lonC, latC, "geo", coords,
    238                                               altitude=t_c_alt,
--> 239                                               date_time=date_time)
    240                       lonE, latE = coord_conv(lonE, latE, "geo", coords,
    241                                               altitude=t_c_alt,

/home/ashtonsethreimer/.local/lib/python2.7/site-packages/davitpy-0.2-py2.7-linux-x86_64.egg/utils/coordUtils.pyc in coord_conv(lon, lat, start, end, altitude, date_time, end_altitude)
    146     if start in dt_sys or end in dt_sys:
    147         assert(date_time is not None),\
--> 148                 "date_time must be provided for: " + dt_string
    149 
    150     # Sanitise inputs.

AssertionError: date_time must be provided for: 
AACGM (mag)
MLT (mlt)

I was tracking down an issue here and it seems as though the DAVIT_LOCAL_TIMEINC environment variable defaulted to 2 on my quick setup. This leads fetch_*_files to skip odd hour files. Should these files really be skipped? I think for a while some radars were generating a new file every hour, so this would miss half of that data. There's other cases where radars are stopped and started in which odd hour files are generated....including triggered modes!

I'm converting radFov to use coord_conv. The altitude is set to zero in the original AACGM call for conversion from geo to mag. Is that how it should stay? I suppose it depends on whether the beam-gate coordinates are meant to be used at altitude or on the ground.

https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/radar/radFov.py#L214

If a user tries to read a record using radDataReadRec(myPtr) but Ptr.ptr is closed, this results in a segmentation fault. Is this something that can be prevented in radDataReadRec?

Example using davitpy on Ubuntu 13.04:

In [1]: import datetime as dt

In [2]: myPtr = pydarn.sdio.radDataOpen(dt.datetime(2012,11,24,5),'kod',eTime=dt.datetime(2012,11,24,9),channel=None, bmnum=3,cp=1313,fileType='fitacf',custType='fitacf',filtered=False, src=None, fileName='data/20121124.0000.03.kod.c.fitacf')
cp data/20121124.0000.03.kod.c.fitacf /tmp/fit/1370440553
Concatenating all the files in to one
cat /tmp/fit/1370440553 > /tmp/fit/1370440553.kod.fitacf
rm /tmp/fit/1370440553

In [3]: myPtr.ptr.close()

In [4]: myBeam = pydarn.sdio.radDataReadRec(myPtr)
/home/ashtonsethreimer/davitpy/bin/davitpy: line 23: 4703 Segmentation fault (core dumped) ipython --profile=davitpy --pylab

While my example reads a local fitacf, the result is the same for remotely pulled fitex (as you might expect).

-Ashton

Sometimes, when installing davitpy on Ubuntu, the shared libraries will compile (dmap and aacgm), but will fail to execute. When running ldd aacgmlib.so we get the following partial output:

libmlt.1.so => not found
libaacgm.1.so => not found
libastalg.1.so => not found
librtime.1.so => not found

Hi, guys. I've noticed that mapObj has both datetime and dateTime keywords. What is the difference? It is not documented.

On a related note, the documentation of some other calls, like pydarn.plotting.overlayFov(), say that these will overlay onto a basemap object. That is no longer true, as you actually need to provide a mapObj with specified datetime attribute.

I think @mrwessel may have some comments on this.

Thanks!

~Nathaniel

Hello all again,

...although this may be mostly for Jef. Just did a fresh install of davitpy here on a sandbox area so I'm not messing with VT's main davitpy. Anywho, I've got some extra code that is trying to access the pydarn.radar.network() object but it's coming up with a missing .radars.sqlite file not found. Where does this file come from?

It should default to the mapObj.dateTime so as not to put things in unexpected places on the plot. This is fixed in #78.

The problem is with the assert at line 382:
https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/sdio/fetchUtils.py#L382

The code is required to be a three-char string, which precludes use of the old single-char codes (f=han, e=pyk, etc.).

Furthermore, the code is required to exist. While the function is useful for retrieving convection map files, it is necessary to feed it a bogus radar code just to make it work, and that's not good practice.

The numpy package for opensuse 12.1 does not contain the file arrayobject.h, and whilst building basemap, compilation fails to find the file.

In order to get around this, two issues must be resolved:

1. You need to download numpy source and build it.
2. You need to use the prefix /usr (default is /usr/local) because include directories while building basemap do not include /usr/local/.

wget http://downloads.sourceforge.net/project/numpy/NumPy/1.7.1/numpy-1.7.1.tar.gz
tar xvf numpy-1.7.1.tar.gz
cd numpy-1.7.1/
python setup.py install --prefix=/usr

The problem is with the assert at line 384:
https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/sdio/fetchUtils.py#L384

Only file types for acf and iq data are permitted, but this function is also useful for retrieving convection map data. Convection map file formats then need to be set in fnamefmt (and done as a list if there is more than one possibility) and a bogus but acceptable file type needs to be fed in anyway, though it is not used.

Either the permitted file types need to be expanded to include convection map types, or the assert needs to be rethought.

I have identified a memory leak in the models.aacgm.aacgmConvArr() function.

Tests show that if you call this function many times in the same ipython or python session, on both Ubuntu 13.04 and OpenSUSE 12.3, your RAM usage will increase but never go back down until the ipython or python session is closed.

If instead you use a python for loop to process an array of latitudes/longitudes with models.aacgm.aacgmConv() instead, RAM usage does not increase with each successive function call.

The following two functions can be used to illustrate the memory leak:

def test_aacgmConvArr():

    from models import aacgm
    import matplotlib
    import numpy as np

    loop_num=20

    lats=(60.0*np.ones(512*512)).tolist()
    lons=(120.0*np.ones(512*512)).tolist()
    heights=(250.0*np.ones(512*512)).tolist()

    start_mem=matplotlib.cbook.report_memory()
    print "aacgmConvArr test started with " + str(start_mem) + " kilobytes of RAM"
    for i in range(0,loop_num):
        mlat, mlon, r = aacgm.aacgmConvArr(lats, lons, heights, 0)
    print "memory usage now "+str(matplotlib.cbook.report_memory())+ " kilobytes of RAM"
    end_mem=matplotlib.cbook.report_memory()
    print "aacgmConvArr test ended with " + str(end_mem) + " kilobytes of RAM"
    print "aacgmConvArr test looped " + str(loop_num) +" times"

def test_aacgmConv():

    from models import aacgm
    import matplotlib
    import numpy as np

    loop_num=20

    lats=(60.0*np.ones(512*512)).tolist()
    lons=(120.0*np.ones(512*512)).tolist()
    heights=(250.0*np.ones(512*512)).tolist()

    start_mem=matplotlib.cbook.report_memory()
    print "aacgmConv test started with " + str(start_mem) + " kilobytes of RAM"
    for i in range(0,loop_num):
        for i in range(0,len(lats)):
            mlat, mlon, r = aacgm.aacgmConv(lats[i], lons[i], heights[i], 0)
    print "memory usage now "+str(matplotlib.cbook.report_memory())+ " kilobytes of RAM"
    end_mem=matplotlib.cbook.report_memory()
    print "aacgmConv test ended with " + str(end_mem) + " kilobytes of RAM"
    print "aacgmConv test looped " + str(loop_num) +" times"

The problem must have something to do with how the aacgmConvArr() function is created with the C python wrapper in the davitpy/models/aacgm/aacgmlib.c file. I'm not sure how to fix this.

Hi,

I have just installed davitpy (thank you for the effort!). Now, it seams that all the models are working as they should, but I do get problems with AACGM. When writing
from models import *
I get:
/home/magnar/Programmering/Python/vtsuperdarn-davitpy-1a1924f/models/aacgm/init.pyc -> aacgmlib: No module named aacgmlib

Anyone now how to deal with this?

Best regards,
Magnar

Hi
I have been trying msis and do not find that the Texo changes with f10.7 as I would expect in fact I alway get the same value Texo= 1027.3K
Any comment?

using the command:

pydarn.plotting.fan.plotFan(dt.datetime(2011,10,2,5),['sas'],fileType='fitacf',interval=120,coords='geo')

works great and makes a nice little fan plot, but using coords='mag' makes a plot in which the data and fov are in completely difference places.

Matt Wessel and I are in the process of fixing this. We are implementing utils.mapObj instead of basemap.Basemap and this seems to be fixing the issue.

Once we have fixed fan.py we will submit a pull request.

Both datetime and dateTime exist. I am working on this.

In radDataTypes the methods readRec (of class radDataPtr) and updateValsFromDict (of class radBaseData) both attempt to use "channel" in the dmap record to determine the channel of the data (as in, the UAF radar operating frequency channel). The problem is, as discussed in #91, that channel was only meant to be used on STEREO radars and is zero for all others. The UAF radars therefore all have the channel listed as zero in the dmap record.

The result of this is that the channel checking in the methods above serves no purpose and causes problems instead. Because channel number is always 0, the methods always think they are dealing with channel 'a' data. This causes failure when trying to read in data from, say, channel 'c', because there is a test for dmap record channel equal to desired channel that can never be satisfied.

As it is, there is an if-else structure that sets the channel of the data to a letter code based on the dmap record channel number, then tests for desired channel == data channel. If we know the data channel from the filename (and that's the only place to get it from) then setting data channel = filename channel and testing for a match to the desired channel is redundant because the files are selected based on the desired channel in the first place. However, there is still the matter of actual STEREO radar data, and I'm working on that...more to follow.

-Missing self for some attributes when used in the draw method of mapObj
-Wrong call to aacgm methods in radFov
-Code for boundaries accidentally excised from plotUtils by #61

I'm working on these as a bugfix. More to follow as the situation progresses...

Hi, guys.

I started making a new notebook to demonstrate basic SuperDARN data plotting (branch: feature_plotting_notebook), especially in response to request from Judy Stephenson. I especially ran into trouble when trying to plot fan plots in magnetic coordinates:

pydarn.plotting.fan.plotFan(datetime.datetime(2013,3,16,16,30),['fhe','fhw'],param='power',gsct=False)

pydarn.plotting.fan.plotFan(datetime.datetime(2013,3,16,16,30),['fhe','fhw'],param='power',gsct=False,coords='mag')

Is anyone else familiar with this problem?

Thanks,
Nathaniel

As stated by the wise @jspaleta, chapter vtsuperdarn/davitpy repository, verse #56 "we need to move to a pydarnrc file that has all the user configurable environment stuff instead of all the environment shell variables. A real config file for all of pydarn environment configurables will fully solve the problem. Like how the matplotlibrc works. And we need to make use of the XDG environment variables and drop or files into the directories defined via XDG like any other application that needs tmp files or config files and data files. Basically we need to stop re-inventing the wheel. I say that with full knowledge that I've invented several of the oddly shaped wheels in the codebase." Seriously though, this is a very good idea and we need to start working on this.

We have a bunch of environment variables that are used by radDataTypes and sdDataTypes and a honestly, the way it is set up is really ugly and messy. A configuration file that handles this would be much cleaner. Also, we have a need for a configuration dictionary for initializing the logging library as discussed in #109. As we keep developing, I think we will only see an increased demand for an rc file.

So let's get working on that. To get the conversation started what is an "rc file"? "rc" is short for "Run command" and you can read more about it here: http://en.wikipedia.org/wiki/Run_commands This might also be interesting and is certainly relevant: http://en.wikipedia.org/wiki/Configuration_file

How would it "look" if we were to implement a davitpyrc file? For an idea, we can study matplotlib. When we run from matplotlib import pyplot, the pyplot.py file runs an import matplotlib, which in turn executes __init__.py here: https://github.com/matplotlib/matplotlib/tree/master/lib/matplotlib

In the __init__.py file, there is a call to import matplotlib.rcsetup (see this line: https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/__init__.py#L197).

So one potential way to package davitpy is to model it after matplotlib. The basic idea would be to have a root level __init__.py in the davitpy directory which is in turn imported by every davitpy module (pydarn, models, etc.). Then an rcsetup.py file can be placed in the davitpy directory that handles reading a .davitpyrc configuration file and has hardcoded defaults to fall back to.

Here is the matplotlib rcsetup.py for reference: https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/rcsetup.py. Of specific interest to us right now is probably the defaultParams dictionary found here https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/rcsetup.py#L479

Here's some more useful information about matplotlibrc: http://matplotlib.org/users/customizing.html

Here's where/how the rcParams dictionary is populated: https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/__init__.py#L1079 and https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/__init__.py#L923 and https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/__init__.py#L833

So, let's have a conversation and figure this out. What's the best way to proceed? Do we need to reorganize the way davitpy is currently packaged? Is davitpy supposed to function as a software library, or a framework (with pydarn, models, etc. as libraries)?

In the init for the network class the code looks for the database ".radars.sqlite" in $DAVIT_TMPDIR first, then tries $HOME and finally file. In the inits for the radar class and the site class, however, the code looks for "radars.sqlite" (note no leading dot) and never checks $HOME, just $DAVIT_TMPDIR and file. My .radars.sqlite happens to live in $HOME, and there is no database of any name in my file (which turns out to be pydarn/radar), and I have not got $DAVIT_TMPDIR set. My questions are:
-Which name is the db supposed to have?
-Where is is supposed to live?
-Which classes are looking for databases in all the wrong places?

Note that this is all going on in the develop branch.

Code links:
network class
https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/radar/radStruct.py#L59
radar class
https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/radar/radStruct.py#L392
site class
https://github.com/vtsuperdarn/davitpy/blob/develop/pydarn/radar/radStruct.py#L605

When I use radDataOpen to create a fitacf file containing more than one day of data, I can only get to the end of the first day with radDataReadRec. Not being fluent in fitacf, I can't even tell whether the subsequent days are present in the file at all. I don't believe we've changed any of the code surrounding the concatenation for use with our USask fitacf files, so maybe it just doesn't work for any fitacf.

While randomly testing different bits of davitpy I discovered that somewhere we lost the ability for the tsyganenko model to work properly. This bug affects the develop branch and the master branch! @sdelarquier since you worked on this implementing this model in davitpy can you help out with this bug please?

Here's the code I used to discover the bug:

from numpy import arange, zeros, ones
from models import tsyganenko
lats = arange(10, 90, 10)
lons = zeros(len(lats))
rhos = 6372.*ones(len(lats))
trace = tsyganenko.tsygTrace(lats, lons, rhos,datetime=datetime(2014,9,17,12))
ax = trace.plot()

Notice that this code should produce a plot of the Earth's dipolar field on the +X GSM side of the Earth. Instead I get something like this:

Something is very wrong here and I'm not exactly sure what is going on.

When calling msis.getF107Ap, I get an index error whenever the time passed to the function is later than approximately 22:00. I am running the msis model for a large dataset, and I am worried that this error also implies that the time for wrong for other hours as well..

ttap = [ tap[dtInd][hrInd-i] for i in range(hrInd+1) ]
IndexError: list index out of range

By the way, thanks for making this!

I'm pretty much in the dark when it comes to most of the data types/structures we're using. It seems to me I that if I want to know the data types we use, I would need to go digging in the source code, play around with different data types, explore their attributes and methods, and find out where and how they are used (e.g. which functions and methods accept which of the datatypes).

I think it would be a good idea to make a notebook which goes through all the different data types/structures we use, and explains how they're structured and how and where they're used, including how they can be of use to the user (e.g. getting data for a time interval and passing that data structure when looping over the time interval to do some plotting, just to name one example).

I am having difficulties installing raydarn on Ubuntu 12.04. I get the following error message when running mastermake:

mpif90 -w -O2 -fbacktrace -g -fno-automatic -c raytrace_mpi.f90 -o raytrace_mpi.o
raytrace_mpi.f90:264.55:

types = (/MPI_REAL, MPI_INTEGER, MPI_REAL, MPI_CHAR/)
                                                   1
Error: Symbol 'mpi_char' at (1) has no IMPLICIT type
make: *** [raytrace_mpi.o] Error 1

Not sure what the problem would be. Does anyone know?

This should probably be a pull request but I'm new to git and not quite sure how it all works.

There is a parenthesis bug in davitpy/pydarn/sdio/sdDataRead.py Line 449:
if len(myList > 0)
should be:
if len(myList) > 0

This method in radDataTypes/sdDataTypes would take a datetime (or possibly a range of datetimes) as input, then find the item(s) in the record index with the closest datetime key(s). This is especially useful because the keys often include milliseconds, making them (typo-prone) work to enter by hand and difficult to specify automatically...without using code to find the nearest to a given datetime, that is. This method would simply make that built-in.

The following was tested on openSUSE 12.3

When executing the python_install_opensuse.sh file using the sudo command, the last two lines of code:

install_dir=$(readlink -f ../..)
echo "source $install_dir/profile.bash" >> ~/.bashrc

Install the "source $install_dir/profile.bash" into the .bashrc for root (ie. /.bashrc) instead of the .bashrc for the user.

I've tested this using a script that only executes the two lines above. When I use sudo, source is installed into the root bashrc and if I don't use sudo, source is install into my bashrc.

A way to get around this whole issue is to replace the above lines of code with:

WHO=$(who am i | sed -e 's/ .*//')
install_dir=$(readlink -f ../..)
echo "source $install_dir/profile.bash" >> /home/${WHO}/.bashrc

Jeff has a solution for the data path, soon to be pulled.

Can we extend said solution to the file format issue (2-hour vs 1-day files)?

I have got my davitpy install to the point where I can run it with the > davitpy command in my regular user account, but once python loads I get problems with autoreload and basemap:

[TerminalIPythonApp] Running code in user namespace: %load_ext autoreload
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/home/matt/davitpy/<ipython-input-1-c5fcb8db5aad> in <module>()
----> 1 get_ipython().magic(u"load_ext autoreload")

/usr/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s, next_input)
   1892                 self._magic_locals = sys._getframe(1).f_locals
   1893             with self.builtin_trap:
-> 1894                 result = fn(magic_args)
   1895             # Ensure we're not keeping object references around:

   1896             self._magic_locals = {}

/usr/lib/python2.7/site-packages/IPython/core/magic.pyc in magic_load_ext(self, module_str)
   3341     def magic_load_ext(self, module_str):
   3342         """Load an IPython extension by its module name."""
-> 3343         return self.extension_manager.load_extension(module_str)
   3344 
   3345     def magic_unload_ext(self, module_str):

/usr/lib/python2.7/site-packages/IPython/core/extensions.pyc in load_extension(self, module_str)
     85         if module_str not in sys.modules:
     86             with prepended_to_syspath(self.ipython_extension_dir):
---> 87                 __import__(module_str)
     88         mod = sys.modules[module_str]
     89         return self._call_load_ipython_extension(mod)

ImportError: No module named autoreload
[TerminalIPythonApp] Running code in user namespace: %autoreload 2
ERROR: Magic function `autoreload` not found.
[TerminalIPythonApp] Running code in user namespace: import pydarn
problem importing radar:  'DBREADUSER'
utils/__init__.py -> utils.plotUtils:  cannot import name basemap
problem importing dbUtils:  'DBREADUSER'
problem importing fan:  No module named basemap

followed by the python command prompt.

From what I read, autoreload is supposed to be bundled with iPython, but somehow it is missing in my case. As for basemap, we've already been through this, and now there are more troubles!

I installed davitpy on a fresh install of ubuntu 13.04 and it went very smoothly, except for one tiny bug.

Despite the existence of line 54 in davitpy/install/debian/python_install_debian.sh

source ~/.bashrc

When the install script tried to execute ./mastermake there was an error because the environment variable ${DAVITPY} did not exist.

Manually executing the source ~/.bashrc and then ./mastermake worked fine of course so, I'm kind of baffled as to why the script had an issue.

If I figure it out, I'll post it here.

Ashton

This is a very important feature still missing from DaViTpy.

Moving this off of the google group and into github since it seemed as though it was a problem with the code and not something that I was doing.

Again, here is a summary of the problem I'm seeing:

I'm running some code that reads in rawacf files for an entire day for one radar, searches for certain characteristics and then moves on to the next day in repeat. I'm doing a radDataOpen for each day. However, after about a month and half or maybe two months of doing this, the code crashes as something seems to fall apart in the copying of data to the local machine as well as the concatenation.

The very basic of what I'm trying to run can be found in my github repository at:
https://raw.githubusercontent.com/ksterne/radops/master/acfd_find.py

Feeding an m x n array to coord_conv when n != 1 causes failure. When the lats and lons are processed for conversion to MLT element 0 is a list so it can't feed that to an AACGM function expecting a float.

Maps plotted in MLT end up showing degrees from magnetic local midnight, not actual MLT.

There is a problem with the __call__ method in the utils.mapObj class. Performing a call to convert latitude and longitude into map coordinates and then attempting to convert those coordinate back in to latitude and longitude fails when converting between coordinate systems.

For example in geo coords:

from utils import mapObj
map = mapObj(coords='geo',projection='stere',llcrnrlon=100, llcrnrlat=0, urcrnrlon=170, urcrnrlat=40,lat_0=54, lon_0=-120,resolution='l',draw=False)
x,y = map(-120,54)
print x,y
> 14898932.7446 -14364789.7586
lon,lat = map(x,y,inverse=True)
print lon, lat
> -119.99999999999999, 54.000000000000014

It works. For a map in geo coords but converting the x,y coords to mag lat and lon:

from utils import mapObj
from models.aacgm import aacgmConv
map = mapObj(coords='geo',projection='stere',llcrnrlon=100, llcrnrlat=0, urcrnrlon=170, urcrnrlat=40,lat_0=54, lon_0=-120,resolution='l',draw=False)
x,y = map(-120,54)
print x,y
> 14898932.7446 -14364789.7586
mlon,mlat = map(x,y,inverse=True,coords='mag')
print mlon, mlat
> 100.0 -1.07111540008e-10
mlon2,mlat2,_ = aacgmConv(54,-120,0,2010,0)
print mlon2,mlat2
> 59.9324622167 -59.9940107681

It fails. And for a map in mag coords but converting the x,y coords to geo lat and lon:

from utils import mapObj
from models.aacgm import aacgmConv
map = mapObj(coords='mag',projection='stere',llcrnrlon=100, llcrnrlat=0, urcrnrlon=170, urcrnrlat=40,lat_0=54, lon_0=-120,resolution='l',draw=False)
x,y = map(-120,54)
print x,y
> 14898932.7446 -14364789.7586
mlon,mlat = map(x,y,inverse=True,coords='geo')
print mlon, mlat
> 100.0 -1.07111540008e-10
mlon2,mlat2,_ = aacgmConv(54,-120,0,2010,1)
print mlon2,mlat2
> 58.8384430722 175.311901385

This also fails.

I think it has to do with the order in which the coordinate conversions versus conversion to/from x,y coordinates takes place. I'll have a look at it and report back.

I am new here. The question is so dumb. Please help.
I followed the instruction and when I try install using "sudo python setup.py install", the compile error popped out. I think I have all the dependencies. What did I do wrong? I attached all the information below.
Thanks in advance!

spurces ['pydarn', 'pydarn.sdio', 'pydarn.plotting', 'pydarn.radar', 'pydarn.proc', 'pydarn.proc.music', 'pydarn.proc.signal', 'gme', 'gme.isr', 'gme.plotting', 'gme.sat', 'gme.ind', 'gme.base', 'utils', 'models', 'models.igrf', 'models.hwm', 'models.tsyganenko', 'models.iri', 'models.raydarn', 'models.msis']
options (after parsing config files):
options (after parsing command line):
option dict for 'aliases' command:
{}
option dict for 'install' command:
{}
running install
Distribution.get_command_obj(): creating 'install' command object
pre-finalize_{unix,other}:
prefix: None
exec_prefix: None
home: None
user: 0
install_base: None
install_platbase: None
root: None
install_purelib: None
install_platlib: None
install_lib: None
install_headers: None
install_scripts: None
install_data: None
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
install_layout: None
old_and_unmanageable: None
single_version_externally_managed: None
post-finalize_{unix,other}():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: $base/local/lib/python$py_version_short/dist-packages
install_platlib: $platbase/local/lib/python$py_version_short/dist-packages
install_lib: None
install_headers: $base/local/include/python$py_version_short/$dist_name
install_scripts: $base/local/bin
install_data: $base/local
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
install_layout: None
old_and_unmanageable: None
single_version_externally_managed: None
post-expand_basedirs():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: $base/local/lib/python$py_version_short/dist-packages
install_platlib: $platbase/local/lib/python$py_version_short/dist-packages
install_lib: None
install_headers: $base/local/include/python$py_version_short/$dist_name
install_scripts: $base/local/bin
install_data: $base/local
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
install_layout: None
old_and_unmanageable: None
single_version_externally_managed: None
config vars:
{'base': '/usr',
'dist_fullname': 'davitpy-0.2',
'dist_name': 'davitpy',
'dist_version': '0.2',
'exec_prefix': '/usr',
'platbase': '/usr',
'prefix': '/usr',
'py_version': '2.7.6',
'py_version_nodot': '27',
'py_version_short': '2.7',
'sys_exec_prefix': '/usr',
'sys_prefix': '/usr',
'userbase': '/home/sean/.local',
'usersite': '/home/sean/.local/lib/python2.7/site-packages'}
post-expand_dirs():
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: /usr/local/lib/python2.7/dist-packages
install_platlib: /usr/local/lib/python2.7/dist-packages
install_lib: None
install_headers: /usr/local/include/python2.7/davitpy
install_scripts: /usr/local/bin
install_data: /usr/local
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
install_layout: None
old_and_unmanageable: None
single_version_externally_managed: None
after prepending root:
prefix: /usr
exec_prefix: /usr
home: None
user: 0
install_base: /usr
install_platbase: /usr
root: None
install_purelib: /usr/local/lib/python2.7/dist-packages
install_platlib: /usr/local/lib/python2.7/dist-packages
install_lib: /usr/local/lib/python2.7/dist-packages/
install_headers: /usr/local/include/python2.7/davitpy
install_scripts: /usr/local/bin
install_data: /usr/local
compile: None
compile: True
optimize: None
force: None
skip_build: 0
record: None
install_layout: None
old_and_unmanageable: None
single_version_externally_managed: None
Distribution.get_command_obj(): creating 'build' command object
setting options for 'easy_install' command:
Distribution.get_command_obj(): creating 'install_lib' command object
Distribution.get_command_obj(): creating 'install_scripts' command object
running bdist_egg
Distribution.get_command_obj(): creating 'bdist_egg' command object
Distribution.get_command_obj(): creating 'egg_info' command object
Distribution.get_command_obj(): creating 'bdist' command object
running egg_info
running build_src
Distribution.get_command_obj(): creating 'build_src' command object
Distribution.get_command_obj(): creating 'build_ext' command object
build_src
building extension "dmapio" sources
building extension "aacgm" sources
building extension "tsygFort" sources
f2py options: []
adding 'build/src.linux-x86_64-2.7/fortranobject.c' to sources.
adding 'build/src.linux-x86_64-2.7' to include_dirs.
adding 'build/src.linux-x86_64-2.7/models/tsyganenko/tsygFort-f2pywrappers.f' to sources.
building extension "hwm07" sources
f2py options: []
adding 'build/src.linux-x86_64-2.7/fortranobject.c' to sources.
adding 'build/src.linux-x86_64-2.7' to include_dirs.
building extension "msisFort" sources
f2py options: []
adding 'build/src.linux-x86_64-2.7/fortranobject.c' to sources.
adding 'build/src.linux-x86_64-2.7' to include_dirs.
adding 'build/src.linux-x86_64-2.7/models/msis/msisFort-f2pywrappers.f' to sources.
building extension "igrf" sources
f2py options: []
adding 'build/src.linux-x86_64-2.7/fortranobject.c' to sources.
adding 'build/src.linux-x86_64-2.7' to include_dirs.
building extension "iri" sources
f2py options: []
adding 'build/src.linux-x86_64-2.7/fortranobject.c' to sources.
adding 'build/src.linux-x86_64-2.7' to include_dirs.
adding 'build/src.linux-x86_64-2.7/models/iri/iri-f2pywrappers.f' to sources.
build_src: building npy-pkg config files
writing davitpy.egg-info/PKG-INFO
writing top-level names to davitpy.egg-info/top_level.txt
writing dependency_links to davitpy.egg-info/dependency_links.txt
Distribution.get_command_obj(): creating 'build_py' command object
reading manifest file 'davitpy.egg-info/SOURCES.txt'
include_pattern: applying regex r'^davitpy.egg-info/.[^/]\Z(?ms)'
adding davitpy.egg-info/PKG-INFO
adding davitpy.egg-info/not-zip-safe
adding davitpy.egg-info/top_level.txt
adding davitpy.egg-info/SOURCES.txt
adding davitpy.egg-info/dependency_links.txt
exclude_pattern: applying regex r'^build/.'
removing build/src.linux-x86_64-2.7/fortranobject.h
removing build/src.linux-x86_64-2.7/models/iri/iri-f2pywrappers.f
removing build/src.linux-x86_64-2.7/fortranobject.c
removing build/src.linux-x86_64-2.7/models/iri/irimodule.c
removing build/src.linux-x86_64-2.7/fortranobject.h
removing build/src.linux-x86_64-2.7/fortranobject.c
removing build/src.linux-x86_64-2.7/models/igrf/igrfmodule.c
removing build/src.linux-x86_64-2.7/fortranobject.h
removing build/src.linux-x86_64-2.7/models/msis/msisFort-f2pywrappers.f
removing build/src.linux-x86_64-2.7/fortranobject.c
removing build/src.linux-x86_64-2.7/models/msis/msisFortmodule.c
removing build/src.linux-x86_64-2.7/fortranobject.h
removing build/src.linux-x86_64-2.7/fortranobject.c
removing build/src.linux-x86_64-2.7/models/hwm/hwm07module.c
removing build/src.linux-x86_64-2.7/fortranobject.h
removing build/src.linux-x86_64-2.7/models/tsyganenko/tsygFort-f2pywrappers.f
removing build/src.linux-x86_64-2.7/fortranobject.c
removing build/src.linux-x86_64-2.7/models/tsyganenko/tsygFortmodule.c
exclude_pattern: applying regex r'^davitpy-0.2/.'
exclude_pattern: applying regex r'/(RCS|CVS|.svn)/'
writing manifest file 'davitpy.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
setting options for 'install_lib' command:
running install_lib
running build_py
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using build_ext
building 'hwm07' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC

compile options: '-Ibuild/src.linux-x86_64-2.7 -I/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include -I/usr/include/python2.7 -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/models/hwm/hwm07module.c
In file included from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/ndarraytypes.h:1761:0,
from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/ndarrayobject.h:17,
from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/arrayobject.h:4,
from build/src.linux-x86_64-2.7/fortranobject.h:13,
from build/src.linux-x86_64-2.7/models/hwm/hwm07module.c:18:
/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
build/src.linux-x86_64-2.7/models/hwm/hwm07module.c:143:12: warning: ‘f2py_size’ defined but not used [-Wunused-function]
static int f2py_size(PyArrayObject* var, ...)
^
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/fortranobject.c
In file included from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/ndarraytypes.h:1761:0,
from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/ndarrayobject.h:17,
from /usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/arrayobject.h:4,
from build/src.linux-x86_64-2.7/fortranobject.h:13,
from build/src.linux-x86_64-2.7/fortranobject.c:2:
/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
compiling Fortran 90 module sources
Fortran f77 compiler: /usr/bin/gfortran -Wall -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: /usr/bin/gfortran -Wall -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: /usr/bin/gfortran -Wall -ffixed-form -fno-second-underscore -Wall -fno-second-underscore -fPIC -O3 -funroll-loops
compile options: '-Ibuild/src.linux-x86_64-2.7 -I/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include -I/usr/include/python2.7 -c'
extra options: '-Jbuild/temp.linux-x86_64-2.7/ -Ibuild/temp.linux-x86_64-2.7/'
gfortran:f90: models/hwm/apexcord.f90
models/hwm/apexcord.f90:71.8:

use apexcord
    1

Fatal Error: Cannot read module file 'apexcord.mod' opened at (1), because it was created by a different version of GNU Fortran
models/hwm/apexcord.f90:71.8:

use apexcord
    1

Fatal Error: Cannot read module file 'apexcord.mod' opened at (1), because it was created by a different version of GNU Fortran
Traceback (most recent call last):
File "setup.py", line 66, in
"Programming Language :: Python"
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/command/install.py", line 59, in run
r = self.setuptools_run()
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/command/install.py", line 53, in setuptools_run
self.do_egg_install()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 88, in do_egg_install
self.run_command('bdist_egg')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 185, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 171, in call_command
self.run_command(cmdname)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install_lib.py", line 21, in run
self.build()
File "/usr/lib/python2.7/distutils/command/install_lib.py", line 111, in build
self.run_command('build_ext')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/command/build_ext.py", line 234, in run
self.build_extensions()
File "/usr/lib/python2.7/distutils/command/build_ext.py", line 446, in build_extensions
self.build_extension(ext)
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/command/build_ext.py", line 364, in build_extension
depends=ext.depends)
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/ccompiler.py", line 202, in CCompiler_compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/distutils/fcompiler/init.py", line 610, in _compile
raise CompileError(msg)
distutils.errors.CompileError: Command "/usr/bin/gfortran -Wall -fno-second-underscore -fPIC -O3 -funroll-loops -Ibuild/src.linux-x86_64-2.7 -I/usr/local/lib/python2.7/dist-packages/numpy-1.8.1-py2.7-linux-x86_64.egg/numpy/core/include -I/usr/include/python2.7 -c -c models/hwm/apexcord.f90 -o build/temp.linux-x86_64-2.7/models/hwm/apexcord.o -Jbuild/temp.linux-x86_64-2.7/ -Ibuild/temp.linux-x86_64-2.7/" failed with exit status 1

Beam 8 ends up pointing down the boresight when it should be beam 7.5; EDIT: this problem is not present in RST, Go, or DaViT. The entire field of view is rotated half a beam (1.62 degrees) to the left. The method beamToAzim in class fov (pydarn/radar/radStruct.py) does the job correctly but is not being used.

Fixed in #70.

Hi,
Just trying to install the tools on my Mac. I've done the github pull, but when I run the mastermake command it fails because it cannot find the makefile in the igrf, raydarn directories. I'm particularly interested in the AAGCM portion of the tools. Is it possible to get the makefile for these models?

Thanks,
Mike

Hi, guys. Here is another bug.

Summary: SHA 74f66c9 breaks access to VT SFTP.

Test code:

import datetime
import os
import matplotlib.pyplot as plt
import pydarn

#result = pydarn.radar.radInfoIo.hdwRead('hdw.dat.bks')

sTime = datetime.datetime(2008,2,22)
eTime = datetime.datetime(2008,2,23)
radar = 'bks'
beam  = 7

### Remote File RTI Plots
fig = plt.figure(figsize=(14,12)) #Define a figure with a custom size.
pydarn.plotting.rti.plotRti(sTime, radar, eTime=eTime, bmnum=beam, figure=fig)
plt.show()

Output Using SHA 74f66c9:

w2naf@w2naf-VirtualBox:~$ ./test.py 
Radars information has been updated.

Looking locally for fitex files with radcode: bks channel: a
Error, no file name formats containing channel exists!

Looking on the remote SFTP server for fitex files
Error, no file name formats containing channel exists!

Sorry, we could not find any data for you :(
error, your pointer does not point to any data
error, no data available for the requested time/radar/filetype combination

Output using previous commit (SHA e5e9dbc):

w2naf@w2naf-VirtualBox:~$ ./test.py 
Radars information has been updated.

Looking locally for fitex files with radcode: bks channel: a
[Errno 2] No such file or directory: '/sd-data/2008/fitex/bks/'
There was a problem reading local data, perhaps you are not at VT?
Will attempt fetching data from remote.

Looking on the remote SFTP server for fitex files
found fitex data on sftp server
Concatenating all the files in to one
cat /tmp/sd/20080222.0002.00.bks.fitex /tmp/sd/20080222.0202.00.bks.fitex /tmp/sd/20080222.0402.00.bks.fitex /tmp/sd/20080222.0602.00.bks.fitex /tmp/sd/20080222.0802.00.bks.fitex /tmp/sd/20080222.1002.00.bks.fitex /tmp/sd/20080222.1202.00.bks.fitex /tmp/sd/20080222.1402.00.bks.fitex /tmp/sd/20080222.1602.00.bks.fitex /tmp/sd/20080222.1802.00.bks.fitex /tmp/sd/20080222.2002.00.bks.fitex /tmp/sd/20080222.2202.00.bks.fitex /tmp/sd/20080223.0000.11.bks.fitex > /tmp/sd/20080222.000000.20080223.000000.bks.a.fitex
rm /tmp/sd/20080222.0002.00.bks.fitex
rm /tmp/sd/20080222.0202.00.bks.fitex
rm /tmp/sd/20080222.0402.00.bks.fitex
rm /tmp/sd/20080222.0602.00.bks.fitex
rm /tmp/sd/20080222.0802.00.bks.fitex
rm /tmp/sd/20080222.1002.00.bks.fitex
rm /tmp/sd/20080222.1202.00.bks.fitex
rm /tmp/sd/20080222.1402.00.bks.fitex
rm /tmp/sd/20080222.1602.00.bks.fitex
rm /tmp/sd/20080222.1802.00.bks.fitex
rm /tmp/sd/20080222.2002.00.bks.fitex
rm /tmp/sd/20080222.2202.00.bks.fitex
rm /tmp/sd/20080223.0000.11.bks.fitex

reached end of data
plotting took: 0:00:20.414270

Hello all,

I've been picking up a few of the updates in the sdio directory in order to see if they work better with the UAF filenaming convention. I've checkout'ed the radDataRead and radDataTypes and fetchUtils.py files from the development branch. First, is there a better place to pull from? I think this is where some work was being done to make some updates.

If so, then I'm getting errors when trying to read in data. Seems like it's breaking in the radDataTypes' readRec() function at:

#do this until we reach the requested start time

and have a parameter match

while(1):
offset=pydarn.dmapio.getDmapOffset(self.__fd)
dfile = pydarn.dmapio.readDmapRec(self.__fd)

Looking in the development dmapio's init.py file in the development branch, I don't see much there. Doing a search of getDmap in this repository doesn't yield any results either. Is there somewhere that this code is hiding or is this still a work in progress?

When running mastermake after building successfully RSTLite and basemap and setting appropriate environment variables, mastermake fails at the following lines. I have no solution.

mpif90 -w -O2 -fbacktrace -g -fno-automatic -c raytrace_mpi.f90 -o raytrace_mpi.o
raytrace_mpi.f90:264.55:

types = (/MPI_REAL, MPI_INTEGER, MPI_REAL, MPI_CHAR/)
                                                   1
Error: Symbol 'mpi_char' at (1) has no IMPLICIT type
make: *** [raytrace_mpi.o] Error 1

The line in question is the one that sets up the source line in the .bashrc file after the dependencies have been installed. ie) echo "source $dir/../profile.bash" >> ~/.bashrc

The path is incorrectly set and so profile.bash is not properly sourced.

This can be fixed in two ways: 1) removing all environment variables by moving things over to a configuration file like what matplotlib uses or 2) properly setting the path to the profile.bash file.

If I have a directory structure with no time keys in it, e.g., /home/matt/mapfiles/, fetchUtils will not find files there because of the way time key searching (lines 203 to 231 in develop) is set up. An ls is only run on the directory if a time key is found in the directory format. If the directory format is being set manually anything should be possible, and if for some reason that isn't allowed a check needs to be added because the current situation produces a somewhat cryptic error.

We are working on hdf5 representations of fitted SD data products.
hdf5 files can not be easily concatted.
So for pydavit to support it the code calling file fetch routines must be enhanced to keep a list of open files, instead of relying on a single open file.

What is needed in RadDataType:

1. private file handle related variables need to become a list of variables
2. Record/Scan Index methods would need to track both offset and file object pointer.
3. Record/Scan Index would need to be extended to use an abstracted offset concept, for different file types...structures files let hdf5 and netcdf would not use binary offsets as a record locator...just index offsets.
4. for netcdf and hdf5 you'd have to require Index methods to fire on init and uss the indexes when iterating over the radDataPtr object.
5. grow an hdf5 object to hang off of the beam record object.

Posting this so we remember to address it at a later date. As @jspaleta said we need to work towards replacing rtypes.h with "calls to types defined in stdint.h" see #44

And see #44 (comment)