Is anyone else having recurring issues with accessing https://jupyterhub.ucar.edu/ch ?

I feel like every time there is a glitch with Cheyenne, it takes until the next Monday morning for the jupyter hub to start working again (which is evening here). I'm trying to move all my analysis to Jupyter, but it doesn't seem like there is an automated way of restarting it after the computer goes down, which makes it quite a lot less useful.

Is anyone else having this issue?

@jhamman can you help with providing an example for how to use DASK to handle computationally expensive jobs. I got the for loop to calculated SVD in the script below (on my fork), but it's pretty slow for a single variable and single model. Is there a slick way to use DASK to speed this up?

notebook also available in /home/wwieder/python/ctsm_py/notebooks/.

https://github.com/wwieder/ctsm_py/blob/master/notebooks/MonthlySVD.ipynb
P.S., if the notebook renders can you find out why it's printing so much output to the screen?

Finally, also good to mask out grids w/ zero GPP (e.g. Antarctic)?

https://docs.google.com/document/d/1pESHA6PTJxS_rRgfW3zwOJAG8UCGXiW_btPdFEqG_cw/edit?usp=sharing

We need to develop a function to transform vector history files (column or patch data) to 4d gridded output (patch, time, lat, lon)

This is a good example of an NCL script that would be helpful to transfer over the a jupyter notebook. @olyson, is there a newer version of this around?

/glade/u/home/wwieder/NCL_code/SpinupStability_v7.ncl

how do users get utils loaded into their environment?

We need to develop a script that handles concatenating a variable (or variables) from monthly history files for looking at results before they've been converted to single variable time series.

xarray reads in the first month of CESM output as Feb 1, not January. One example of a work around is provided here, but maybe there's a better approach @jhamman ?

Can we use existing tools for regridding multi-model ensembles (e.g. LUMIP)

how do we make panel plots efficiently?

Examples of how to use catalogues to input data and operate on multiple models / ensemble members.

From briefly surveying the group, it seems that there are several approaches to calculating weighted regional and global means of land model output. Moving to python provides an opportunity to standardize the approach.

This issue was inspired by the area_wgt variable in SimpleExample.ipynb (is there a better way to reference specific lines in a notebook on github?), and the desire to replicate the behavior of NCL functions like wgt_areaave and wgt_areaave2.

I've been using the "ctypes" python library to run compiled fortran code from python.

Here is documentation on ctypes: https://docs.python.org/3/library/ctypes.html

My workflow:

1. use a build script (bash, make, etc) to compile a set of fortran modules into shared objects.
2. ctypes python procedures can then be called in a python script to associate each of those shared fortran objects as a callable python type. Its like loading the modules and saving them as a python class.
3. The ctypes objects can then be used to call the fortran procedures, the arguments to the fortran procedures will be re-cast as special "ctypes" data types, with equivalents for doubles, ints, and ways to handle vector arguments. You can handle dynamic fortran memory too, which will be released when the python script exits.

I typically use this type of workflow to perform "scientific" unit testing. For instance, visualizing allometry, allocation, and water retention curves in FATES.

Notes:

1. I've been happily surprised that this workflow CAN actually support OOP (object oriented programming), and I've been able to use python to allocate fortran class structures, call those structures, and analyze results.

2. I've also developed some python tools that read in a CDL version of our FATES parameter files and save the parameters in a python derived type.

Here is a link to the current folder with all this stuff in FATES (although I have some updates coming down the pipe soon): https://github.com/NGEET/fates/tree/master/functional_unit_testing

If anyone has a simple test case they would like to try, feel free to reach out to me with the fortran module and we could try crafting up a way to call if via python.

Hi, all
I used the below code to read output from CLM and CESM, but I found sth wired.
ds_exp = import_case(exp, myVars=variables, timeSlice=time_slice, alter_lons=True)

for CESM, the time slot seems correct

for CLM, the time slot sees has 1 month misalignment

Is there a distinction between which tools should go here vs. in CTSM's tools/contrib directory (https://github.com/ESCOMP/CTSM/tree/release-clm5.0/tools/contrib)? Currently, most of the tools there seem to be for pre-processing / setting up cases rather than post-processing, though one exception is https://github.com/ESCOMP/CTSM/blob/release-clm5.0/tools/contrib/SpinupStability.ncl . Is that a good distinction (at least for starters)?

I don't have strong feelings on this myself, but would be interested in what others think - @wwieder @olyson @ekluzek

It would be nice to have examples for creating weighted sums & time series for:

• Regions (for specified bounding boxes)
• Arbitrary areas (e.g. shape files)

This has to be a common function. at last try the esmlab wrapper wasn't working. Maybe GeoCAT has something?

Weighting and visualizing vertically resolved data (e.g. by soil layer)