ncar / aerosol-workshop-driver Goto Github PK

add netcdf to Dockerfile and CMakeLists.txt

Add an almost_equals() utility function in each language to apply a tolerance to floating-point comparisons in all tests.

In the Great Reorganization, we renamed src/ai_util.F90 to src/aero_util.F90, but we didn't rename the Fortran module. Not a big deal at the moment, but we should fix this when it makes sense.

create mock aerosol models in C/C++/Fortran to demonstrate how an aerosol package can extend the draft interface

The last thing the driver does before breaking everything down is it plots aerosol optics host data, which lives in a set of 1D arrays whose indices identify wavelengths. This should be straightforward, but involves a decision on a plotting package (matplotlib, anyone?)

Finish implementing the Array class in C++, C, and Fortran. Arrays will store a 1D array of floating-point numbers of a precision specified as a CMake option. Internal array storage can be application specific, but an Array must be able to copy-in/copy-out to 1D arrays of primitive type floating point numbers of the same precision in C++, C, and Fortran.

Arrays can be constructed from:

• the number of array elements, and an optional initial value that defaults to 0
• a 1D array of primitive type floating point numbers in the native language

Allow aerosol model developers to register their model with the factory at build time and move factory functions to interface library code.

One of our participants is trying to get NetCDF4 working, and noticed that the following libraries are required (excluding the Fortran netcdff library for Fortran programs):

-lnetcdf -lhdf5_hl -lhdf5 -lm -ldl -lz -lcurl

This means we need to handle HDF5 libraries, the libz compression library, and libcurl.

Finish Grid class in each language to represent 1D grids. Grids comprise a set of contiguous sections along the grid axis. Sections are defined by a lower bound (inclusive) and an upper bound (exclusive). The points along the grid axis that correspond to the lowermost section's lower bound, the shared bounds between adjacent sections, and the uppermost grid section's upper bound are referred to as the grid interfaces. The set of sections and interfaces are always treated in ascending order.

Grids may be constructed from:

• a lower overall bound, a common grid width (upper bound - lower bound), and a number of sections.
• a set of interfaces

Grids must provide:

• the number of sections
• the number of interfaces
• an Array object holding geometric mid-point values for each section
• an Array object holding interface values
• the geometric mid-point value at a given section index
• the interface value at a given interface index
• equality operator based on set of sections and their bounds
• clone (deep-copy) function
• finalize function

All indexing must follow native language conventions. Adjustments to indices should be made in bridge functions.

The base Grid class in C++, C, and Fortran will use an Array object to hold the array of interface values that define the grid.

We need a slide deck with some high-level ideas:

• circles and arrows identifying host and aerosol models and how they communicate
• basic data structures and their roles
• an animated walkthrough of what the driver does and the data structures it uses
• a cheat sheet with terminology and API essentials
• ???

Finish the documentation with full instructions for adding a new aerosol model, running the tests, and analyzing results.

Add an Interpolator class in C++, C, Fortran. An interpolator is used to interpolate data in an Array from one Grid to another.

Instances of Interpolator are constructed from:

• a source Grid and a destination Grid

An Interpolator object can take data from an Array of the dimensions of the source Grid and interpolate them onto an Array of the dimensions of the destination Grid according to the Interpolator's interpolation algorithm.

Compiling on MacOS 12.3 gives warnings related to some of the NetCDF function calls:

/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:145:15: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
    if(retval = nc_open(description_file, NC_NOWRITE, &ncid)) ERR(retval);
       ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:145:15: note: place parentheses around the assignment to silence this warning
    if(retval = nc_open(description_file, NC_NOWRITE, &ncid)) ERR(retval);
              ^
       (                                                    )
/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:145:15: note: use '==' to turn this assignment into an equality comparison
    if(retval = nc_open(description_file, NC_NOWRITE, &ncid)) ERR(retval);
              ^
              ==
/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:146:15: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
    if(retval = nc_close(ncid)) ERR(retval);
       ~~~~~~~^~~~~~~~~~~~~~~~
/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:146:15: note: place parentheses around the assignment to silence this warning
    if(retval = nc_close(ncid)) ERR(retval);
              ^
       (                      )
/Users/mattdawson/git-repos/aerosol-workshop-driver/my_model/template/c/my_model.c:146:15: note: use '==' to turn this assignment into an equality comparison
    if(retval = nc_close(ncid)) ERR(retval);
              ^
              ==
2 warnings generated.

Similar warnings are generated for/my_model/template/cpp/my_model/cpp

Set up basic driver/interface for Fortran and C++, with simple API. Add CMake and Docker configurations. Set up testing with GH Actions

Radiation packages should be able to indicate which optical properties they are interested in:

• total optical depth (tau)
• scattering optical depth (tau * omega)
• asymmetric scattering optical depth (tau * omega * g)
• any number of scattering phase functions

Add valgrind to all tests to ensure there are no memory leaks

For the hackathon, we need to give participants a "problem of interest." Then they will construct a state representation for their model that captures "the essentials," whatever that means. @bnmurphy has assembled some data that can be used to construct a problem description, but we should make sure that the description has sufficient detail to serve as a starting point for our driver/optics exercise.

@jeff-cohere @bnmurphy - what do each of our organizations want in terms of copyright/licensing? NCAR typically uses Apache 2.0, but I don't think there is any problem with using another license if EPA or DOE has stricter rules. I can also include each organization in the copyright statement if you send me the text you usually use.

Remove boilerplate comments from headers as well.

Compiling either with or without netcdf, compilation succeeds but errors like the one listed below appear when "make test". This was reproducible with intel versions 19.1.3.304, 2021.3.0, and 2021.4.0.

./fortran_driver "my model" ../my_model/my_model_data.nc
*** Error in `./fortran_driver': malloc(): memory corruption: 0x000000000077ad40 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x740ef)[0x2aaaabd920ef]
/lib64/libc.so.6(+0x79646)[0x2aaaabd97646]
/lib64/libc.so.6(+0x7ba3f)[0x2aaaabd99a3f]
/lib64/libc.so.6(__libc_malloc+0x50)[0x2aaaabd9b3d0]
./fortran_driver(_mm_malloc+0x20)[0x4b8bd0]
./fortran_driver[0x45594a]
./fortran_driver[0x40c3e7]
./fortran_driver[0x406df5]
./fortran_driver[0x404822]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2aaaabd3e725]
./fortran_driver[0x404729]
======= Memory map: ========
00400000-0050c000 r-xp 00000000 00:39 18468548                           /gpfsm/dnb53/ktsigari/git_projects/aerosol-workshop-driver/build/fortran_driver
0070c000-0070d000 r--p 0010c000 00:39 18468548                           /gpfsm/dnb53/ktsigari/git_projects/aerosol-workshop-driver/build/fortran_driver
0070d000-00711000 rw-p 0010d000 00:39 18468548                           /gpfsm/dnb53/ktsigari/git_projects/aerosol-workshop-driver/build/fortran_driver
00711000-0079b000 rw-p 00000000 00:00 0                                  [heap]
2aaaaaaab000-2aaaaaacc000 r-xp 00000000 fe:01 33575023                   /lib64/ld-2.22.so
2aaaaaacc000-2aaaaaad0000 r--p 00000000 00:00 0                          [vvar]
2aaaaaad0000-2aaaaaad2000 r-xp 00000000 00:00 0                          [vdso]
2aaaaaad2000-2aaaaaad5000 rw-p 00000000 00:00 0
2aaaaaaf6000-2aaaaaafc000 rw-p 00000000 00:00 0
2aaaaaccc000-2aaaaaccd000 r--p 00021000 fe:01 33575023                   /lib64/ld-2.22.so
2aaaaaccd000-2aaaaacce000 rw-p 00022000 fe:01 33575023                   /lib64/ld-2.22.so
2aaaaacce000-2aaaaaccf000 rw-p 00000000 00:00 0
2aaaaaccf000-2aaaaae26000 r-xp 00000000 00:3f 16241449                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libirng.so
2aaaaae26000-2aaaab025000 ---p 00157000 00:3f 16241449                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libirng.so
2aaaab025000-2aaaab026000 r--p 00156000 00:3f 16241449                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libirng.so
2aaaab026000-2aaaab039000 rw-p 00157000 00:3f 16241449                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libirng.so
2aaaab039000-2aaaab071000 r-xp 00000000 00:3f 15773754                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5
2aaaab071000-2aaaab271000 ---p 00038000 00:3f 15773754                   /gpfsm/dulocal/sles12/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5

Currently, our drivers contain mostly commented-out code. After we finish the mock aerosol model code, we can uncomment the code, make any necessary changes, and run it to prove to ourselves that it works.

This probably also involves calling a plotting routine to draw some graphics.

1. Do we compute optical properties on grid interfaces or midpoints? CAM computes on grid interfaces, so maybe let's do that.
2. Does our linear interpolator recover linear and piecewise linear functions exactly?

PR #39 may not have fixed the entire problem of build errors due to unfound .mod files from Fortran source. It did fix the problem of looking for *.mod0 files, but I'm still seeing the same kind of error:

Error copying Fortran module "tests/array/test_array_helper.mod". Tried "tests/array/TEST_ARRAY_HELPER.mod" and "tests/array/test_array_helper.mod".

A solution that has worked so far has been to switch to a serial build, i.e., use make instead of make -j. I'll try to debug more tomorrow.

Just wanted to document the work-around (serial make) in case participants encounter this issue before I sort it out.