rupertgladstone / fisoc Goto Github PK

How do we want to implement thermodynamic coupling? Do we want the ISM to pass a heat flux to the OM? Implement this. How important is it (i.e. how bad are the assumptions the OM currently makes when implementing the thermodynamics?)?

Data are currently exchanged between the OM and ISM on a 2d grid/mesh in 2d space. In reality the exchange occurs on a surface (well, leaving aside "mushy" layers...) in 3D space. ESMF is capable of allowing us to use 2D grids/meshes in 3D space, but using this would increase complexity of the regridding operations.
The vertical coords of the grid would need to be updated during the run.
The regridRouteHandle (an ESMF object currently created only during initialisation) would also need to be updated during the run.
How bad is the current simplification? Is it good enough that we don't need to implement the third dimension?

I have recently encountered some errors when using strict compiler flags. By "strict", I mainly mean the floating point flags (divide by zero etc), but also we need to make sure that all components compile and run ok with bounds checking and different optimization levels. I will investigate these issues more closely after a further Elmer merge.

Note the FISOC build inherits the ESMF FFLAGS: I need to modify the FISOC makefile info statement to incorporate this (doing this now (mid Jan 2018) in devel). The implication is that currently if ESMF is compiled with fp flags then these are inherited by FISOC and will cause crashes at run time. For now we compile without these flags but in the long run we aim to identify the cause of the errors and tidy the code.

Rupert

Tidy the FISOC branch in the Elmer github repository so that it can be merged into the main Elmer/Ice branch. Then FISOC can more easily benefit from all new code development in Elmer.

We should implement some standard tests that can easily be run through a test script. We should do this AFTER the alpha version is up and running.

The tests should cover both serial and parallel builds for all available components (ROMS, OM_dummy, Elmer, Pattyn, ISM_dummy).

All variables in the OM2ISM list need to be contained in the union of the ReqVars and DerVars lists. There is currently no check for this, which means that a typo can lead to a variable not being passed from OM to ISM and no warning being given. A sanity check is needed for this in FISOC_OM. Probably the same issue exists on the ISM side and a check needs to be added there too.

Noticed some subroutine arguments without intent statements in FISOC_OM.f90. Need to check over FISOC - all arguments should have intent statements. (additional note: add this to coding standards in FISOC manual, along with no implicit variables)

Build a list of valid config options, give a warning about unrecognised config options (not fatal in general because ppl might need to add their own). Add list location to manual. Can use same list in emacs mode for editing config file.

FISOC currently assumes that the OM fields will be defined on an ESMF_grid object and the ISM fields will be defined on an ESMF_mesh object. Will future components require some flexibility in this? In particular, what type of grid/mesh should BISICLES use?

gets written to default fortran filename instead of name defined in FISOC config file.

Elmer fields import/export

Work with Eric Larour and Helene Seroussi to implement ISSM coupling option.

New wrapper for Pattyn ISM, add the ISM itself to FISOC repo, map flowline vars onto 2D (unit width) grid/mesh.
To be used for testing/verifying coupling/regridding. It will also be implemented directly into ROMS.

Write a new wrapper for the ISM such that it reads in forcing data which are then passed to the OM at run time. This provides a way of testing some of FISOC's features without the flexibility of full coupling. It may also be a convenient way to force the ROMS OM.

Implement following options:
OM_cav_Simple OM uses most recent ISM cavity geom directly
OM_cav_Rate OM uses ISM rate of cavity change (from 2 most recent ISM steps) to evolve OM cavity
OM_cav_RateCorr As above but with drift correction
OM_cav_Linterp OM uses linear interpolation in time (from 2 most recent ISM steps)

Note: rates assume ISM lags OM temporally. Linterp assumes the opposite (i.e. ISM is one step ahead).

Integer type isnt big enough for expressing run lengths in seconds. The following error is reported in the logs.

0161121 160746.960 INFO PET0 Running with ESMF Version 6.3.0rp1
20161121 160746.968 INFO PET0 FISOC_caller.f90:42 Initialised ESMF framework
20161121 160746.971 INFO PET0 FISOC_caller.f90:61 Loaded FISOC configuration file
20161121 160746.974 INFO PET0 FISOC_caller.f90:81 Completed FISOC_parent creation and registration
20161121 160746.975 INFO PET0 FISOC_utils.f90:313 used runLength_ISM_steps to set the end time
20161121 160746.975 WARNING PET0 ESMCI_Clock.C:1692 ESMCI::Clock::validate() timeStep positive for negative startTime to stopTime range).
20161121 160746.975 ERROR PET0 ESMCI_Clock.C:159 ESMCI_ClockCreate(new) Value out of range Internal subroutine call returned Error
20161121 160746.976 ERROR PET0 ESMF_Clock.F90:520 ESMF_ClockCreateNew() Value out of range - Internal subroutine call returned Error
20161121 160746.976 ERROR PET0 FISOC_utils.f90:324 Value out of range - Passing error in return code

Rupert's original response to this problem:
This seems to be a large integer problem. FISOC calculates the run length in seconds as num ISM timesteps * ISM timestep size in seconds. But a standard integer isn't big enough. Not sure Int8 is big enough either, but esmf interfaces expect int4 so it wouldn't help...

2 workarounds:

1. Use fewer timesteps.
2. Go back to previous method of specifying start and end time.

Is there a defensive compiler flag that will cause run time error when this happens? Currently it just sticks junk into run length when the int is too big, which is not very helpful.

Suggestions welcome. Probably best thing is to modify runlength code to use days instead of secs. Might need some rounding, and for testing the user might need secs as an option... so maybe putting both days and secs as options controlled by config file?

Main time stepping control should be in caller, not parent. Not a functional change, but will be required if FISOC is ever embedded in a larger ESMF application.

States should be created in generic wrappers not by parent (remove "saved" states from parent module). Tidying issue, not functional.

We currently implement "tight" coupling where we simply specify a ratio that determines the number of OM timesteps per ISM timestep.
An additional mode we intend to implement is a long timescale mode, where the OM is run to approx. steady state, but is only called infrequently, after a "significant" change in the cavity geometry. So we have to program FISOC to decide when a significant change has occurred.

Add an example FISOC_config.rc file for the dummy-dummy case to test FISOC install. Some of the necessary parameters (OM_outputInterval, possibly others) could have set default values as well.

FISOC_Ex1 runs with a mesh of quads but regridding crashes with mesh of triangles. Suspect unique nodes issue, or node ordering issue, fix in progress.

Currently there are some physical parameters (e.g. uniform density of the ice sheet/shelf) that are set independently in both ISM and OM components. Should we instead set these through the FISOC config file and overwrite (and or check, maybe give a warning) at run time whatever the components currently set independently?

The log files (from ESMF) are rather unwieldy in large parallel runs. Add options to suppress (all but one of) them or dump them out of the way in a user defined sub directory.

ROMS, when closed boundaries are used, gives zero melt in cells at the edge of the domain. Probably due to zero u and v velocities. We should either modify ROMS to allow non-zero melt to be calculated or modify FISOC to allow cells at the edge of the domain (not the edge of partitions) to be populated by some kind of nearest neighbour approach (possibly apply a mask to them in ROMS so FISOC know which ones to populate - perhaps by an ESMF utilty?).

Time information centrally is managed by FISOC accessing the run time config file and using this information to set an ESMF clock and alarms. But it is not clear whether the model specific wrappers should use global vars, subroutine arguments or direct access to the FISOC clock when they need time information. Probably making the FISOC_clock globally available is the cleanest, but consistency would be beneficial in any case.

ESMF_FieldRegridStore is called during coupler initialisation in between phases 1 and 2 of gridded component initialisation.

I view this as a weakness of ESMF_FieldRegridStore. Intended fix: write a wrapper in the FISOC repo for ESMF_FieldRegridStore in which field data are saved prior to and written back in after the ESMF_FieldRegridStore call.

Currently FISOC imports the Elmer mesh in serial runs. Needs to function in parallel runs.

Currently the ISM run method is called first then the OM run method, with a sequential model. If we stick with sequential it might be best to run the OM first since FISOC may calculate the geometry change rate to be passed to the OM based on the geometry from the ISM in the last 2 timesteps. This means that, with the current ordering, the change rate will alsways be zero the first time the OM is called. We could reverse the OM/ISM ordering or we could make the order a user choice at runtime.