trondkr / model2roms Goto Github PK
View Code? Open in Web Editor NEWModel2roms is a Python toolbox for creating forcing files required to run Regional Ocean Modeling System (ROMS)
License: MIT License
Model2roms is a Python toolbox for creating forcing files required to run Regional Ocean Modeling System (ROMS)
License: MIT License
Good Morning brow,
Thank you for your excelent job. This code is amazing.
I have a one simple question about your code. In...
def define_subset_for_indata(self):
# Subset the input data. The more you subset the less memory is needed for calculations
# and the faster the process is performed. The subset is initially performed in IOsubset.py
subset = np.zeros(4)
if self.outgrid_name == "NS8KM":
return subset[40, 70, -30, 40]
elif self.outgrid_name == "A20":
return subset[30, 90, -179, 360]
else:
What is the coordinates order? south, north, west, east?
Best regards.
Hi @trondkr ,
We are having problems running m2r.
The script stops before the second time step (monthly and daily)
@uesleisutil and I used your script to create ini and bry with glorys data.
The problem is that:
when we ran with your old script the u,v,ubar and vbar variables appear skewed (like in the figure appended).
But when we use the new script (08.2020 Updated): the run stopped in the first time-step (using the same grd, and glorys data), also it not generates the ice variables.
Trying to solve the skewed problem, we changed the lines (bellow) in interp2d, but it didn't work.
def setupESMFInterpolationWeights(confM2R):
if confM2R.useesmf:
print("regridSrc2Dst at RHO points")
confM2R.grdMODEL.fieldSrc_rho = ESMF.Field(confM2R.grdMODEL.esmfgrid, "fieldSrc", staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.fieldDst_rho = ESMF.Field(confM2R.grdROMS.esmfgrid, "fieldDst",staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.regridSrc2Dst_rho = ESMF.Regrid(confM2R.grdMODEL.fieldSrc_rho, confM2R.grdMODEL.fieldDst_rho,regrid_method=ESMF.RegridMethod.BILINEAR,unmapped_action=ESMF.UnmappedAction.IGNORE)
print("regridSrc2Dst at U points")
confM2R.grdMODEL.fieldSrc_u = ESMF.Field(confM2R.grdMODEL.esmfgrid_u, "fieldSrc", staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.fieldDst_u = ESMF.Field(confM2R.grdROMS.esmfgrid_u, "fieldDst_u",staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.regridSrc2Dst_u = ESMF.Regrid(confM2R.grdMODEL.fieldSrc_u, confM2R.grdMODEL.fieldDst**_rho**,regrid_method=ESMF.RegridMethod.BILINEAR,unmapped_action=ESMF.UnmappedAction.IGNORE)
print("regridSrc2Dst at V points")
confM2R.grdMODEL.fieldSrc_v = ESMF.Field(confM2R.grdMODEL.esmfgrid_v, "fieldSrc", staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.fieldDst_v = ESMF.Field(confM2R.grdROMS.esmfgrid_v, "fieldDst_v",staggerloc=ESMF.StaggerLoc.CENTER)
confM2R.grdMODEL.regridSrc2Dst_v = ESMF.Regrid(confM2R.grdMODEL.fieldSrc_v, confM2R.grdMODEL.fieldDst**_rho**,regrid_method=ESMF.RegridMethod.BILINEAR,unmapped_action=ESMF.UnmappedAction.IGNORE)
The error message is:
Segmentation Fault - Core Dumped
Dear Tron:
Thanks in advance to share, maintain and support the model2roms code.
I´m try to make the ROMS forcing files with “soda3.3.2_5dy_ocean_reg” dataset, downloaded from:
http://dsrs.atmos.umd.edu/DATA/soda3.3.2/REGRIDED/ocean/
After configure the configM2R.py file, with the next parameters:
and the proper's define_ocean_forcing_data_path, define_roms_grid_path and the roms out grid stretching properties,
when I run the, runM2R.py file, there next error appear:
File "/Work/Python/model2roms/model2roms.py", line 274, in get_3d_data
str(confM2R.input_varnames[varN]), float(data.min()), float(data.max())))
UnboundLocalError: local variable 'data' referenced before assignment
I check the “get_3d funtion” in the model2roms.py file, and realize that there is not any option to mach with the “confM2R.ocean_indata_type == "SODA3_5DAY"” case. So I added the next code in the model2roms.py :
` if confM2R.ocean_indata_type == "SODA3_5DAY":
data = cdf.variables[confM2R.input_varnames[varN]][0, :, :, :]
data = np.where(data.mask, confM2R.fillvaluein, data)_
`
All runs ok, with no errors, but I’m not sure about the solution, the result files are inconsistent. Can you confirm that the added code is right? Any other suggestion?
Hi,
(1) Thanks for your efforts in writing model2roms. I'm trying to use it but I had the following error about the absence of module called 'extrapolate'. I was wondering whether you forgot to include it in the package.
exec(open('runM2R.py').read())
Traceback (most recent call last):
File "", line 1, in
File "", line 4, in
File "/home/jkansong/model2roms/model2roms.py", line 5, in
import interp2D
File "/home/jkansong/model2roms/interp2D.py", line 4, in
import extrapolate as ex
ModuleNotFoundError: No module named 'extrapolate'
(2) In runM2R.sh (and also in the README) you have a script called 'main.py' but I can't find this in the package. By the way, I used git to clone model2roms to my Ubuntu system. I don't know if that's the reason.
(3) Inside configM2R.py you have the following code snippet to grab a subset of, say global, data. But it's not clear to me what the numbers are. Are they the latitudes and longitudes of the region to subset? If so, which ones a latitudes and which are longitudes. It's also more confusing because the indices in subset are not consistent for NS8KM versus A20.
if self.outgrid == "NS8KM":
subset[1] = 40
subset[1] = 70
subset[2] = -30
subset[3] = 40
if self.outgrid == "A20":
subset[0] = 30
subset[1] = 90
subset[2] = -179
subset[3] = 360
(4) It'll be helpful if you could provide a minimal example of how to run model2roms.
Thanks very much!
-Joseph K. Ansong
Hi,
Where do I define the output grid (from my model), and the input data, from GLORYS?
Hii there
I was having a problem with the compilation of interpolation.f90 and it crashed with the following error
Compiling barotropic.f90 to create ==> barotropic.so
Compiling interpolation.f90 to create ==> interpolation.so
error: Command "/usr/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -I/tmp/tmpbuss0hc2/src.linux-x86_64-3.7 -I/opt/apps/ANACONDA/lib/python3.7/site-packages/numpy/core/include -I/opt/apps/ANACONDA/include/python3.7m -c -c /tmp/tmpbuss0hc2/src.linux-x86_64-3.7/interpolation-f2pywrappers2.f90 -o /tmp/tmpbuss0hc2/tmp/tmpbuss0hc2/src.linux-x86_64-3.7/interpolation-f2pywrappers2.o -J/tmp/tmpbuss0hc2/ -I/tmp/tmpbuss0hc2/" failed with exit status 1
Compiling fill.f90 to create ==> extrapolate.so
Compilation finished and results written to file => compile.log
As per your suggestion, I used
export LDFLAGS="-undefined dynamic_lookup -bundle"
f2py -c -m interpolation interpolation.f90
and it worked perfect.
Thanks for updating the LDFLAG fix for python3.7. I was not on github previously so I did not update this issue previously.
Thanks again.
I am using HYCOM data as input for my ROMS domain and I have a different landmask than the HYCOM grid, thus needing to fill quite a number of spots
I am using the laplace filter and fill.f90 on every regular level of my HYCOM data after interpolating it to rho grid like in the function do_hor_interpolation_regular_grid, however I found the result to be worse than my naive nearest neighbor filling and when I checked the ini files it fills in weird values that's 10 degrees colder than the surranding data points... thus causing the ROMS to blow up very early ...
the deepgreen color points are around 16 degrees while the majority of data here is 28 degrees C
Am I using the code wrong or is this the intended behaviour of the filling algorithm?
DISCLAIMER: I am using my own version of code to download and interpolate this to the 2d grid and only using the fill.f90 and laplace filter as a test to see if I can do better in filling...
Hi Trond,
I used WOAMONTHLY for the climatology data but I had an error with the naming of the variables
since WOA uses 'X' for 'longitude' and 'Y' for 'latitude' (I think). I thought I had fixed it by
putting the following code snippet in configM2R.py
# code from Trond to fix SODA3_5DAY
if self.ocean_indata_type == 'SODA3_5DAY':
self.lonname = "xt_ocean"
self.latname = "yt_ocean"
self.depthname = "st_ocean"
self.lonname_u = "xu_ocean"
self.latname_u = "yu_ocean"
self.lonname_v = "xu_ocean"
self.latname_v = "yu_ocean"
self.timeobject = []
# code added by Joseph for WOA data
if self.ocean_indata_type == 'WOAMONTHLY':
self.lonname = "X"
self.latname = "Y"
self.depthname = "Z"
but I got another error about variable names shown later below.
===THE FOLLOWING WAS THE ERROR I HAD BEFORE ADDING THE CODE ABOVE:
Started Fri Nov 13 12:25:09 2020
=> Conversions run for climatological months
==> The following variables will be interpolated: ['temperature', 'salinity']
=>All horisontal interpolations will be done using ESMF-ESMPy (module ESMF)
=>Output files are written in format: NETCDF4
=>Output grid file is: /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
Starting logfile for ESMF
Creating init for grid object GH1
---> Initialized GRD object for grid type ROMS
GRD file : opennetcdf opened file /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
--->Setting up vertical coordinates using self.vtransform: 2 self.vstretching: 2
Creating init for grid object GH1
---> Initialized GRD object for grid type FORCINGDATA
============================================================================
==BELOW IS THE ERROR I GOT AFTER TRYING TO FIX THE VARIABLE NAME ISSUE===
==AND I THINK IT STILL HAS TO DO WITH VARIABLE NAMES ISSUE. STILL TRYING TO FIX
IT BUT WANTED TO BRING IT TO YOUR ATTENTION. THANKS.=====
Started Fri Nov 13 12:32:29 2020
=> Conversions run for climatological months
==> The following variables will be interpolated: ['temperature', 'salinity']
=>All horisontal interpolations will be done using ESMF-ESMPy (module ESMF)
=>Output files are written in format: NETCDF4
=>Output grid file is: /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
Starting logfile for ESMF
Creating init for grid object GH1
---> Initialized GRD object for grid type ROMS
GRD file : opennetcdf opened file /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
--->Setting up vertical coordinates using self.vtransform: 2 self.vstretching: 2
Creating init for grid object GH1
---> Initialized GRD object for grid type FORCINGDATA
GRD file : opennetcdf opened file /home/jkansong/roms/DATASETS/WOAMONTHLY/temperature_monthly_1deg2001.nc
---> Assuming regular grid type for FORCINGDATA
---> Using dimension names X and Y and Z
NetCDF Error: variable longitude in /home/jkansong/roms/DATASETS/WOAMONTHLY/temperature_monthly_1deg2001.nc : NetCDF: Variable not found
Traceback (most recent call last):
File "/home/jkansong/model2roms/runM2R.py", line 46, in
run()
File "/home/jkansong/model2roms/runM2R.py", line 29, in run
model2roms.convertMODEL2ROMS(confM2R)
File "/home/jkansong/model2roms/model2roms.py", line 412, in convertMODEL2ROMS
confM2R.grdMODEL.createobject(confM2R)
File "/home/jkansong/model2roms/grd.py", line 85, in createobject
self.esmfgrid_u = ESMF.Grid(filename=self.grdfilename, filetype=ESMF.FileFormat.GRIDSPEC,
File "/home/jkansong/anaconda3/envs/model2roms/lib/python3.9/site-packages/ESMF/util/decorators.py", line 64, in new_func
return func(*args, **kwargs)
File "/home/jkansong/anaconda3/envs/model2roms/lib/python3.9/site-packages/ESMF/api/grid.py", line 345, in init
self._struct = ESMP_GridCreateFromFile(filename, filetype,
File "/home/jkansong/anaconda3/envs/model2roms/lib/python3.9/site-packages/ESMF/util/decorators.py", line 76, in new_func
return func(*args, **kwargs)
File "/home/jkansong/anaconda3/envs/model2roms/lib/python3.9/site-packages/ESMF/interface/cbindings.py", line 720, in ESMP_GridCreateFromFile
raise NameError('ESMC_GridCreateFromFile() failed with rc = '+str(rc))
NameError: ESMC_GridCreateFromFile() failed with rc = -1
Dear Tron:
Thank you very much for designing such a nice python toolbox, but it seems to be designed for Linux, can you add support for windows?
=> Conversions run from year/month: 2015/1 to 2015/3
==> The following variables will be interpolated: ['temperature', 'salinity', 'ssh', 'uvel', 'vvel', 'uice', 'vice', 'aice', 'hice']
=>All horisontal interpolations will be done using ESMF-ESMPy (module ESMF)
=>Output files are written in format: NETCDF4
=>Output grid file is: /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
Starting logfile for ESMF
Creating init for grid object GH1
---> Initialized GRD object for grid type ROMS
GRD file : opennetcdf opened file /home/jkansong/roms/Projects/Ghana4th/input/roms_grd.nc
--->Setting up vertical coordinates using self.vtransform: 2 self.vstretching: 2
Creating init for grid object GH1
---> Initialized GRD object for grid type FORCINGDATA
==> Starting loop over time
days []
days []
days []
Traceback (most recent call last):
File "/home/jkansong/model2roms/runM2R.py", line 46, in
run()
File "/home/jkansong/model2roms/runM2R.py", line 31, in run
clim2bry.writebry(confM2R)
File "/home/jkansong/model2roms/clim2bry.py", line 55, in writebry
clim = Dataset(confM2R.clim_name, 'r')
File "netCDF4/_netCDF4.pyx", line 2358, in netCDF4._netCDF4.Dataset.init
File "netCDF4/_netCDF4.pyx", line 1926, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'gh1_clim_GLORYS_20150115_to_20150331.nc'
Thanks.
When I try to create forcing files using toolbox model2roms, the created forcing files don't seem to be correct.
I've cloned the model2roms directory to my MacBook and changed the code to match my settings.
Compiling and running the code works however the output does not seem to be correct (see figure init_forcing_zeta) compared to the input reanalysis (see figure glory_1993_15_01).
From the terminal log I think that the problem might be caused by the 'fillvalue'. However, I've tried many different values that I thought might be correct and this does not solve the problem.
What I see in the log is:
Horizontal interpolation:N/A%|#
So maybe the problem lies with the horizontal interpolation
Edit: this is because the package progressbar was not installed
Does anyone know what might cause the weird output here?
Below the images I pasted parts of the configM2R.py file that I've updated.
I know only running for 2 months is very short but I'm now just running test cases to see if I've solved the problem.
return {'SODA': ['temperature', 'salinity', 'ssh', 'uvel', 'vvel'],
'SODA3': ['temperature', 'salinity', 'ssh', 'uvel', 'vvel'],
'SODA3_5DAY': ['temperature', 'salinity', 'ssh', 'uvel', 'vvel'],
'GLORYS': ['temperature', 'salinity', 'ssh', 'uvel', 'vvel'],
'WOAMONTHLY': ['temperature', 'salinity'],
'NORESM': ['temperature', 'salinity', 'ssh', 'uvel', 'vvel', 'ageice', 'uice', 'vice', 'aice', 'hice',
'hs',
'O3_c', 'O3_TA', 'N1_p', 'N3_n', 'N5_s', 'O2_o']}[
self.ocean_indata_type]
# Define the corresponding name of the variables in the input dataset files. This list needs to correspond
# exactly with the list given in the function define_global_varnames:
def define_input_data_varnames(self):
return {'SODA3': ['temp', 'salt', 'ssh', 'u', 'v'],
'SODA3_5DAY': ['temp', 'salt', 'ssh', 'u', 'v'],
'GLORYS': ['thetao', 'so', 'zos', 'uo', 'vo'],
'NORESM': ['templvl', 'salnlvl', 'sealv', 'uvellvl', 'vvellvl', 'iage', 'uvel', 'vvel', 'aice', 'hi',
'hs', 'dissic', 'talk', 'po4', 'no3', 'si', 'o2']}[self.ocean_indata_type]
# Define the path to where the ROMS grid can be found
def define_roms_grid_path(self):
try:
return {'A20': '../oceanography/A20/Grid/A20niva_grd_v1.nc',
'ROHO160': '../oceanography/NAUTILOS/Grid/norfjords_160m_grid.nc_A04.nc',
'ROHO800': '../oceanography/ROHO800/Grid/ROHO800_grid_fix3.nc',
'NS_test': 'NorthSea4_smooth01_sponge_nudg.nc'}[self.outgrid_name]
except KeyError:
return KeyError
# Define the abbreviation for the run, which is used to name output files etc.
def define_abbreviation(self):
return {"A20": "a20",
"Antarctic": "Antarctic",
"ROHO160": "roho160",
"ROHO800": "roho800",
"NS_test": "NS_test",}[self.outgrid_name]
def define_ocean_forcing_data_path(self):
try:
return {'SODA3': "../oceanography/copernicus-marine-data/SODA3.4.2/",
'SODA3_5DAY': "/Volumes/DATASETS/SODA2002/", # "/cluster/projects/nn9297k/SODA3.3.2/",
'NORESM': "/cluster/projects/nn9412k/A20/FORCING/RCP85_ocean/",
'GLORYS': "/Users/iriskeizer/Documents/ROMS/code/model2roms3/forcing/"}[self.ocean_indata_type]
except KeyError:
return KeyError
# IN GRIDTYPES ------------------------------------------------------------------------------
# Define what grid type you want to interpolate from (input MODEL data)
# Currently supported options:
# 1. NORESM, 2. GLORYS, 3. SODA3, 4. SODA3_5DAY
self.ocean_indata_type = 'GLORYS'
self.atmos_indata_type = 'ERA5'
# Define what grid type you wnat to interpolate from: Can be Z for SIGMA for ROMS
# vertical coordinate system or ZLEVEL. also define the name of the dimensions in the input files.
# Options:
# 1. SIGMA (not properly implemented yet), 2. ZLEVEL
self.ingrid_type = "ZLEVEL" # "ZLEVEL"
# Define the names of the geographical variables in the input files. These may
# differ depending how the variable is located in a grid (e.g. Arakawa C grid - ROMS). In
# SODA 3.3.1 the u and v location is defined by xu_ocean, yu_ocean while temperature is
# located in xt_ocean, yt_ocean.
self.grd_type = 'regular'
self.lon_name = "longitude"
self.lat_name = "latitude"
self.depth_name = "depth"
self.lon_name_u = "longitude"
self.lat_name_u = "latitude"
self.lon_name_v = "longitude"
self.lat_name_v = "latitude"
if self.ocean_indata_type in ['SODA3_5DAY','SODA3']:
self.lon_name = "xt_ocean"
self.lat_name = "yt_ocean"
self.depth_name = "st_ocean"
self.lon_name_u = "xu_ocean"
self.lat_name_u = "yu_ocean"
self.lon_name_v = "xu_ocean"
self.lat_name_v = "yu_ocean"
self.time_object = []
self.time_name = "time"
self.realm = "ocean"
self.fillvaluein = -32767
# OUT GRIDTYPES ------------------------------------------------------------------------------
# Define what grid type you want to interpolate to
# Options: This is just the name of your grid used to identify your selection later
self.outgrid_name = 'NS_test' # "ROHO800", "A20", "ROHO160"
self.outgrid_type = "ROMS"
# Define number of output depth levels
self.nlevels = 30
# Define the grid stretching properties (leave default if uncertain what to pick)
self.vstretching = 4
self.vtransform = 2
self.theta_s = 7.0
self.theta_b = 2.0
self.tcline = 50.0
self.hc = 50.0
# DATE AND TIME DETAILS ---------------------------------------------------------
# Define the period to create forcing for
self.start_year = 1993
self.end_year = 1993
self.start_month = 1
self.end_month = 2
self.start_day = 16
self.end_day = 15
Hi @trondkr
I'm using model2roms to create initial and boundary conditions files for a ROMS simulation, using GLORYS output as a source dataset. The package is running and generating files with consistent physical patterns. However, these files have wrong and very weird datetime, with years around 3550 (slice of code below). This issue turns out to be a big problem when setting a new simulation with atmospheric forcing because dates don't match.
=========================================================================
Created INIT file
set inittime in grd.py for time-index to print (current=0)
The time stamp for ROMS .in file saved to initial file is=> 3554-11-13 00:00:00
DSTART = 586896.0
TIME_REF = seconds since 1948-01-01 00:00:00
=========================================================================
The problem also repeats in the boundary conditions files:
=>Creating initial Boundary (BRY) file ./SWA5_bry_GLORYS_20181201_to_20181210.nc
clim2bry.py => Appending data to file ./SWA5_bry_GLORYS_20181201_to_20181210.nc for time 3554-11-13 00:00:00
Finished Thu Jun 11 01:33:43 2020
Any ideas on what is causing the weird years?
Hi Trond,
Just letting you know there was a bug when I changed the target grid to mine (rather than roho800). The problem was that the class s_coordinate(object) had Tcline defined as a attribute, while it is called (and defined elsewhere) as tcline.
I've changed Tcline to tcline everywhere to be safe, but I think the only real problem was with the attribute definition. You can either update it yourself or I'll include it in my next pull request.
Hello Trond
I am having an issue with the fill_value of GLORYS (-32767). More exactly in interp2D.py
where the 2D interpolation is done
field = confM2R.grdROMS.regridSrc2Dst_rho(confM2R.grdMODEL.fieldSrc_rho, confM2R.grdROMS.fieldDst_rho)
I noted that the input array has the correct fill_value, but the interpolation is made considering the fill_value cells which affect the interpolation output.
I set the fill value in configM2R.py
self.fillvaluein = -32767
I have also made a try with SODA3 reanalysis and in that case M2R worked correctly.
I noted that SODA3 has the Fillvalue and the missing_value attributes, but the GLORYS only has a Fillvalue attribute. For that reason, I added the missing_value attribute to the GLORYS file, but that did not make any effect.
Do you have any hint to solve the fill_value issue in interp2D.py with GLORYS12V1?
Best Regards,
José
In this project , I have many missing values and how can I filling it??Will they affect ROMS mode operation?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.