trondkr / era5-roms Goto Github PK

Hello @trondkr sorry for the bother.

I don't understand what is happening, I am setting

        self.start_year = 2017
        self.end_year = 2020
        self.project = "IEA"

with

        return {'IEA': '80/-40/45/15',
                'A20': '90/-180/40/180'}[project]

and I am getting the following:

2023-02-10 12:52:12,079 INFO Request is failed
2023-02-10 12:52:12,079 ERROR Message: the request you have submitted is not valid
2023-02-10 12:52:12,080 ERROR Reason:
request['verbose'][0]: False is not of type 'string'
2023-02-10 12:52:12,080 ERROR Traceback (most recent call last):
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
2023-02-10 12:52:12,080 ERROR result = cached(context.method, proc, context, context.args, context.kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
2023-02-10 12:52:12,080 ERROR result = proc(context, *context.args, **context.kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in call
2023-02-10 12:52:12,080 ERROR return p(*args, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in call
2023-02-10 12:52:12,080 ERROR return self.proc(context, *args, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/mars.py", line 48, in internal
2023-02-10 12:52:12,080 ERROR return mars(context, request, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/mars.py", line 18, in mars
2023-02-10 12:52:12,080 ERROR **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/preprocess_request.py", line 24, in preprocess_request
2023-02-10 12:52:12,080 ERROR requests = enforce_minimal_schema(requests, context)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/request_checking.py", line 54, in enforce_minimal_schema
2023-02-10 12:52:12,080 ERROR return enforce_schema(requests, schema, context)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cds-common/cds_common/request_schemas/enforce_schema.py", line 29, in enforce_schema
2023-02-10 12:52:12,080 ERROR from None
2023-02-10 12:52:12,080 ERROR cdsinf.exceptions.BadRequestException:
2023-02-10 12:52:12,080 ERROR request['verbose'][0]: False is not of type 'string'

The funny thing is that this very same code was working very well last December.
Any help and suggestion you could provide?

Thank you very much in advance.

When testing the current output of this package for an Arctic ROMS model it seems that ROMS has either 1) a problem with one of the variables that prevent seasonality to emerge in the ROMS solution 2) the internal interpolation in ROMS is broken.. This needs to be further investigated

Thanks for this great project, but I encountered a small issue while using the code.
In the ECMWF_convert_to_ROMS.py, line 99, function write_to_ROMS_netcdf_file is defined and netcdf_file is used as a parameter, which gives the name of the netcdf file to be written in line 120.
However, in line 120,the code is netcdf_roms_filename = f"{out_filename[0:-3]}_roms.nc", which use out_filename instead of netcdf_file, causing the problem: NameError: name 'out_filename' is not defined.
I wonder if the code may need to be modified. Looking for reply. Thanks

I try to download data for ROMS Rutgers. I can use it but I can download 10 variables. My other problem is the code does not change the time properly and my header file is for 1900. When I start running the model, I faced a problem.
netcdf era5_msnlwrf_year_2014 {
dimensions:
longitude = 36 ;
latitude = 29 ;
time = 4380 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int time(time) ;
time:units = "hours since 1948-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short msnlwrf(time, latitude, longitude) ;
msnlwrf:scale_factor = 0.00339088604676628 ;
msnlwrf:add_offset = -91.074113655914 ;
msnlwrf:_FillValue = -32767s ;
msnlwrf:missing_value = -32767s ;
msnlwrf:units = "W m**-2" ;
msnlwrf:long_name = "Mean surface net long-wave radiation flux" ;

// global attributes:
:Conventions = "CF-1.6" ;
:history = "2022-06-04 09:23:44 GMT by grib_to_netcdf-2.24.3: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data1/adaptor.mars.internal-1654334612.669915-11911-5-14831270-9472-4c48-9faf-6bc9abe5ac36.nc /cache/tmp/14831270-9472-4c48-9faf-6bc9abe5ac36-adaptor.mars.internal-1654334292.9382155-11911-7-tmp.grib" ;
data:

longitude = 12, 12.25, 12.5, 12.75, 13, 13.25, 13.5, 13.75, 14, 14.25, 14.5,
14.75, 15, 15.25, 15.5, 15.75, 16, 16.25, 16.5, 16.75, 17, 17.25, 17.5,
17.75, 18, 18.25, 18.5, 18.75, 19, 19.25, 19.5, 19.75, 20, 20.25, 20.5,
20.75 ;

latitude = 45.9, 45.65, 45.4, 45.15, 44.9, 44.65, 44.4, 44.15, 43.9, 43.65,
43.4, 43.15, 42.9, 42.65, 42.4, 42.15, 41.9, 41.65, 41.4, 41.15, 40.9,
40.65, 40.4, 40.15, 39.9, 39.65, 39.4, 39.15, 38.9 ;

time = 999312, 999314, 999316, 999318, 999320, 999322, 999324, 999326,
999328, 999330, 999332, 999334, 999336, 999338, 999340, 999342, 999344,
999346, 999348, 999350, 999352, 999354, 999356, 999358, 999360, 999362,
999364, 999366, 999368, 999370, 999372, 999374, 999376, 999378, 999380,
999382, 999384, 999386, 999388, 999390, 999392, 999394, 999396, 999398,
999400, 999402, 999404, 999406, 999408, 999410, 999412, 999414, 999416,
999418, 999420, 999422, 999424, 999426, 999428, 999430, 999432, 999434,
999436, 999438, 999440, 999442, 999444, 999446, 999448, 999450, 999452,

Hello sir,
I am trying to use your script ERA5-ROMS. Thanks a lot for your such a generous contribution. Although I had an issue. After running the toolbox using according to Run the toolbox section of Readme.md file, I am not sure where the file is being saved(i mean its location and filename). May be it's a bug or maybe I am unable to use it in a proper way. Also is there a way that I can use this script to create forcing on my own grid present in netcdf format ?

Thanks a lot for your help in advance @trondkr

Hi Trond,
Could you please check whether there's a bug in ECMWF_tools.py. I specified both the start and end dates to be 2015 in ECMWF_querry.py, but when the download completed it started to grab data for 2016. Thanks again for this tool!

Regards,
Joseph K. Ansong

Great code！But I had a problem when I ran it: the download was too slow (20~100kb/s). I simply modified the code, now I can call Internet Download Manager (IDM) for multi-threaded download, the speed can reach 8mb/s. But the whole process of data download and processing is not elegant enough (need to run twice and comment out some statements). I wonder if the author has any plans to add a multi-threaded download module to the code. Thanks.

Hi Trond,

Thanks for this great piece of code, but I came into some issue that ROMS model (I'm using COAWST actually) can't read the forcing data. It seems ROMS doesn't like the latitude to be from 90 to -90, I modified "ECMWF_convert_to_ROMS.py" with following:

vnc = f1.createVariable('lat', 'd', 'lat', fill_value=fill_val)
vnc.long_name = 'Latitude'
vnc.units = 'degree_north'
vnc.standard_name = 'latitude'
vnc[:] = latitude[::-1]



vnc = f1.createVariable(metadata['roms_name'], 'd', (metadata['time_name'], 'lat', 'lon'), fill_value=fill_val)
vnc.long_name = metadata["name"]
vnc.standard_name = metadata["short_name"]
vnc.coordinates = "lon lat {}".format(metadata['time_name'])
vnc.units = var_units
vnc.missing_value = fill_val
vnc[:, :, :] = data_array[:,::-1,:]

and ROMS is running. Not sure if it's an error with ROMS or COAWST, but I think it may be worth bringing it to you.

Cheers,
Yi

Hi Trondkr. I'm using your script to download the ERA5 data to use in my ROMS domain. However, the download is apparently doing very slowly, and after some time the process is killed unexpectedly, at different times (in my failed attempts to download and process the data). It can be something wrong with the solicitation? do you know if we have a limit of downloads of ERA5 data?

I am trying to download:

• 'reanalysis-era5-single-levels'
• 3hourly data from 1998 to 2018. (10m u,v wind, 2m temperature, mean sea level pressure, mean surface downward long and short wave radiation fluxes, total cloud cover, total precipitation, specific humidity)
• domain: -30/-140/-85/40

The download starts, it is downloaded 1998 u_wind, v_wind, they are converted to ROMS, but it stops when it finishes the 2m-temperature, or before...