opendap / hyrax Goto Github PK

Our user reported incomplete data download cases from this server

https://goldsmr5.gesdisc.eosdis.nasa.gov/opendap/

All cases appear to use the fileout netcdf handler for the .nc4 responses with subsetting, and mostly for the M2I3* datasets under 'MERRA2', which are very large granules themselves. A typical URL is:

https://goldsmr5.gesdisc.eosdis.nasa.gov/opendap/MERRA2/M2I3NPASM.5.12.4/2017/03/MERRA2_400.inst3_3d_asm_Np.20170324.nc4.nc4?T[0:7][0:41][0:360][0:575],H[0:7][0:41][0:360][0:575],O3[0:7][0:41][0:360][0:575],QV[0:7][0:41][0:360][0:575],PHIS[0:7][0:360][0:575],PS[0:7][0:360][0:575],lat[0:360],time[0:7],lon[0:575],lev

While these are large data requests that can overwhelm the server sometimes (presumably when the download incompleteness occur?), a data download would either be a success or failure (It appears that the httpd has status 200 in the access logs).

Caching in the server was turned on. I wonder if cache management in server can interfere with a download, given the pattern of the failures which were occasional and quasi-periodical.

Hyrax versions
I'm using the hyrax-ncwms docker container:
https://hub.docker.com/r/opendap/hyrax_ncwms

This issue affect all versions after 1.16.4.

It is present in all the versions I have tested.

• 1.16.5: I have tested: 1.16.5-0, 1.16.5-287, 1.16.5-289
• 1.16.6: I have tested: 1.16.6-2
• 1.16.7: I have tested: 1.16.7-1, 1.16.7-5, 1.16.7-6

It doesn't affect version 1.16.4 and earlier.

Issue
The viewers link from the OPeNDAP Hyrax web page is broken, for every hosted files.

Example
For the dataset data/hdf5/grid_1_2d.h5, the viewers URL points to:
http://localhost:8080/viewers?dapService=/opendap/hyrax&datasetID=///data/hdf5/
instead of
http://localhost:8080/opendap/viewers/viewers?dapService=/opendap/hyrax&datasetID=/data/hdf5/grid_1_2d.h5

The STARE library is at 0.16.2 in your dependencies, but we need 0.16.3.

Please reference this issue for all CovJSON Response Handler for OPeNDAP project issues.

-Corey

At three places in hyrax_build.sh, there is a reference to "tomcat-7" while hyrax_clone.sh actually fetches tomcat-8. This causes tomcat to not be installed if you follow the suggested procedure (source ./spath.sh; ./hyrax_clone.sh; ./hyrax_build.sh).

ant-junit needs to be in the list of required packages for hyrax_build.sh to succeed. It has been added to the README in PR #1.

Hi, is it normal that the live data in Hyrax's data directory does not update until Tomcat is restarted?

HDFEOS2.cc: In member function ‘void HDFEOS2::File::handle_grid_cf_attrs()’:
HDFEOS2.cc:1769:43: error: ‘HUGE’ was not declared in this scope
 1769 |                     float tempfillvalue = HUGE;
      |                                           ^~~~

From a phone conversation (Jim and I) we believe this is related to an obsoleted value.h pre-processor symbol.

We believe it should be changed to MAXFLOAT.