Code Monkey home page Code Monkey logo

Comments (4)

lesserwhirls avatar lesserwhirls commented on May 18, 2024

Tagging @DennisHeimbigner

from netcdf-java.

lesserwhirls avatar lesserwhirls commented on May 18, 2024

Ok, I think I've found the root issue here, so I'll cut right to the chase. The NCEI server does not deal with redirects of encoded urls properly.

Let's follow the steps of a request to http://www.ncei.noaa.gov/thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time[0:1:2]

We can immediately see an issue if we make a request, but encode the brackets:

curl -G -v "http://www.ncei.noaa.gov/thredds/dodsC/cdr/gridsat/GridSat-Aggre
gation.ncml.ascii?time%5b0:1:2%5d"
*   Trying 205.167.25.171...
* TCP_NODELAY set
* Connected to www.ncei.noaa.gov (205.167.25.171) port 80 (#0)
> GET /thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time%5b0:1:2%5d HTTP/1.1
> Host: www.ncei.noaa.gov
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Fri, 24 Jan 2020 20:21:09 GMT
< Server: Apache
< Location: https://www.ncei.noaa.gov/thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time%255b0:1:2%255d
< Content-Length: 310
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.ncei.noaa.gov/thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time%255b0:1:2%255d">here</a>.</p>
</body></html>
* Closing connection 0

Notice in the response header, the Location field has been double encoded. Specifically, the initial request uses time%5b0:1:2%5d as the query, whereas the redirection location shows time%255b0:1:2%255d.

Now if we try to use that location, we get:

curl -G -v "https://www.ncei.noaa.gov/thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time%255b0:1:2%255d"
*   Trying 205.167.25.178...
  <snip>
> GET /thredds/dodsC/cdr/gridsat/GridSat-Aggregation.ncml.ascii?time%255b0:1:2%255d HTTP/1.1
> Host: www.ncei.noaa.gov
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Date: Fri, 24 Jan 2020 20:31:51 GMT
< Server: Apache-Coyote/1.1
< Strict-Transport-Security: max-age=31536000
< XDODS-Server: opendap/3.7
< Content-Description: dods-error
< Content-Type: text/plain
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Headers: X-Requested-With, Content-Type
< Connection: close
< Transfer-Encoding: chunked
<
Error {
    code = 403;
    message = "Request too big=1.1117421067232E7 Mbytes, max=50.0";
};
* Closing connection 0

@oxelson - this smells like a mod_* gotcha (where mod_* is not mod_jk). Have you ran into this before?

We also see in the 403 response header that the Server field is set as Apache-Coyote/1.1, which also makes me think there are proxy configuration issues at play on their end.

from netcdf-java.

oxelson avatar oxelson commented on May 18, 2024

@lesserwhirls no, I've not seen this before with mod_jk, but that doesn't mean it's not a mod_* proxy issue. That said, I'd really need to know what their environment looks like (apache/tomcat configs) to see if the server environment is causing the encoding issues.

from netcdf-java.

lesserwhirls avatar lesserwhirls commented on May 18, 2024

I'll reach out to NCEI to see what's going on with the double encoding on the http→https redirect.

from netcdf-java.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.