mmcgrana / ring Goto Github PK

Implementation of temp-file-store in the ring.middleware.multipart-params.temp-file creates too many futures objects and this leads to too many threads running on web services that handles many multipart uploads...

P.S. I'll try to refactor code to use single expiration thread or something like...

At https://github.com/mmcgrana/ring/blob/master/ring-core/src/ring/middleware/session/cookie.clj#L92

should there be a

(binding [read-eval false] ) ?

Otherwise, it seems -- suppose some other vulnerability allowed the AES key to leak -- then we suddenly have the situation where the client can do arbitrary code execution by read macros?

I'm developping a webapp which has two components, one is a JSON REST service, the other the app itself. Both are written in Clojure with Compojure.

When running the app with run-jetty, is there a way to run the service with a specific context path? A simple solution would be to allow additional WARs to be specified and runned by the server. Is it possible?

I need to use http://jetty.codehaus.org/jetty/jetty-6/apidocs/org/mortbay/jetty/Request.html#getLocale() to get the locale from the Accept-Language string.

How do I access the Jetty request directly for this?

When client does not specify its charset in Content-Type header of a request, a server use default charset(ISO-8859-1). Therefore parameters of a request is encoded in other than default charset without telling what charset they use , the params can't be decoded properly via wrap-params middleware. And i see that a complete accurate auto-detecting charset is almost impossible, IMO, it might be good that a user have a chance to use their own function to detect charset when wrap-param middleware process decode params.

Wrap-file without wrap-content-type makes IE9 to ignore css files, because their content type is not set.

What about making wrap-file to use wrap-content-file or at least adding the warning to wrap-file docstring?

I am wrapping my response handler with wrap-reload but when I change one of my route files it does not reload the code as a result of it. Not sure if this is an issue with ns-tracker or wrap-reload.

@weavejester This one might be an issue with ns-tracker and since you made that and this I figured I'd just tag you, let me know if this is in poor taste. :)

OS: OSX Lion 10.7.4
Command to start process using foreman: lein trampoline run -m gameapi.core
Example Code:

(ns gameapi.core
    (:gen-class)
    (:use [ring.adapter.jetty]
          [ring.middleware.reload]
          [ring.middleware.stacktrace])
    (:require [com.tnrglobal.bishop.core :as bishop]
              [gameapi.routes :as routes]))

(def env 
    "Global Environment: DEV or PROD"
    (or (System/getenv "ENV") "DEV"))

(defn main
    "Exposes the main function for bootstrapping the application."
    [& args]
    (let [port (Integer/parseInt (or (System/getenv "PORT") "3000"))
          base-handler (bishop/handler routes/routes)
          handler (if (= env "DEV") (wrap-reload base-handler)
                      base-handler)]
        (println handler)
        (println (str "Webserver starting on port " port " with env " env))
        (run-jetty handler {:port port})))

(defn -main
    [& args]
    (main args))

In ring-core 1.2.1, I'm getting this explosion:

Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest, compiling:(ring/middleware/multipart_params.clj:39:5)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6567)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3624)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6562)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5708)
    at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5139)
    at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3751)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6558)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6548)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.access$100(Compiler.java:37)
    at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:529)
    at clojure.lang.Compiler.analyzeSeq(Compiler.java:6560)
    at clojure.lang.Compiler.analyze(Compiler.java:6361)
    at clojure.lang.Compiler.analyze(Compiler.java:6322)
    at clojure.lang.Compiler.eval(Compiler.java:6623)
    at clojure.lang.Compiler.load(Compiler.java:7064)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5018.invoke(core.clj:5530)
    at clojure.core$load.doInvoke(core.clj:5529)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5336)
    at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
    at clojure.core$load_lib.doInvoke(core.clj:5374)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$load_libs.doInvoke(core.clj:5417)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:621)
    at clojure.core$use.doInvoke(core.clj:5507)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at compojure.handler$eval11534$loading__4910__auto____11535.invoke(handler.clj:1)
    at compojure.handler$eval11534.invoke(handler.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6619)
    at clojure.lang.Compiler.eval(Compiler.java:6608)
    at clojure.lang.Compiler.load(Compiler.java:7064)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5018.invoke(core.clj:5530)
    at clojure.core$load.doInvoke(core.clj:5529)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5336)
    at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
    at clojure.core$load_lib.doInvoke(core.clj:5374)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$load_libs.doInvoke(core.clj:5413)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:619)
    at clojure.core$require.doInvoke(core.clj:5496)
    at clojure.lang.RestFn.invoke(RestFn.java:3894)

At the point in my code where I wrap the app in:

      ring.middleware.multipart-params/wrap-multipart-params 

What's baffling to me is that the line cited above in ring-core is the last line of this function:

(defn- file-item-seq
  "Create a seq of FileItem instances from a request context."
  [context]
  (file-item-iterator-seq
    (.getItemIterator (FileUpload.) context)))

What that has to do with HttpServletRequest is utterly opaque to me.

Sadly, I haven't been able to isolate it any further, because I have very little understanding of the underlying Java stuff, and no idea what a HttpServletRequest is, why it's in ring, or why it could be exploding.

FWIW, the server I'm using here is aleph, not jetty. If this is a problem with aleph, please let me know and I'll ask Zach about it instead

It seems that the keyword-params middleware doesn't accept a map with keys that are keywords instead of strings.

Multipart-params produces such a map on file upload. The input's name is passed as a string, but the hash with the file metadata contains keywords :size, :tempfile, etc.

The wrap-file middleware currently takes a string argument, but it should take an option map in order to be consistent with the other core middleware.

In order to maintain backward compatibility, and as a form of syntax sugar, we should still support supplying a string argument.

The wrap-resource middleware should also be changed.

While trying to compile my own project with warn-on-reflection in Clojure 1.3.0 I got the following warnings in ring code.

Reflection warning, ring/util/response.clj:50 - reference to field isDirectory can't be resolved.
Reflection warning, ring/util/response.clj:52 - reference to field exists can't be resolved.
Reflection warning, ring/util/servlet.clj:64 - call to setContentType can't be resolved.
Reflection warning, ring/util/servlet.clj:78 - call to setCharacterEncoding can't be resolved.
Reflection warning, ring/middleware/multipart_params/byte_array.clj:14 - call to toByteArray can't be resolved.
Reflection warning, ring/middleware/resource.clj:15 - call to substring can't be resolved.
Reflection warning, ring/middleware/session/cookie.clj:81 - reference to field length can't be resolved.
Reflection warning, ring/middleware/session/cookie.clj:81 - reference to field length can't be resolved.
Reflection warning, ring/middleware/session/cookie.clj:85 - reference to field getBytes can't be resolved.
Reflection warning, ring/middleware/session/cookie.clj:86 - reference to field getBytes can't be resolved.

It is possible that there are more.

Create a charset function that adds or changes the charset of a response.

In ring.middleware.wrap-cookies:

• The :max-age key should support JodaTime Intervals (via clj-time)
• The :expires key should support JodaTime DateTimes and Java Dates

Existing Ring 1.0 behaviour should not be affected (i.e. :max-age should handle an integer, etc.)

(See: https://github.com/seancorfield/clj-time)

Setting headers, particularly cache-control, can be very important when serving static files. This is especially true in an environment like Heroku, where you're relying on a server-side HTTP cache to keep app requests down.

I propose a simple and flexible solution: supporting a new key in opts called :headers. You can pass in a function or a map.

If you pass in a map, then that becomes the headers map.

If you pass in a function, then the function will receive the path along with opts. The developer-supplied function must return a map of headers. The purpose of allowing a function is to provide the greatest possible flexibility. (Developers may want to set headers differently depending on the path.)

Sorry that I don't have time to write a patch now. I just wanted to pass this idea along. Thanks!

In trying to use the CookieStore session store using non-default attributes, I encountered the following stack trace:

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
at clojure.lang.LazySeq.sval(LazySeq.java:47)
at clojure.lang.LazySeq.seq(LazySeq.java:56)
at clojure.lang.RT.seq(RT.java:450)
at clojure.core$seq.invoke(core.clj:122)
at ring.util.servlet$set_headers.invoke(servlet.clj:86)
at ring.util.servlet$update_servlet_response.invoke(servlet.clj:124)
at ring.adapter.jetty$proxy_handler$fn__731.invoke(jetty.clj:18)
at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

It appears as if one of the headers is set to a lazy-seq which is then trying to walk over a nil. This app works fine with the standard memory store, so I believe this to be an issue with the cookie store in particular. I reported this to the Noir mailing list and Chris thought it was a bug in Ring, so I am opening this issue.

The code which produces this is here:

https://github.com/codeslinger/nooz/blob/0b2f08bcb7efce23a41894c91dce234087bd581c/src/nooz/server.clj#L17

If I remove the :session-cookie-attrs line from the configuration map, everything works properly. Having it in, though, produces the above stacktrace and a 500 error on every request to the server from then on.

Is the correct way to destroy a session just sending back:

{:session {}}

or

{:session nil}

This is not documented in the Session wiki page.

Any plans to upgrade to jetty 7.x?

• Inline CSS should be in a resource
• Use new Hiccup functions like html5
• Update the set of ring keys.

In wrap-reload after the new code is loaded, the old handler is still used to serve
the current request meaning that to see any code changes you have to refresh the page
twice which is seriously annoying.

wrap-reload probably needs to be passed in a var containing the top level application
handler, so that if code is reloaded it can be dereferenced and used to server this
current request.

Add some new middleware that proxies HTTP HEAD requests as GETs and the removes the body from the response.

This will make it easier to support the HEAD method in Ring apps.

Let's clean up the README, assume most people are going to use Ring via a project.clj dependency, and remove the examples folder from the repository (examples can be better stored and documented in the wiki or a linked gist).

IE send an url not url-encoded when "Send UTF-8 query strings..." flag of Internet Option is unset. And parameters of this request can't be processed properly by wrap-params, which decode just only url-encoded url. In my app i use and make wrap-not-url-ecoded-params middleware. But IMO, it might be good for wrap-params middleware to process a not-url-encoded-params.

As of Ring 0.3.11, parameters with the same name are no longer being vectorized correctly. The following code will contain only the last value:

(defn nested-test [_]
  (html
     [:head [:title "test"]]
     [:body
        [:form {:action "/submit" :method "POST"}
        [:input {:type :text :name "mything[]" :value "10"}]
        [:input {:type :text :name "mything[]" :value "20"}]
        [:input {:type :submit}]]]))

(defn submit [request]
   (println (request :params)))

Expected:
{:mything ["10" "20"]}

Actual:
{:mything ["20"]}

For complete code and more context see this thread:
https://groups.google.com/d/msg/clj-noir/Hub0ica7X6c/-f22jkEEr0sJ

Public functions should have {:added "1.0"} metadata to define which version they were added in.

I can url-encode an XML string. When I url-decode the returned value, I'm getting an index out-of-bounds on String in percent-decode (v 1.1.0):

xception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 3
at java.lang.String.charAt(String.java:686)
at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
at clojure.string$replace_by.invoke(string.clj:58)
at clojure.string$replace.invoke(string.clj:82)
at ring.util.codec$percent_decode.doInvoke(codec.clj:31)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at ring.util.codec$url_decode.doInvoke(codec.clj:51)
at clojure.lang.RestFn.invoke(RestFn.java:410)

Just a doc error in https://github.com/mmcgrana/ring/wiki/Sessions

(def app
  (wrap-session handler {:cookie-attrs {:max-age 3600}))

Forgot a }

The ring-devel leiningen project includes a dependency on clj-stacktrace 0.2.2. Unfortunately, that version of clj-stacktrace is not compatible with recent versions of swank-closure (1.3.4 or newer). The "Troubleshooting" section of https://github.com/technomancy/swank-clojure describes the incompatibility; it breaks connecting to the Emacs REPL.

Could you please upgrade the version of clj-stacktrace specified in ring-devel/project.clj to 0.2.4?

If wrap-resource is added to the handler stack ahead of Compojure routes meant to serve the root (index) route while running from a jar file, it will mistakenly identify the top resource directory as a file and return a zero-length result, preventing any inner handlers (such as Compojure routes) from serving the root path. This behavior does not occur when running from a source tree; everything works fine when not in a jar, with identical code.

This seems to be the same problem described at https://groups.google.com/forum/#!topic/ring-clojure/P6BU4xc7p-0 .

Possible fixes:

• Don't serve any zero-length results from wrap-resource, as suggested in that thread.
• Detect directories in some other way, and don't serve them
• Don't serve the root route, at least, since requests for the root of a site are common and are never going to be correctly served by a zero-length reply.

I am happy to do the actual coding for the fix if the maintainers approve one of these strategies for incorporation into Ring.

Because it sets :body to nil, and :body is used to set Content-Length.

Create a new namespace, ring.util.io in ring-core. The namespace should have:

• Functions for turning an OutputStream into an InputStream
• Copy ring.util.test/string-input-stream into ring.util.io
• Deprecate ring.util.test/string-input-stream

Reminder.

SSL client certificate support should have some tests.

(ns ring-session-test.core)

(use 'ring.middleware.session)

(defn handler [{session :session uri :uri}]
  (let [res {:status 200
             :headers {"Content-Type" "text/plain"}
             :body (str session)
             :session {:a 1}}]
    (if (= (session :a) 1)
      (assoc res :session-cookie-attrs {:max-age 3600})
      res)))

(def app
  (wrap-session handler))

What I wanted is that the session cookie is sent on the first request
and then set the expiry date on another one. But on the second
request there is no cookie sent - I copy the headers below. If I add
:session-cookie-attrs on the first response - then it gets there in
the Cookie:

Set-Cookie:ring-session=00bd9357-a388-4714-ac1b-4e66def7e2f7;Max-Age=3600;Path=/

But on the second not.

Request URL:http://localhost:3000/che
Request Method:GET
Status Code:200 OK

Request Headersview source

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Cookie:ring-session=519317a4-6543-42b5-ab6d-49cf0bba8bba
Host:localhost:3000
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.17 (KHTML,
like Gecko) Chrome/24.0.1312.56 Safari/537.17

Response Headersview source

Content-Length:6
Content-Type:text/plain;charset=ISO-8859-1
Date:Wed, 30 Jan 2013 13:47:30 GMT
Server:Jetty(7.6.1.v20120215)

The decision to allow url-decode to return nil for improperly encoded strings (instead of throwing an exception) now means that cookie handling needs to accomodate this newly-endorsed return value.

Frankly, I think raising an exception was the right behavior, but since nil is now a valid return value, the normalize-quoted-strs function now fails with an NPE if any cookie value cannot be decoded.

In some cloud environments, such as Heroku, the number of threads available to a process is limited.

The Jetty adapter should therefore support a :max-threads option for limiting the number of threads the web server uses.

See here: https://groups.google.com/forum/#!topic/compojure/miJlyihsLNM

If the ring.util.response/resource-response function is given a path to a directory, it will except instead of returning nil.