dakrone / clj-http Goto Github PK
View Code? Open in Web Editor NEWAn idiomatic clojure http client wrapping the apache client. Officially supported version.
Home Page: http://clojars.org/clj-http
License: MIT License
An idiomatic clojure http client wrapping the apache client. Officially supported version.
Home Page: http://clojars.org/clj-http
License: MIT License
In my project.clj file I typically include :warn-on-reflection true
, which revealed the following warnings for clj-http:
Reflection warning, clj_http/cookies.clj:65 - call to parse can't be resolved. Reflection warning, clj_http/cookies.clj:94 - call to formatCookies can't be resolved. Reflection warning, clj_http/cookies.clj:96 - reference to field getValue can't be resolved. Reflection warning, clj_http/core.clj:67 - call to org.apache.http.conn.scheme.Scheme ctor can't be resolved. Reflection warning, clj_http/core.clj:120 - call to java.lang.Integer ctor can't be resolved. Reflection warning, clj_http/core.clj:122 - call to java.lang.Integer ctor can't be resolved. Reflection warning, clj_http/core.clj:140 - call to org.apache.http.impl.client.DefaultHttpClient ctor can't be resolved.
Would be nice if you could eliminate them, and also include the warning option in your project.clj file.
I'm using clj-http in request testing a service. Through the underlying HTTPClient4, it'd be great if the base get/post/etc methods could provide a hook for getting info about the request (URL + headers) for reporting during the testing - maybe a switch to return both request + response data? Probably there's a better clojure idiom that fits here. The alternative is to setup an embedded proxy (which I haven't had much luck with, at least using browsermob-proxy).
I need a way to set one part of a multipart/form-data POST to have a specific content-type. But it looks as if the name of the part is used as its type instead. This line: https://github.com/dakrone/clj-http/blob/master/src/clj_http/core.clj#L75 and the others like it pass keytext
to the constructors of the ContentBody
objects in place of the mime-type.
Whenever I send a request message to a different agent in the body of a with-connection-pool macro, I get a "java.lang.IllegalStateException: Connection pool shut down"
Example:
(def tst-agnt (agent nil))
(http/with-connection-pool {:threads 1} (send tst-agnt (fn _)))
using [clj-http "0.5.0"] , [org.clojure/clojure "1.4.0"]
Thanks
Some redirections look like "/bat" relative to the current url.
I just discovered, much to my surprise, that my code that has a try+/catch block with a (catch Exception e...) at the end does not catch "403" exceptions thrown by wrap-exceptions. It does catch everything else, e.g. other exceptions, just not those thrown by wrap-exceptions.
This is at least unintuitive and I'm not quite sure what is the right way to go, but: I filed an issue with slingshot (scgilardi/slingshot#24) and now I'm thinking whether it is a good idea for a library such as clj-http to throw exceptions using throw+.
I think (I'm not 100% certain, though) that while using try+/throw+ within the library is fine, exceptions that do not get caught should follow the standard Java conventions. I did not expect to see throw+ in clj-http and I had to look at the source code to see it.
Because of the ordering in wrap-request, a URL like http://user:password@host:port/path
will not have basic auth info extracted out of the URL. In order for this to work, the ->
form in wrap-request must have wrap-url
after wrap-user-info
, and wrap-user-info
after wrap-basic-info
.
Is this unintentional, or are we supposed to just put :basic-auth keys directly in the request map?
I posted the details here...
Hi,
client/get sometimes yields a 400 error from some sites. When I try to reproduce this using openConnection on a java.net.URL or the Apache HttpClient directly, this doesn't happen. The only difference I see in the headers sent is the addition of the Content-Length header which is set to 0 in the clj-http request. Not all servers respond with a 400 though. One that consistently does is http://thepiratebay.se.
What is forcing the addition of the said header? Any way to prevent this?
Hi,
I have been experimenting some difficulties in a scenario where a redirect comes with set-cookie header that is then ignored.
Is this a bug, or not? or am I missing something?
Hey,
Automatic encoding detection for webpages fails on a number of asian pages like this one.
(:require [clj-http.client :as http ])
(http/get "http://www.yomiuri.co.jp/" {:as :auto})
The reason is that for some reason clojure downloads the charset line in the source in the following way:
content="text/html; charset=Shift_JIS"
(This is from the output of of the command I issued above)
The backslash after "Shift_JIS" shouldn't be there and is preventing the correct pattern matching. (Lines 203-232 of client.clj). A simple fix would be to match the backslash in something like:
(?)? So that it is ignored in the re-find. I could writer a simple patch if you agree to that approach. If not could you suggest for a more idiomatic way to solve the problem?
Hi,
clj-http
appears to be creating and destroying a connection for each request. This is fine in many circumstances but when dealing with an API persistent connections are preffered for time and resource savings. Are persistent connections a non-objective for clj-http
or is it something that you would like to see happen?
A simple approach would be to provide a http-client
constructor in core
and allow the resulting client to be passed into the request
fn. (If no client is passed in the current behvaviour would be preserved and a new connection would be created and closed.) Special care would have to be given for when a connection times out so it can be reestablished. The client could be wrapped in an atom
to provide thread safety. I haven't thought too much about it but the appraoch seems reasonable... WDYT?
Thanks!
It'd be helpful if the get, head, post, put, delete functions in client.clj were marked ^:dynamic; in Clojure 1.3, variables need to be explicitly marked with ^:dynamic to be rebindable.
Using the clojure.core/binding macro to replace those functions allows using different HTTP libraries during testing. In this case, I have test code written to use clj-http, and I am also using The Grinder, a load testing tool that has its own instrumented HTTP library. Via rebinding, the same test code can be run from the REPL using clj-http when manual testing is needed and from The Grinder using its HTTP library when load testing is needed.
Currently, I need to write a ^:dynamic wrapper function around my clj-http calls to make this work. It'd be better if those functions could be rebound directly.
I've got a local SOCKS proxy running on port 9999. When I pass :proxy-host "127.0.0.1", :proxy-port 9999
in the options map, then an "org.apache.http.NoHttpResponseException: The target server failed to respond" is thrown.
I ran into a bit of trouble while writing my Github API library. The Github downloads API requires you to upload files to AWS, and aws requires multipart form data. There doesn't seem to be a way to do this in clj-http.
clj-http seems to have an issues with generating headers for POST requests:
Tested with clj-http-0.3.3 and this client code:
(client/post "http://localhost:8888/test" {:form-params {:foo "bar"}})
The library makes an assumption that a non-200/300 code is something that should be handled using exceptions. Better to return the reponse map and let the developer's application decide whether or not an exception is needed. The current behavior forces a developer to use try/catch blocks defensively wherease the proposed behavior would allow the application to make simpler calls with clj-http.
The JDK doesn't like certs with leading wildcards (like *.hostname.com), and it turns out the only way to handle them is to write your own verifier.
An example of a verifier that might work here: http://stackoverflow.com/questions/3135679/android-httpclient-hostname-in-certificate-didnt-match-example-com-ex/3136980#3136980
When I make this request via cURL, I see that my server is returning the response header Content-Length
:
HTTP/1.1 200 OK
Date: Fri, 05 Oct 2012 03:07:10 GMT
Vary: accept-charset, accept
Content-Type: text/html;charset=UTF-8
Content-Length: 939
Server: Jetty(7.6.1.v20120215)
When I make the same request with clj-http 0.5.5, using Clojure 1.4 and Java 7, Content-Length
is missing:
my-ns.core=> (:headers (client/get "http://localhost:3000/terms" {:accept "*/*"}))
{"date" "Fri, 05 Oct 2012 03:10:45 GMT", "vary" "accept-encoding, accept-charset, accept", "content-type" "text/html;charset=UTF-8", "connection" "close", "server" "Jetty(7.6.1.v20120215)"}
When using the cookie-store, I get an extra 'cookie' (notice lowercased) header set in my second request.
GET http://aspectoweb.local/ShowQuestion.aspx?rid=d4cf71ec-5224-42da-b2b6-9a92c5aacfd2 HTTP/1.1
Connection: close
accept-encoding: gzip, deflate
cookie:
Host: aspectoweb.local
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
Cookie: ASP.NET_SessionId=zbf1npc5f4q1c21a3cang551
This apparently causes problems (at least with ASP.NET), so the cookie is not recognized.
If I replay the request without the extra 'cookie:' header, everything works fine.
Method that I'm using to fetch (set up with Fiddler as proxy.. experimented with redirects as well)
(defn- fetch-url [url cookies]
(let [request (client/get url {:cookie-store cookies :follow-redirects true :response-interceptor (fn [resp ctx] (println ctx)) :proxy-host "127.0.0.1" :proxy-port 8888})
content (parse-as-string request)]
request))
Against Clojure 1.3.0:
[exec] Reflection warning, clj_http/cookies.clj:29 - reference to field getName can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:36 - reference to field getPath can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:34 - reference to field getDomain can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:31 - reference to field getComment can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:37 - reference to field getPorts can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:37 - reference to field getPorts can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:35 - reference to field getExpiryDate can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:35 - reference to field getExpiryDate can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:40 - reference to field getVersion can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:32 - reference to field getCommentURL can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:33 - reference to field isPersistent can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:38 - reference to field isSecure can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:39 - reference to field getValue can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:65 - call to parse can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:94 - call to formatCookies can't be resolved.
[exec] Reflection warning, clj_http/cookies.clj:96 - reference to field getValue can't be resolved.
[exec] Reflection warning, clj_http/core.clj:48 - call to java.lang.Integer ctor can't be resolved.
[exec] Reflection warning, clj_http/core.clj:50 - call to java.lang.Integer ctor can't be resolved.
Currently, when passing {:debug true}
or {:save-request? true}
, the :body that is returned looks like:
:body #<StringEntity org.apache.http.entity.StringEntity@714c7f58>
There should be an easier way to debug the bodies of requests.
When I'm debugging a request that's getting a 400 Bad Request header, or a 401 Authorization Required from the server, it's really helpful to actually be able to read the response body, or view the headers being sent to the server, so I can debug what's wrong with the query. I know clj-http is a wrapper on top of an Apache library, but I can't figure out how to get the response body on a 400 bad request through the clj-http interface.
I'd appreciate some help; I'm new to Clojure, I've never used this library before and a similar POST request using curl is working fine.
(http/head "http://www.payback.pl/" {:throw-exceptions false})
[Thrown class java.lang.NullPointerException]
Backtrace:
0: java.io.ByteArrayInputStream.<init>(ByteArrayInputStream.java:89)
1: clj_http.util$gunzip.invoke(util.clj:39)
2: clojure.lang.AFn.applyToHelper(AFn.java:163)
3: clojure.lang.AFn.applyTo(AFn.java:151)
4: clojure.core$apply.invoke(core.clj:542)
5: clj_http.client$update.doInvoke(client.clj:11)
6: clojure.lang.RestFn.invoke(RestFn.java:446)
7: clj_http.client$wrap_decompression$fn__3119.invoke(client.clj:66)
8: clj_http.client$wrap_input_coercion$fn__3130.invoke(client.clj:87)
9: clj_http.client$wrap_output_coercion$fn__3125.invoke(client.clj:74)
10: clj_http.client$wrap_basic_auth$fn__3157.invoke(client.clj:149)
11: clj_http.client$wrap_accept$fn__3139.invoke(client.clj:109)
12: clj_http.client$wrap_accept_encoding$fn__3144.invoke(client.clj:121)
13: clj_http.client$wrap_content_type$fn__3135.invoke(client.clj:100)
14: clj_http.client$wrap_form_params$fn__3173.invoke(client.clj:175)
15: clj_http.client$wrap_method$fn__3168.invoke(client.clj:164)
16: clj_http.cookies$wrap_cookies$fn__3073.invoke(cookies.clj:114)
17: clj_http.client$head.doInvoke(client.clj:235)
18: clojure.lang.RestFn.invoke(RestFn.java:424)
Now, this is a misbehaving server, as a quick test with curl shows:
curl -X HEAD -v http://www.payback.pl/
* About to connect() to www.payback.pl port 80 (#0)
* Trying 194.64.230.139... connected
* Connected to www.payback.pl (194.64.230.139) port 80 (#0)
> HEAD / HTTP/1.1
> User-Agent: curl/7.20.1 (x86_64-apple-darwin10.3.0) libcurl/7.20.1 OpenSSL/0.9.8r zlib/1.2.5
> Host: www.payback.pl
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 29 Sep 2011 08:43:04 GMT
< Server: Apache-Coyote/1.1
< Set-Cookie: JSESSIONID=459FE9EBA627536F8CC0319005320D88; Path=/pb
< Set-Cookie: _tracking_cookie=20110929104304927_51U6I6; Expires=Fri, 28-Sep-2012 08:43:04 GMT; Path=/
< Content-Type: text/html;charset=utf-8
< Content-Language: de-DE
< Content-Length: 28704
< Vary: Accept-Encoding,User-Agent
< Cache-Control: no-cache, no-store, must-revalidate, max-age=0, proxy-revalidate, no-transform, private
< Pragma: no-cache
< Set-Cookie: www-cookie=22088202.64545.0000; path=/
<
* transfer closed with 28704 bytes remaining to read
* Closing connection #0
curl: (18) transfer closed with 28704 bytes remaining to read
but I would expect clj-http to do something more reasonable than throw an exception, especially since I've requested that exceptions not be thrown. Parse the partial reply? Return nil? Return a result map with :status set to nil? I don't really know what the best choice is.
There are a lot of apis that will return html instead of json when an error status is returned. clj-http tries to parse the body before checking status code which often then masks the status code error with a parse exception. For now, I simply removed :as :json and use the following as a work around:
(-> (client/get ...)
(update-in [:body] (comp walk/keywordize-keys json/parse-string))
add a key like :trace-redirects in the response containing the ordered list of traversed uris. This would allow clients to know what was the final uri fetched more easily than now (I think the only way is to set :save-request? and check the url inside the req). It could also be very useful for caching purpose (knowing all the uris which are equivalent to the terminal one).
If you're interested, I can write the patch.
When I run code using clj-http in the repl, I'm seeing a ton of logging messges like...
DEBUG pool-1-thread-18 20111109 095547,513 org.apache.http.impl.conn.SingleClientConnManager ] Releasing connection org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter@4ea6c971
DEBUG pool-1-thread-18 20111109 095547,513 org.apache.http.impl.conn.SingleClientConnManager ] Released connection open but not reusable.
DEBUG pool-1-thread-18 20111109 095547,513 org.apache.http.impl.conn.DefaultClientConnection ] Connection shut down
I haven't passed in any options that would turn these on - only using :throw-exceptions false
How can I turn these off?
This section doesn't tell you how to send a body map that will be converted to JSON automatically.
https://github.com/dakrone/clj-http#input-coercion
(client/post "http://localhost:5000/v1/categories" {:body {:name "Computers"} :content-type :json})
gives me error:
CompilerException java.lang.RuntimeException: Unable to resolve symbol: => in this context, compiling (NO_SOURCE_PATH:1)
ClassCastException clojure.lang.PersistentArrayMap cannot be cast to [B clj-http.core/request (core.clj:220)
I have also tried
(client/post "http://localhost:5000/v1/categories" {:body {:name "Computers"} :as :json})
with the same error.
How do I tell the library to convert it to JSON?
Currently for uploading a file, the multipart logic is using :name for both the part-name and the file-name, would it be possible to add the flexibility to define different value for part-name and file-name?
Redirecting after a POST is very common, but wrap-redirects
only follows redirects on GET and HEAD requests (and only follows 303
responses on HEADs, which seems very odd). Reading the relevant spec sections, it seems that any valid 3xx
response code should be followed, with the :method
always altered to a :get
.
This is technically contrary to a couple of RFCs, but the spec explicitly says:
However, most
existing user agent implementations treat 302 as if it were a 303
response, performing a GET on the Location field-value regardless
of the original request method.
In general, I'd rather track what browsers (and API clients) do in general — or at least have an option to :redirect-posts
.
Thoughts?
The following renders an error:
(clj-http.core/generate-query-string {
:timestampBegin "2012-11-26 10:00:00 UTC"
:timestampEnd "20121127 10:00:00 UTC"})
where the {} is the query-params. It gives the following result:
"timestampBegin=2012-11-26+10%3A00%3A00+UTC×tampEnd=2012%C2%AD11%C2%AD27+10%3A00%3A00+UTC"
Please note that the dashes in timestampBegin is not correctly URL-encoded, on the contrary to timestampEnd. This must be a bug.
Expected behaviour is that all params are correctly URLencoded.
When a server fails to respond completely, I'd still expect :throw-exceptions false to translate the exception to a response map (even if the map is mostly empty)
When executing a GET operation I got this from a Ring server:
org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
at clj_http.core$request.invoke(core.clj:180)
at clojure.lang.Var.invoke(Var.java:365)
at clj_http.client$wrap_query_params$fn__952.invoke(client.clj:137)
at clj_http.client$wrap_user_info$fn__961.invoke(client.clj:162)
at clj_http.client$wrap_url$fn__975.invoke(client.clj:184)
at clj_http.client$wrap_redirects$fn__913.invoke(client.clj:46)
at clj_http.client$wrap_decompression$fn__918.invoke(client.clj:63)
at clj_http.client$wrap_input_coercion$fn__929.invoke(client.clj:88)
at clj_http.client$wrap_output_coercion$fn__924.invoke(client.clj:75)
at clj_http.client$wrap_exceptions$fn__906.invoke(client.clj:32)
at clj_http.client$wrap_basic_auth$fn__956.invoke(client.clj:152)
at clj_http.client$wrap_accept$fn__938.invoke(client.clj:109)
at clj_http.client$wrap_accept_encoding$fn__943.invoke(client.clj:122)
at clj_http.client$wrap_content_type$fn__934.invoke(client.clj:100)
at clj_http.client$wrap_form_params$fn__971.invoke(client.clj:179)
at clj_http.client$wrap_method$fn__966.invoke(client.clj:167)
at clj_http.cookies$wrap_cookies$fn__752.invoke(cookies.clj:114)
at clj_http.client$get.doInvoke(client.clj:234)
at clojure.lang.RestFn.invoke(RestFn.java:424)
...more...
It should clearly say what was it expecting and what did it find instead. Without that, it's hard to debug what's wrong.
clj-http 0.3.x conflicts with clj-json.
I don't know which library is bad, but clj-http 0.2.7 doesn't conflict.
$ lein new foo
$ cd foo
$ vim project.clj
(defproject foo "1.0.0-SNAPSHOT"
:description "FIXME: write description"
:dependencies [[org.clojure/clojure "1.3.0"]
[clj-http "0.3.1"]
[clj-json "0.5.0"] ])
$ vim src/foo/core.clj
(ns foo.core
(:require [clj-http.client :as client]) )
(defn -main [])
$ lein deps
$ lein run -m foo.core
Exception in thread "main" java.lang.RuntimeException: java.lang.VerifyError: class org.codehaus.jackson.smile.SmileParser overrides final method getBinaryValue.(Lorg/codehaus/jackson/Base64Variant;)[B
at clojure.lang.Util.runtimeException(Util.java:165)
at clojure.lang.Compiler.eval(Compiler.java:6476)
at clojure.lang.Compiler.eval(Compiler.java:6455)
(..snip..)
I can't seem to find a tag associated with release 0.2.5....
In 0.2.1, decompression doesn't seem to be working right:
user=> (use 'clj-http.client)
WARNING: get already refers to: #'clojure.core/get in namespace: user, being replaced by: #'clj-http.client/get
nil
user=> (get "http://api.clojuredocs.org/examples/println")
EOFException Unexpected end of ZLIB input stream java.util.zip.InflaterInputStream.fill (InflaterInputStream.java:223)
This is in a fresh clojure 1.3 REPL.
I discovered this while updating your cd-client to 1.3 and 0.2.1 of clj-http. This only happens with 0.2.1. It works fine with the older version.
Many pages on the web have through ill-intent or plain negligence cyclic redirections which will stall the client if it follows them blindly. Allowing to specify a maximum number of redirections to follow makes the process optionally more robust. The patch is backward-compatible and comes with a test.
When I try to retrieve a page with encoding different from UTF-8, I get \� instead of non-latin letters. Probably it's because of this line of code: https://github.com/dakrone/clj-http/blob/master/src/clj_http/client.clj#L80
As a temporary workaround, I manually create a string object with right encoding:
(-> url (client/get {:as :byte-array}) :body (String. "windows-1251"))
Can the project.clj dependency for commons codec be changed to 1.3 instead of 1.5? This won't make a practical difference as the only method out of that library that clj-http uses also exists in 1.3.
However pulling in commons codec 1.5 into any project by virtue of including clj-http can lead to compile-time vs. run-time conflicts on Android as Android contains a built-in version of commons codec 1.3 (as per source here)
I'm getting an encoded signature back in a cookie. The actual value received (from tcpdump) is:
{"identity": "{\"realm\": \"usdev150\", \"value\": \"{\\\"customer\\\": \\\"root\\\", \\\"realm\\\": \\\"usdev150\\\", \\\"entity_type\\\": \\\"user\\\", \\\"expires\\\": 1341591987.901052, \\\"user\\\": \\\"/root/root\\\", \\\"groups\\\": [\\\"/root\\\"]}\", \"signature\": \"APKYYWYI2X3VD0twdWneg+fGNt8yuyyaDbACoXhIkjVJlwB4Vnm4UqLvbTvBN4IpFZUSFK9h3uGEND9ndyzuLoDtdN/K6T2NkhxOECU7YKZ9XyLlVB0wA8Pv57+TsAYpTF7Jhhdq4YYMzvdBNeskhpbig1trFlP42hYp6R1BpBSnLRr3+cmlnuUfpNqEFze6HJgsZgHMHKB0CSn/NSTReH4sAIRJLrtNW9QRz7hlbUY0uTZejkuLlH+EQIZJDkVAPiN1owwMaPXU1KR49BJYuxH23JXRELSGy2wCql2YK9Fvhi/DXqMMwUw/RBAwVYwog3+jUY9Gv/WF1emEGM64cBnwTQ6C\"}"}
while the value I get in the debug output is:
{\"identity\": \"{\\\"realm\\\": \\\"usdev150\\\", \\\"value\\\": \\\"{\\\\\\\"customer\\\\\\\": \\\\\\\"root\\\\\\\", \\\\\\\"realm\\\\\\\": \\\\\\\"usdev150\\\\\\\", \\\\\\\"entity_type\\\\\\\": \\\\\\\"user\\\\\\\", \\\\\\\"expires\\\\\\\": 1341591987.901052, \\\\\\\"user\\\\\\\": \\\\\\\"/root/root\\\\\\\", \\\\\\\"groups\\\\\\\": [\\\\\\\"/root\\\\\\\"]}\\\", \\\"signature\\\": \\\"APKYYWYI2X3VD0twdWneg fGNt8yuyyaDbACoXhIkjVJlwB4Vnm4UqLvbTvBN4IpFZUSFK9h3uGEND9ndyzuLoDtdN/K6T2NkhxOECU7YKZ9XyLlVB0wA8Pv57 TsAYpTF7Jhhdq4YYMzvdBNeskhpbig1trFlP42hYp6R1BpBSnLRr3 cmlnuUfpNqEFze6HJgsZgHMHKB0CSn/NSTReH4sAIRJLrtNW9QRz7hlbUY0uTZejkuLlH EQIZJDkVAPiN1owwMaPXU1KR49BJYuxH23JXRELSGy2wCql2YK9Fvhi/DXqMMwUw/RBAwVYwog3 jUY9Gv/WF1emEGM64cBnwTQ6C\\\"}\"}
Apart from the escaping from printing, you can see that each time there is a '+' character in the original value, I receive a space character.
=> (client/get "http://anthropologynow.tumblr.com/post/8047984462/stressing-motherhood-how-biology-and-social-inequality")
ClassCastException clojure.lang.LazySeq cannot be cast to java.lang.CharSequence clojure.core/re-matcher (core.clj:4286)
In order not to copy bodies, allow byte array bodies that are set as the Entity with no coercion.
I am using [clj-http "0.5.0"]
The output of
(require '[clj-http.client :as client])
(def headers {"User-Agent" "Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20100101 Firefox/13.0.1"
"Accept" "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,text/png,/;q=0.5"
"Accept-Language" "en-us,en;q=0.5"
"Connection" "keep-alive"
"Content-type" "application/x-www-form-urlencoded"
"Accept-Encoding" "gzip, deflate"})
(client/get "http://www.google.com" {:headers headers})
are nonsensical characters.
There's no way to set the flag on the Apache client that tells it not to follow redirects; instead it appears that they're always followed. I want to handle a 302 response myself, rather than getting the result of the redirected request.
I'm testing a server by sending requests to it through clj-http. When doing a multipart post, I get this output
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond
Apr 17, 2012 1:18:26 PM org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request
The fact that my server isn't responding is the issue I'm trying to fix, but the fact that the request is being sent three times muddles the process up. It would be nice if this could be disabled.
=> (client/get "http://google.com")
{:cookies {"PREF" {:domain ".google.com", ... }, :status 302, :headers {"x-xss-protection" "1; mode=block", "server" "gws", "x-frame-options" "SAMEORIGIN", "content-type" "text/html; charset=UTF-8", "date" "Fri, 28 Oct 2011 16:05:00 GMT", "cache-control" "private", "location" "http://www.google.co.uk/", "content-length" "221", "connection" "close"}, :body "<HTML><HEAD><meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF=\"http://www.google.co.uk/\">here</A>.\r\n</BODY></HTML>\r\n"}
=> (client/get "http://www.nytimes.com:80/2011/10/29/business/the-spotlight-now-shines-on-italy-common-sense.html")
{:cookies {"NYT-S" {:domain ".nytimes.com", :expires #<Date Sun Nov 27 16:18:10 GMT 2011>, :path "/", :value "0Mzh9PJwQ663rDXrmvxADeHCyZ5RAjhv6MdeFz9JchiAIUFL2BEX5FWcV.Ynx4rkFI", :version 0}}, :status 302, :headers {"date" "Fri, 28 Oct 2011 16:18:10 GMT", "server" "Apache/2.2.3 (CentOS)", "location" "http://www.nytimes.com:80/2011/10/29/business/the-spotlight-now-shines-on-italy-common-sense.html?_r=1", "content-length" "0", "connection" "close", "content-type" "text/html; charset=UTF-8"}, :body ""}
Does clj-http support AsyncClient?
Using the clojure download (4.3MB)
http://repo1.maven.org/maven2/org/clojure/clojure/1.3.0/clojure-1.3.0.zip
In a browser, the download takes about 5s
In a REPL with clj-http.client/get, the download takes nearly 10 minutes
Similar behavior observed for a 1MB download (2sec v 2.5min)
This limits the usefulness of clj-http for working with big gets in a program.
This is not a bug report, just a thought to start a conversation.
In several of ClojureWerkz libraries that use clj-http
we ended up doing URL encoding of paths. I am curious if this is something that clj-http
should provide.
Are there any reasons why clj-http
or Apache HTTP components don't do automatic URL encoding for paths or offer it as an option?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.