pyr / net Goto Github PK

Hi. Tuned some perf tests (Aleph, Immutant) for https://www.techempower.com/benchmarks/ and was hinted by @mpenet that Net would be interesting too. The perf in my test seems really bad, so I guess I'm doing something wrong.

• immutant: https://github.com/ikitommi/FrameworkBenchmarks/tree/net/frameworks/Clojure/immutant

➜  net git:(net) wrk -t2 -c100 -d2s http://localhost:8080/json
Running 2s test @ http://localhost:8080/json
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.14ms    1.54ms  47.88ms   99.16%
    Req/Sec    46.07k     4.50k   53.70k    88.10%
  192395 requests in 2.10s, 28.07MB read
Requests/sec:  91521.48
Transfer/sec:     13.35MB

• net: https://github.com/ikitommi/FrameworkBenchmarks/tree/net/frameworks/Clojure/net

➜  net git:(net) wrk -t2 -c100 -d2s http://localhost:8080/json
Running 2s test @ http://localhost:8080/json
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.32ms    4.16ms  15.90ms   59.93%
    Req/Sec     3.43k     1.69k    6.31k    72.73%
  7551 requests in 2.03s, 575.17KB read
  Socket errors: connect 0, read 7726, write 4, timeout 0
Requests/sec:   3712.82
Transfer/sec:    282.81KB

what am I doing wrong?

Because of the (try (finally )) in response-handler client.clj

Apr 27, 2018 8:25:00 AM io.netty.channel.AbstractChannelHandlerContext invokeExceptionCaught
WARNING: An exception 'java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocol
s/ReadPort found for class: nil' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handli
ng the following exception:
java.lang.IllegalArgumentException: No implementation of method: :as-buffer of protocol: #'net.ty.buffer/Bufferizable found for class: io.nett
y.handler.codec.http.DefaultHttpResponse

Recent code relies on core.async's executor for a number of things and the need for a specific executor to be provided with the config could now be forfeited.

The two remaining operations done on the provided executor are now:

• get-response whose sole blocking operation is to call the provided ring-handler
• The synchronized call to close-future

#error {
:cause Only supported on Linux
:via
[{:type clojure.lang.Compiler$CompilerException
:message java.lang.UnsatisfiedLinkError: failed to load the required native library, compiling:(core.clj:32:10)
:at [clojure.lang.Compiler$InvokeExpr eval Compiler.java 3700]}
{:type java.lang.UnsatisfiedLinkError
:message failed to load the required native library
:at [io.netty.channel.epoll.Epoll ensureAvailability Epoll.java 88]}
{:type java.lang.ExceptionInInitializerError
:message nil
:at [io.netty.channel.epoll.Epoll Epoll.java 39]}
{:type java.lang.IllegalStateException
:message Only supported on Linux
:at [io.netty.channel.epoll.Native loadNativeLibrary Native.java 177]}]
:trace
[[io.netty.channel.epoll.Native loadNativeLibrary Native.java 177]
[io.netty.channel.epoll.Native Native.java 61]
[io.netty.channel.epoll.Epoll Epoll.java 39]
[io.netty.channel.epoll.EpollEventLoop EpollEventLoop.java 56]
[io.netty.channel.epoll.EpollEventLoopGroup newChild EpollEventLoopGroup.java 134]
[io.netty.channel.epoll.EpollEventLoopGroup newChild EpollEventLoopGroup.java 35]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 84]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 58]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 47]
[io.netty.channel.MultithreadEventLoopGroup MultithreadEventLoopGroup.java 59]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 104]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 91]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 68]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 52]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 45]
[net.ty.bootstrap$epoll_event_loop_group invokeStatic bootstrap.clj 79]

In net.tcp/server the returned function doesn't terminate client connections. Am I misunderstanding how this should work or what it should do (I've never used Netty before finding this project)?

To duplicate:

• Add some code to the chat server example to hang on to the returned function (I dumped it into an atom)
• Run the chat server example
• Connect to the server twice (I used netcat in two terminals)
• Call the saved saved function but don't close the terminals or quit either netcat instance.
• Continue sending messages between clients.
  • This continues until the clients quit (close! does work in a handler), the read timeout is reached, or the JVM terminates.

Is there a measured need to avoid reflection? Several things break when type hints are applied (channel/channel, write! and write-and-flush! no longer working on groups, etc.)

I'm trying to figure out how to use the http.client library. Is there a way to get a string output with a request?

(require 'net.http.client :as client)

(def bs (async/<!! (:body (client/request {:request-method :get :uri "http://www.bing.com"}))))
  
(slurp (binary/input-stream (.nioBuffer bs)))
=> ""

How to reproduce it

• run a server
• curl -X "A : B" http://localhost:32415

Result

it hangs.

Expected result

sends a 400

Using Epoll on the server initially throws for a protocol problem. Once that's fixed (PR #16) I get

io.netty.channel.AbstractChannel$AbstractUnsafe closeForcibly
WARNING: Failed to close a channel.
java.lang.IllegalStateException: channel not registered to an event loop
	at io.netty.channel.AbstractChannel.eventLoop(AbstractChannel.java:158)
	at io.netty.channel.epoll.AbstractEpollChannel.doDeregister(AbstractEpollChannel.java:134)
	at io.netty.channel.epoll.AbstractEpollChannel.doClose(AbstractEpollChannel.java:111)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.closeForcibly(AbstractChannel.java:703)
	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:337)
	at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:283)
	at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:279)
	at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:261)
	at net.ty.bootstrap$bind_BANG_.invokeStatic(bootstrap.clj:125)
	at net.ty.bootstrap$bind_BANG_.invoke(bootstrap.clj:122)
	at net.tcp$server.invokeStatic(tcp.clj:21)
	at net.tcp$server.invoke(tcp.clj:16)

Here's my server call (which worked with some patches, merged in beta4, before the protocols)

(tcp/server {:handler (pipeline)
                    :group (EpollEventLoopGroup.)
                    :channel EpollServerSocketChannel}
                    "localhost" 50000)

Everything works fine without Epoll.