Code Monkey home page Code Monkey logo

net's Introduction

net: A clojure Netty companion

⚠️ use of net is now discouraged in favor of aleph ⚠️

Build Status

Net provides a clojure foundation to implement asynchronous networking based on Netty.

It is much narrower in scope and features than aleph, which you might want to look into if you want a full-fledged asynchronous programming toolkit for clojure.

net is rather geared towards people with prior Netty knowledge wanting to keep the same workflow in, hopefully, idiomatic Clojure, and nothing but standard clojure facilities.

  • Light facades around Netty concepts such as channels, pipelines, channel initializers and bootstraps
  • Facilities to create TLS client and server contexts from PEM files
  • Ring-like HTTP(S) server facade
  • HTTP(S) client
  • Simple interface to create TCP server with optional TLS support
  • Clojure core.async support
  • Memory-efficient transducers for collections of ByteBufs

Documentation

Net now has full API Documentation and Guides.

Installation

    [[spootnik/net "0.3.3-beta39"]]

Changelog

0.3.3-beta36

  • Support SNI for TLS requests

0.3.3-beta34

  • Add draining facilities for unexpectedly closed channels

0.3.3-beta32

  • Reliability fixes in HTTP client
  • Allow disabling of SSL certificate validation
  • Allow more options to be supplied to the HTTP server

0.3.3-beta28

  • Ensure lost clients do not result in filled core.async channel buffers

0.3.3-beta27

  • Start introducing bridged channels for tighter integration between Netty and core.async.
  • Ensure event-loop-groups can be parameterized
  • Allow supplying query args as a map in HTTP requests

0.3.3-beta26

  • Play nice with non-epoll systems

0.3.3-beta23

  • Adapt net.http.client to match server behavior: responses contain a core.async Channel body
  • Handle transforms over the body, i.e: help callers supply a transducer when requesting
  • Better handle error conditions
  • Add net.core.async/timeout-pipe to help ensure upstream feeds a channel fast enough

0.3.3-beta19

  • Handle Netty out-of-band /bad-request requests from HttpObjectDecoder

0.3.3-beta18

  • Improve behavior on early downstream channel closing

0.3.3-beta17

  • Fix a memory leak when closing the body channel from a handler
  • Add clojure tools.deps.alpha support

0.3.3-beta16

  • Fix all reflection warnings
  • Fix calls to deprecated signatures in Netty

0.3.3-beta15

  • No more dependency on tools.logging
  • Better dispatch for pipelines

0.3.3-beta14

  • Depend on Netty 4.1.19.Final
  • Add ByteBuf collection transducers
  • Handle Http server callback on an executor
  • Improve Netty facade
  • Add ByteBuf manipulation tooling
  • Circumvent CLJ-1814 as much as possible

0.3.3-beta9

  • Depend on Netty 4.1.8.Final

0.3.3-beta8

  • specs for http server options

0.3.3-beta7

  • Small fixes

0.3.3-beta6

  • Allow user-supplied executor for responses (thanks @mpenet).
  • Fix HTTP-related regressions introduced by reflection work.

0.3.3-beta4

  • Ensure all calls do not need reflection.
  • Correctly terminate clients in tcp server shutdown fn.

0.3.3-beta3

  • Break HandlerAdapter into several protocols

0.3.3-beta2

  • Add documentation and guides
  • Improved specs

0.3.3-beta1

  • Rework HTTP support to be aligned with jet
  • Provide a single HTTP server interface, which allows aggregating or streaming body content.

0.2.20

  • Allow user-supplied max body size

0.2.19

  • Bugfix release for 0.2.18
  • More restrictive specs

0.2.18

  • Convenience macros to create encoders and decoders.

0.2.17

  • core.spec schemas instead of prismatic schema
  • Rely on Netty 4.1.6
  • Additional sugar for futures and channels

Thanks

License

Copyright © 2015, 2016, 2017 Pierre-Yves Ritschard, MIT License.

net's People

Contributors

arnaudgeiser avatar bcachet avatar brutasse avatar crhough avatar geverding avatar greut avatar ifesdjeen avatar mcorbin avatar mpenet avatar pyr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

net's Issues

Net performance

Hi. Tuned some perf tests (Aleph, Immutant) for https://www.techempower.com/benchmarks/ and was hinted by @mpenet that Net would be interesting too. The perf in my test seems really bad, so I guess I'm doing something wrong.

➜  net git:(net) wrk -t2 -c100 -d2s http://localhost:8080/json
Running 2s test @ http://localhost:8080/json
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.14ms    1.54ms  47.88ms   99.16%
    Req/Sec    46.07k     4.50k   53.70k    88.10%
  192395 requests in 2.10s, 28.07MB read
Requests/sec:  91521.48
Transfer/sec:     13.35MB
➜  net git:(net) wrk -t2 -c100 -d2s http://localhost:8080/json
Running 2s test @ http://localhost:8080/json
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.32ms    4.16ms  15.90ms   59.93%
    Req/Sec     3.43k     1.69k    6.31k    72.73%
  7551 requests in 2.03s, 575.17KB read
  Socket errors: connect 0, read 7726, write 4, timeout 0
Requests/sec:   3712.82
Transfer/sec:    282.81KB

what am I doing wrong?

Type hints and reflection

Is there a measured need to avoid reflection? Several things break when type hints are applied (channel/channel, write! and write-and-flush! no longer working on groups, etc.)

on Windows7

#error {
:cause Only supported on Linux
:via
[{:type clojure.lang.Compiler$CompilerException
:message java.lang.UnsatisfiedLinkError: failed to load the required native library, compiling:(core.clj:32:10)
:at [clojure.lang.Compiler$InvokeExpr eval Compiler.java 3700]}
{:type java.lang.UnsatisfiedLinkError
:message failed to load the required native library
:at [io.netty.channel.epoll.Epoll ensureAvailability Epoll.java 88]}
{:type java.lang.ExceptionInInitializerError
:message nil
:at [io.netty.channel.epoll.Epoll Epoll.java 39]}
{:type java.lang.IllegalStateException
:message Only supported on Linux
:at [io.netty.channel.epoll.Native loadNativeLibrary Native.java 177]}]
:trace
[[io.netty.channel.epoll.Native loadNativeLibrary Native.java 177]
[io.netty.channel.epoll.Native Native.java 61]
[io.netty.channel.epoll.Epoll Epoll.java 39]
[io.netty.channel.epoll.EpollEventLoop EpollEventLoop.java 56]
[io.netty.channel.epoll.EpollEventLoopGroup newChild EpollEventLoopGroup.java 134]
[io.netty.channel.epoll.EpollEventLoopGroup newChild EpollEventLoopGroup.java 35]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 84]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 58]
[io.netty.util.concurrent.MultithreadEventExecutorGroup MultithreadEventExecutorGroup.java 47]
[io.netty.channel.MultithreadEventLoopGroup MultithreadEventLoopGroup.java 59]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 104]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 91]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 68]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 52]
[io.netty.channel.epoll.EpollEventLoopGroup EpollEventLoopGroup.java 45]
[net.ty.bootstrap$epoll_event_loop_group invokeStatic bootstrap.clj 79]

using the http.client library

I'm trying to figure out how to use the http.client library. Is there a way to get a string output with a request?

(require 'net.http.client :as client)

(def bs (async/<!! (:body (client/request {:request-method :get :uri "http://www.bing.com"}))))
  
(slurp (binary/input-stream (.nioBuffer bs)))
=> ""

optimistic finally clause

Because of the (try (finally )) in response-handler client.clj

Apr 27, 2018 8:25:00 AM io.netty.channel.AbstractChannelHandlerContext invokeExceptionCaught
WARNING: An exception 'java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocol
s/ReadPort found for class: nil' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handli
ng the following exception:
java.lang.IllegalArgumentException: No implementation of method: :as-buffer of protocol: #'net.ty.buffer/Bufferizable found for class: io.nett
y.handler.codec.http.DefaultHttpResponse

http-server should not rely on an executor

Recent code relies on core.async's executor for a number of things and the need for a specific executor to be provided with the config could now be forfeited.

The two remaining operations done on the provided executor are now:

  • get-response whose sole blocking operation is to call the provided ring-handler
  • The synchronized call to close-future

server w/Epoll Exceptions

Using Epoll on the server initially throws for a protocol problem. Once that's fixed (PR #16) I get

io.netty.channel.AbstractChannel$AbstractUnsafe closeForcibly
WARNING: Failed to close a channel.
java.lang.IllegalStateException: channel not registered to an event loop
	at io.netty.channel.AbstractChannel.eventLoop(AbstractChannel.java:158)
	at io.netty.channel.epoll.AbstractEpollChannel.doDeregister(AbstractEpollChannel.java:134)
	at io.netty.channel.epoll.AbstractEpollChannel.doClose(AbstractEpollChannel.java:111)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.closeForcibly(AbstractChannel.java:703)
	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:337)
	at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:283)
	at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:279)
	at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:261)
	at net.ty.bootstrap$bind_BANG_.invokeStatic(bootstrap.clj:125)
	at net.ty.bootstrap$bind_BANG_.invoke(bootstrap.clj:122)
	at net.tcp$server.invokeStatic(tcp.clj:21)
	at net.tcp$server.invoke(tcp.clj:16)

Here's my server call (which worked with some patches, merged in beta4, before the protocols)

(tcp/server {:handler (pipeline)
                    :group (EpollEventLoopGroup.)
                    :channel EpollServerSocketChannel}
                    "localhost" 50000)

Everything works fine without Epoll.

Close but not terminate?

In net.tcp/server the returned function doesn't terminate client connections. Am I misunderstanding how this should work or what it should do (I've never used Netty before finding this project)?

To duplicate:

  • Add some code to the chat server example to hang on to the returned function (I dumped it into an atom)
  • Run the chat server example
  • Connect to the server twice (I used netcat in two terminals)
  • Call the saved saved function but don't close the terminals or quit either netcat instance.
  • Continue sending messages between clients.
    • This continues until the clients quit (close! does work in a handler), the read timeout is reached, or the JVM terminates.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.