finagle / finagle-serial Goto Github PK

View Code? Open in Web Editor NEW

62.0 62.0 9.0 436 KB

Create Finagle servers and clients that use the serialization library of your choice!

Home Page: https://twitter.com/finagle

License: Apache License 2.0

Scala 99.40% Thrift 0.60%

finagle-serial's Issues

Migrate to simple build.sbt configuration instead of Build.scala

Add support for Scala Pickling library

Github page: https://github.com/scala/pickling

Add support for Scodec library

Github page: https://github.com/scodec/scodec

Add unit tests for the core project

The integration testing for scodec is pretty okay, but the rest needs some help (especially core).

Publish API documentation on GitHub Pages

I added sbt-unidoc, but haven't set up sbt-ghpages. These should probably go under /docs/ to leave room for a project site and for consistency with Finagle, Util, etc.

Service test doesn't work with types involving lists

I'm able to use testFunctionService for signatures involving non-container types but when I use List or Vector there appear to be serialization errors that don't occur when I test encode/decode manually.

package blah

import io.github.finagle.serial.scodec.ScodecSerial
import io.github.finagle.serial.tests.SerialIntegrationTest
import org.scalacheck._
import org.scalatest.FunSuite
import scodec.Attempt.Successful
import scodec.bits.BitVector
import scodec.codecs._
import scodec.{Codec, DecodeResult}

class ListSpec extends FunSuite with ScodecSerial with SerialIntegrationTest {

  implicit val listCodec = list(utf8_32)

  implicit val intCodec = int32

  implicit val lists: Arbitrary[List[String]] = Arbitrary {
    Gen.nonEmptyContainerOf[List, String](Gen.alphaStr)
  }

  test("List encode decode") {
    check((p: List[String]) => {
      listCodec.encode(p).flatMap(listCodec.decode) === Successful(DecodeResult(p, BitVector.empty))
      }
    )
  }

  test("List service test") {
    testFunctionService[List[String], Int]((p1: List[String]) => p1.length)
  }

}

The first test passes but the second fails with this (technically I had to modify a future with onFailure to dump this, but it is the root cause):

io.github.finagle.serial.CodecError: cannot acquire 32 bits from a vector that contains 7 bits
    at io.github.finagle.serial.scodec.ScodecSerial$fresh$macro$49$1.from(ScodecSerial.scala:27)
    at io.github.finagle.serial.scodec.ScodecSerial$fresh$macro$49$1.from(ScodecSerial.scala:27)
    at scodec.Transformer$$anon$7$$anonfun$apply$12.apply(Transform.scala:121)
    at scodec.DecodeResult.map(DecodeResult.scala:17)
    at scodec.Codec$$anon$2$$anonfun$decode$1.apply(Codec.scala:201)
    at scodec.Codec$$anon$2$$anonfun$decode$1.apply(Codec.scala:201)
    at scodec.Attempt$Successful.map(Attempt.scala:92)
    at scodec.Codec$$anon$2.decode(Codec.scala:201)
    at scodec.Codec$$anon$2.decode(Codec.scala:201)
    at scodec.codecs.ToCoproductCodecs$$anon$8$$anon$1.decode(CoproductCodec.scala:108)
    at scodec.codecs.ToCoproductCodecs$$anon$8$$anonfun$1$$anon$2.decode(CoproductCodec.scala:120)
    at scodec.Decoder$$anon$2.decode(Decoder.scala:46)
    at scodec.GenCodec$$anon$2.decode(GenCodec.scala:76)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Attempt$Successful.flatMap(Attempt.scala:94)
    at scodec.Decoder$$anon$3.decode(Decoder.scala:54)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Attempt$Successful.flatMap(Attempt.scala:94)
    at scodec.Decoder$$anon$3.decode(Decoder.scala:54)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Decoder$$anon$3$$anonfun$decode$2.apply(Decoder.scala:54)
    at scodec.Attempt$Successful.flatMap(Attempt.scala:94)
    at scodec.Decoder$$anon$3.decode(Decoder.scala:54)
    at scodec.codecs.CoproductCodec$Discriminated.decode(CoproductCodec.scala:60)
    at scodec.Decoder$$anon$4.decode(Decoder.scala:62)
    at scodec.GenCodec$$anon$2.decode(GenCodec.scala:76)
    at scodec.Codec$$anon$1.decode(Codec.scala:191)
    at scodec.Codec$$anon$2.decode(Codec.scala:201)
    at io.github.finagle.serial.scodec.ScodecSerial$class.decodeRep(ScodecSerial.scala:103)
    at com.chartboost.protocol.request.ListSpec.decodeRep(ListSpec.scala:12)
    at com.chartboost.protocol.request.ListSpec.decodeRep(ListSpec.scala:12)
    at io.github.finagle.serial.Serial$Client$$anon$4$$anonfun$apply$1$$anonfun$apply$2.apply(Serial.scala:161)
    at io.github.finagle.serial.Serial$Client$$anon$4$$anonfun$apply$1$$anonfun$apply$2.apply(Serial.scala:160)
    at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:986)
    at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:985)
    at com.twitter.util.Promise$Transformer.liftedTree1$1(Promise.scala:112)
    at com.twitter.util.Promise$Transformer.k(Promise.scala:112)
    at com.twitter.util.Promise$Transformer.apply(Promise.scala:122)
    at com.twitter.util.Promise$Transformer.apply(Promise.scala:103)
    at com.twitter.util.Promise$$anon$1.run(Promise.scala:381)
    at com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:178)
    at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:136)
    at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:207)
    at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:92)
    at com.twitter.util.Promise.runq(Promise.scala:350)
    at com.twitter.util.Promise.updateIfEmpty(Promise.scala:721)
    at com.twitter.util.Promise.update(Promise.scala:694)
    at com.twitter.util.Promise.setValue(Promise.scala:670)
    at com.twitter.concurrent.AsyncQueue.offer(AsyncQueue.scala:111)
    at com.twitter.finagle.netty3.transport.ChannelTransport.handleUpstream(ChannelTransport.scala:55)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at com.twitter.finagle.mux.transport.Netty3Framer$Framer.handleUpstream(Netty3Framer.scala:31)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142)
    at com.twitter.finagle.netty3.channel.ChannelStatsHandler.messageReceived(ChannelStatsHandler.scala:68)
    at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142)
    at com.twitter.finagle.netty3.channel.ChannelRequestStatsHandler.messageReceived(ChannelRequestStatsHandler.scala:32)
    at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Add code coverage via coveralls.io

Add JSON over HTTP benchmark

It's not as direct a comparison as thrift-mux, but would help to show where this falls in the spectrum of performance people put up with.

Fix Thrift benchmark for Scala 2.11

Currently the Thrift benchmark crashes SBT with the following error on 2.11 (2.10 works just fine):

Annotation generator had thrown the exception.
java.lang.IncompatibleClassChangeError: i.g.f.s.thriftscala.Small and i.g.f.s.thriftscala.Small$Proxy$class disagree on InnerClasses attribute
        at java.lang.Class.getDeclaringClass0(Native Method)
        at java.lang.Class.getDeclaringClass(Class.java:1106)
        at java.lang.Class.getEnclosingClass(Class.java:1142)
        at java.lang.Class.getCanonicalName(Class.java:1231)
        at org.openjdk.jmh.generators.reflection.RFClassInfo.getQualifiedName(RFClassInfo.java:67)
        at org.openjdk.jmh.generators.core.BenchmarkGenerator.buildAnnotatedSet(BenchmarkGenerator.java:213)
        at org.openjdk.jmh.generators.core.BenchmarkGenerator.generate(BenchmarkGenerator.java:108)
        at org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator.main(JmhBytecodeGenerator.java:100)
        at pl.project13.scala.sbt.SbtJmh$.generateBenchmarkJavaSources(SbtJmh.scala:81)
        at pl.project13.scala.sbt.SbtJmh$$anonfun$jmhSettings$6.apply(SbtJmh.scala:28)
        at pl.project13.scala.sbt.SbtJmh$$anonfun$jmhSettings$6.apply(SbtJmh.scala:28)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:63)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.Execute.work(Execute.scala:235)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Probably a Scrooge problem, but needs more investigation.

Fix handling of errors when encoding exceptions with scodec

Currently any errors that happen while Scodec is encoding request or response types are correctly wrapped in CodecError (not ServerApplicationError), but if encoding fails while encoding an exception, you'll get the CodecError string wrapped in a ServerApplicationError.

This isn't a terribly big deal, since exceptions should be easy to encode and shouldn't usually fail, but it wouldn't be too hard to fix.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

finagle / finagle-serial Goto Github PK

finagle-serial's Issues

Migrate to simple build.sbt configuration instead of Build.scala

Add support for Scala Pickling library

Add LICENSE

Add CONTRIBUTING.md

Add support for Scodec library

Add unit tests for the core project

Publish API documentation on GitHub Pages

Service test doesn't work with types involving lists

Add code coverage via coveralls.io

Add JSON over HTTP benchmark

Fix Thrift benchmark for Scala 2.11

Fix handling of errors when encoding exceptions with scodec

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent