Code Monkey home page Code Monkey logo

proto's Introduction

proto

Scaladex

Lightweight and fast serialization library for Scala 2/3 based on Protocol Buffers with macros magic.

Motivation

Serialization library for Scala that can be used either for long term store models and short term models. With easy to migrate possibility.

  • Lightweight
  • Fast
  • Protocol Buffers compatible
  • No .proto files
  • No model convertation
  • Type safe
  • Serialization/deserialization without changes in models
  • Possibility to use specific types in model

Install

Add dependency:

libraryDependencies += "io.github.zero-deps" %% "proto" % "latest.integration"

Dependency as a git-submodule is also supported.

Benchmark #1

data library scala-2 scala-3
data decode java 44079.139 38979.697
data decode jackson 176941.468 188555.562
data decode jsoniter-scala 483788.001 no support for Scala 3
data decode boopickle 2885610.648 no support for Scala 3
data decode proto 3383845.458 3776688.591
data decode scalapb 3270691.564 3893847.420
data library scala-2 scala-3
data encode java 220444.268 217484.396
data encode jackson 431318.803 384863.249
data encode jsoniter-scala 1054650.233 no support for Scala 3
data encode boopickle 1520834.519 no support for Scala 3
data encode proto 3186951.441 2965427.382
data encode scalapb 3628779.864 3972905.402
data library scala-2 scala-3
msg decode jsoniter-scala 3486552.303 no support for Scala 3
msg decode proto 5825174.170 6395557.251
msg decode scalapb 4898257.671 6902064.854
data library scala-2 scala-3
msg encode jsoniter-scala 6372602.760 no support for Scala 3
msg encode proto 6487748.959 6745673.393
msg encode scalapb 9202135.451 9056962.541

environment

Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
16 GB 2133 MHz LPDDR3
Java 15

run benchmark

sbt
project bench
++ 3.0.0
jmh:run -i 2 -wi 1 -f1 -t1

Benchmark #2

data library scala-3
data decode java 92130,460
data decode jackson 517036,354
data decode proto 6716619,956
data library scala-3
data encode java 537462,511
data encode jackson 882065,311
data encode proto 9380874,587
data library scala-3
msg decode proto 11733555,275
data library scala-3
msg encode proto 18486833,582

environment

Apple M1
16 GB
Java 21

run benchmark

sbt
project bench
++ 3.0.0
jmh:run -i 2 -wi 1 -f1 -t1

Usage

You can pick one of the way how to define field number:

  • with annotation @proto.N and use caseCodecAuto
  • explicitly specify nums caseCodecNums(Symbol("field1")->1, Symbol("field2")->2)
  • field numbers by index caseCodecIdx

You can use annotation @proto.RestrictedN to restrict usage of specified field numbers. Can be used with classes or traits.

import scala.collection.immutable.TreeMap
import proto.{encode, decode, N}
import proto.{caseCodecIdx, caseCodecNums, caseCodecAuto}

final case class VectorClock(versions: TreeMap[String, Long])
@RestrictedN(3,4)
final case class Equipment(@N(1) id: String, @N(2) tpe: String)
final case class Car(id: String, color: Int, equipment: List[Equipment], vc: VectorClock)

implicit val tuple2Codec = caseCodecIdx[Tuple2[String, Long]] //codec for TreeMap[String, Long]

implicit val vectorClockCodec = caseCodecIdx[VectorClock]
implicit val equipmentCodec = caseCodecAuto[Equipment]
implicit val carCodec = caseCodecNums[Car]('id->1, 'color->4, 'equipment->2, 'vc->3)

val vc = VectorClock(versions=TreeMap.empty)
val equipment = List(Equipment(id="1", tpe="123"), Equipment(id="2", tpe="456"))
val car = Car(id="1", color=16416882, equipment=equipment, vc=vc)
//encode
val bytes: Array[Byte] = encode(car)
//decode
val car2: Car = decode[Car](bytes)

More examples in testing.scala

Publishing

sbt +publishSigned
open https://oss.sonatype.org/#stagingRepositories

proto's People

Contributors

bosyi avatar doxtop avatar plokhotnyuk avatar shmishleniy avatar tellnobody1 avatar yuriymazepin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

proto's Issues

Warning on latest scala 2.13 for case classes containing collections

Auto-application to () is deprecated. Supply the empty argument list () explicitly to invoke method result,
[error] or remove the empty argument list from its definition (Java-defined methods are exempt).
[error] In Scala 3, an unapplied method like this will be eta-expanded into a function.
[error] implicit val containsCollectionCodec = caseCodecIdx[ContainsCollection];

case class ContainsCollection(c: List[String])
implicit val containsCollectionCodec = caseCodecIdx[ContainsCollection]

scala 2.12 support

Currently Spark 3.0 supports only scala 2.12. It will be very convenient if proto supports scala 2.12 as well. I've created a fork for 2.12 only, and only a few changes are needed. However I don't know how to get it to work on 2.12 and 2.13 at the same time.

update benchmark results

  • drop chill because it does not support scala 2.13
  • drop scodec because it is not convenient to work with bytes and case classes
  • update versions
  • update data model
  • add dto definiton to proto file
  • benchmark dto model
  • add boopickle
  • add kryo-macros

tree codec

Tree data structure is quite complicated by design and it should be checked that codecs for it is effective.

  • Node(1, Seq(Node(2))
  • manual coding (without recursion)
  • benchmark
    • compare tree with flatten list
      • Seq(Node(1), Node(2, 1)))
    • compare with key-value tree
      • tree.root -> Node(1), tree.1.xs.top -> 1, tree.1.xs.1 -> None, tree.1.xs.1.data -> Node(2)
  • macros codec for scalaz.Tree

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.