This is a set of benchmarks that compares the performance of various RPC libraries/frameworks in specific situations. The benchmark suite is created as part of the rpclib project.

The following libraries are or will be benchmarked:

• rpclib
• cap'n'proto rpc
• Apache Thrift
• grpc
• +1: Direct function calls (for comparison)

The results can be found in this repository (json files and plots, plus a Jupyter Notebook). I've also written a blog post about this: http://szelei.me/rpc-benchmark-part1/

Currently, this benchmark set can only be compiled on Linux. Even though all libraries are cross-platform, the conan packages I'm using aren't.

• A reasonably modern Linux distribution (tested on Arch Linux and Ubuntu 16.04)
• g++ 5.4 or later, or clang-3.8 or later
• cmake 3.x

mkdir build && cd build
conan install .. --build missing
cmake ..
make

If you can't find any conan repo.

• Install rpclib by building from source
• Install google-benchmark from source
• Install capnp
• Install grpc

Regenerate source code to match version and move source files to equivalent folders

#capnp
capnp compile -oc++ ./idl/capnp_service.capnp

#grpc
protoc -I=./idl --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` ./idl/grpc_service.proto

#thrift
thrift --gen cpp ./idl/thrift_service.thrift

I suggest using the run_benchmarks.sh script. It took about 40 minutes no my machine to complete all benchmark runs.

These are the benchmarks that are implemented or will be implemented for each library from the above list.

• get_answer: Perform a function call that returns a number and deserialize the result (so that it's available for the user code as an int)
• get_blob: Perform a function call that returns a large blob and deserialize it into a byte buffer. This is performed in increasing sizes.
• get_struct: Perform a function call that returns a nested struct with lots of fields and lots of data in it and intentionally bad alignment.

Each library is set up to create local client and server that communicate through a TCP socket.

This benchmark suite uses Google Benchmark to manage the benchmarking. Each library has a fixture that is set up and torn down in each benchmark run once (similarly to unit tests). The setup and teardown performance is not measured. Each microbenchmark is performed in a number of iterations (determined by Google Benchmark). All runs are performed five times. The results are the means, medians and standard deviations of the remaining three measured times.

rpclib is a modern msgpack-rpc implementation for C++. It provides a very clean interface inspired by Boost.Python. See rpclib.net for more information.

Probably yes, please send a PR. I'm not an expert in the other libraries, so it's entirely possible that I used them wrong.

One or more of the following reasons:

• I did not know about the library
• There was no conan package for it and I didn't have time to make one
• I didn't have time to learn the library
• The library wasn't to my knowledge widespread enough to be interesting for this comparison.

However, by all means, please let me know if there is one that is worth looking at. Bonus points if you make a conan package and/or pull request with benchmark code for it.