Code Monkey home page Code Monkey logo

http2comm's Introduction

C++ HTTP/2 wrapper library

License: MIT Documentation Ask Me Anything ! Maintenance Main project workflow

This library is based on @tatsuhiro-t nghttp2 library (https://github.com/nghttp2/nghttp2). It offers a quick way to instantiate a client or server and define their virtual methods to process the requests and answers properly.

You could check an example of use at h2agent project, where the server side capability is used to mock an HTTP/2 service within a testing ecosystem.

Project image

This image is already available at github container registry and docker hub for every repository tag, and also for master as latest:

$> docker pull ghcr.io/testillano/http2comm:<tag>

You could also build it using the script ./build.sh located at project root:

$> ./build.sh --project-image

This image is built with ./Dockerfile. Both ubuntu and alpine base images are supported, but the official image uploaded is the one based in ubuntu.

Usage

To run compilation over this image, just run with docker. The entrypoint (check it at ./deps/build.sh) will fall back from cmake (looking for CMakeLists.txt file at project root, i.e. mounted on working directory /code to generate makefiles) to make, in order to build your source code. There are two available environment variables used by the builder script of this image: BUILD_TYPE (for cmake) and MAKE_PROCS (for make):

$> envs="-e MAKE_PROCS=$(grep processor /proc/cpuinfo -c) -e BUILD_TYPE=Release"
$> docker run --rm -it -u $(id -u):$(id -g) ${envs} -v ${PWD}:/code -w /code \
          ghcr.io/testillano/http2comm:<tag>

Build project with docker

Builder image

This image is already available at github container registry and docker hub for every repository tag, and also for master as latest:

$> docker pull ghcr.io/testillano/http2comm_builder:<tag>

You could also build it using the script ./build.sh located at project root:

$> ./build.sh --builder-image

This image is built with ./Dockerfile.build. Both ubuntu and alpine base images are supported, but the official image uploaded is the one based in ubuntu.

Usage

Builder image is used to build the project library. To run compilation over this image, again, just run with docker:

$> envs="-e MAKE_PROCS=$(grep processor /proc/cpuinfo -c) -e BUILD_TYPE=Release"
$> docker run --rm -it -u $(id -u):$(id -g) ${envs} -v ${PWD}:/code -w /code \
          ghcr.io/testillano/http2comm_builder:<tag>

You could generate documentation passing extra arguments to the entry point behind:

$> docker run --rm -it -u $(id -u):$(id -g) ${envs} -v ${PWD}:/code -w /code \
          ghcr.io/testillano/http2comm_builder::<tag>-build "" doc

You could also build the library using the script ./build.sh located at project root:

$> ./build.sh --project

Build project natively

This is a cmake-based building library, so you may install cmake:

$> sudo apt-get install cmake

And then generate the makefiles from project root directory:

$> cmake .

You could specify type of build, 'Debug' or 'Release', for example:

$> cmake -DCMAKE_BUILD_TYPE=Debug .
$> cmake -DCMAKE_BUILD_TYPE=Release .

You could also change the compilers used:

$> cmake -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DCMAKE_C_COMPILER=/usr/bin/gcc

or

$> cmake -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_C_COMPILER=/usr/bin/clang

Requirements

Check the requirements described at building dockerfile (./Dockerfile.build) as well as all the ascendant docker images which are inherited:

http2comm builder (./Dockerfile.build)
   |
nghttp2 (https://github.com/testillano/nghttp2)

Build

$> make

Clean

$> make clean

Documentation

$> make doc
$> cd docs/doxygen
$> tree -L 1
     .
     ├── Doxyfile
     ├── html
     ├── latex
     └── man

Install

$> sudo make install

Optionally you could specify another prefix for installation:

$> cmake -DMY_OWN_INSTALL_PREFIX=$HOME/mylibs/ert_http2comm
$> make install

Uninstall

$> cat install_manifest.txt | sudo xargs rm

Integration

CMake

Embedded

Replication

To embed the library directly into an existing CMake project, place the entire source tree in a subdirectory and call add_subdirectory() in your CMakeLists.txt file:

add_subdirectory(ert_http2comm)
...
add_library(foo ...)
...
target_link_libraries(foo PRIVATE ert_http2comm::ert_http2comm)
FetchContent

Since CMake v3.11, FetchContent can be used to automatically download the repository as a dependency at configure type.

Example:

include(FetchContent)

FetchContent_Declare(ert_http2comm
  GIT_REPOSITORY https://github.com/testillano/http2comm.git
  GIT_TAG vx.y.z)

FetchContent_GetProperties(ert_http2comm)
if(NOT ert_json_POPULATED)
  FetchContent_Populate(ert_http2comm)
  add_subdirectory(${ert_http2comm_SOURCE_DIR} ${ert_http2comm_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()

target_link_libraries(foo PRIVATE ert_http2comm::ert_http2comm)

Contributing

Please, execute astyle formatting (using frankwolf image) before any pull request:

$> sources=$(find . -name "*.hpp" -o -name "*.cpp")
$> docker run -i --rm -v $PWD:/data frankwolf/astyle ${sources}

http2comm's People

Contributors

testillano avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

http2comm's Issues

Implement basic congestion control

Implement a basic algorithm based in:

  • number of busy threads
  • size of FIFO queue dispatcher (when used)

Provide a maximum queue size to be allowed. Disabled by default (-1) this algorithm will activate the congestion situation when no idle consumers are available (busy threads are equal to consumers launched) and queue size is over the maximum provided.

Simplify method enum class to use string

Is your feature request related to a problem? Please describe.
Method in 3rd party library is std::string but we use enum class.
So, we need conversions (takes time) and introduce complexity without need.

Describe the solution you'd like
Use std::string instead.

Fix coredump on delayed responses for broken connection

Describe the bug
Delayed responses over broken connection crashes on response post.

To Reproduce
Provision a delay of 2 seconds and send a request with a smaller timeout (1 second).
When the connection is broke, the delayed event crashes.

Include general server sequence

Is your feature request related to a problem? Please describe.
A monotonically increased sequence should be managed at this library, to simplify upper layers.

Describe the solution you'd like
Use an atomic uint64 and increment within on_data lambda

Refactor metrics families' names

Reorganize metrics to better fit grafana expectatives. Redundant family name with "source" label to ease identification. Provide class name (which will be application name plus context information: module or http2 endpoint), to better configure externally.

Provide reception timestamp in microseconds to server receive() virtual method

Is your feature request related to a problem? Please describe.
This reception timestamp should be shared with the application user in order to better sinchronize this value with application events regarding latencies measured in this library metrics.

Describe the solution you'd like
Modify receive() at Http2Server class.

Maximum Request Body Size must not be static within Stream class

Is your feature request related to a problem? Please describe.
The current design manages the maximum request body for every stream, regardless the server instantiated by the application.
This is because the value is an static member of stream class.

Describe the solution you'd like
Maximum request body size should be non-static and controlled under http2 server class.

Additional context
Memory reservation size will be affected by traffic managed for foreign servers, as currently it is stored in an static atomic variable.

Improve reception performance

Is your feature request related to a problem? Please describe.
stringstream has worse performance than string, also we should provide to upper layer, a better usable interface. Const std::string reference is much better than a shared pointer to stringstream buffer.

Make configurable the server activity keepalive

Is your feature request related to a problem? Please describe.
The read timeout can be configurable in nghttp2 API.

Describe the solution you'd like
We will add this new parameter in the interface. Keep default value of 60 seconds as nghttp2 does.

Performance optimizations

Is your feature request related to a problem? Please describe.
Sometimes, library user don't need to process the request body received.
In such case, transforming a stringstream to std::string (via str()) is overkilling.
We will modify the interface to pass shared pointer to initial stringstream, and this could be
trnasformed on-demand later.

Also, we will calculate stringstream size withouth wasting time building str(). We will use tellg and seekg.

Dynamic threads grow until maximum provided

Is your feature request related to a problem? Please describe.
Just an improvement to allow to provide a maximum value for number of threads, so the threads initially created could grow to a top value configured.

Describe the solution you'd like
Add a max value for worker threads which defaults to initial worker threads.
Http2Server interface adds a new size_t parameter, so it is non backward compatible.

Describe alternatives you've considered
Threads are not decreased when no longer needed, so be careful when configuring the max value.

ReceptionId must not be static within Stream class

Is your feature request related to a problem? Please describe.
The current design increases monotonically the reception id for every stream, regardless the server instantiated by the application.
This is because the value is an static member of stream class.

Describe the solution you'd like
Reception id sequence should be non-static and controlled under http2 server class.

Additional context
Current approach is causing h2agent CT fail for this test:
FAILED server-data_operation/delete_data_test.py::test_001_i_want_to_delete_partial_internal_data_after_storing_some_traffic_events
This is because administrative flows are also increasing globally the sequence, but we want that sequence specific of each server.
Administrative server is not using that secuence, but also will be increased for every administrative request and only for them. Traffic requests only should increase traffic server sequence and admin traffic only should increase admin server sequence.

Improve timeout implementation for client connections

Is your feature request related to a problem? Please describe.
Timeout is poorly managed.

Describe the solution you'd like
We need to have an agreement to signal timeout event.
We need also to discard further receptions on timeout

Describe alternatives you've considered
Add a timed_out boolean flag to response task.

Fix memory leak with delay timers

Describe the bug
There is a tiny memory leak if delay responses are used for http2 server.

To Reproduce
long term traffic with a provision including delays in responses.

Improve metrics design for grafana

Family names never must e dynamic, so, class name is not an endpoint or something that could change (at least for clients).
Also, application/process name must not be provided in class name to be part of family name.
So, class name (fo1r Http2Server and Http2Client classes) shall be static. And the source label, which is dynamic, and could carry application/process name and client endpoints name, will be provided on enableMetricss() method.

Reserve initial memory for request body storage

Is your feature request related to a problem? Please describe.
Implement configurable pre reserve memory allocation for request body string container. This avoids possible reallocations done by std::string append() when the string object is constructed from scratch without initial memory reservation.

Describe the solution you'd like
The amount used for that reservation is the maximum size registered for incoming request bodies, something stored in static atomic member for Stream class.
The behavior will be configurable through virtual method preReserveRequestBody().

Add busy threads count to queue dispatcher

Is your feature request related to a problem? Please describe.
Just to improve queue dispatcher monitoring, it would be interesting to know the
number of worker threads doing processing of data received.

Add dynamic counters for status codes

Is your feature request related to a problem? Please describe.
We would improve metrics capabilities adding dynamic counters for every different registered status code.
This must be done for both server and client endpoints.

New virtual method to ignore request bodies

Is your feature request related to a problem? Please describe.
Add new virtual method to ignore request bodies

A new virtual method could be reimplemented to
ignore the request body storage under some
conditions, in order to speed up the performance
with high load.

This is useful when huge requests are received and not need to use their bodies.

Describe the solution you'd like
ignoreRequestBody, returns bool to ignore (false on base class default implementation).
Provide req structure to analyze.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.