threading model, handle async request about netflex HOT 3 OPEN

Hi @dio ,

Thanks for your interest in Netflex!

May I ask you what you want to know in particular concerning async request handling?
If you are talking from a user point of view, as long as you registered you route, it will be executed asynchronously once a client connects to your server and contacted the registered route:

//! netflex server
netflex::http::server server;

//! routes
//! /:var_name provides you a way to define URLs including variables
server.add_route({netflex::http::method::GET, "/users/:user_name/articles/:post_id",
  [](const netflex::http::request& request, netflex::http::response& response) {

    //! This is executed asynchronously for each relevant client requests

    response.set_body("What's up?!\n");
    response.add_header({"Content-Length", "12"});
  }});

//! run server
server.start("0.0.0.0", 3001);

Or maybe you are looking for more in-depth details concerning the implementation of the library itself?

Just let me know, I'll try to answer your question as much as I can :)

Best

from netflex.

👋

Or maybe you are looking for more in-depth details concerning the implementation of the library itself?

Yes, please. I've been playing with facebook's proxygen and envoy, they are both based on libevent and have a thread per loop (I'm oversimplifying it, more on envoy threading model: https://blog.envoyproxy.io/envoy-threading-model-a8d44b922310 I think it is very similar to proxygen). Moreover, for proxygen, it separates the type of thread pools: IO and CPU based. I'd like to understand how will you take this with NetFlex.

I'm also interested in pink (https://github.com/Qihoo360/pink).

Thanks 🙌!!!

from netflex.

Hey @dio,

Sorry for the time to reply, kind of working on several libraries at the same time, so not always easy to manage the time between each :)

Currently, Netflex is based on tacopie for the network part. That is a C++ TCP networking libraries I developed originally for cpp_redis and that aim to be as lightweight as possible without any dependencies.

Tacopie relies on an io_service to manage the read and write events.

Basically, the io_service has a background thread calling select() to listen for incoming read and write events for specific sockets. The select() sleep indefinitely until an event occurs to reduce the CPU consumption. However, tacopie uses the self-pipe trick to manually wake up the select() if necessary (typically, if the set of sockets to watch after has changed). Whenever a read or write event is detected, the event is processed and a callback chain will be called to propagate the event to the library user.

Those callbacks are executed by some workers. A worker is basically a thread of a thread pool. The user can change the size of this thread pool to increase or decrease the number of workers depending on the workload. The more workers, the more callbacks can be executed simultaneously which can be necessary if you start dealing with a heavy workload. Because it is a thread pool, threads are created up-front, which means the cost of getting the task processed by one of the worker is reduced. The other advantage is that the size of the threadpool can be changed at runtime, meaning the program can adapt automatically depending on the current needs.

The main difference is that tacopie is not based on a generic event loop for which any kind of tasks can be pushed into. Instead it is based on a thread monitoring the socket activity and then delegating tasks to workers.
I haven't looked at proxygen, but seems like the separation between IO and CPU based thread is the equivalent of my select() thread vs the workers.

Let me know if you have any other questions or if some points are still confused for you.

Best

from netflex.