threading model in dw_node about distwalk HOT 6 CLOSED

Another interesting article on SO_REUSEPORT: https://lwn.net/Articles/542629/; according to the article, we can simply adopt the second option without much worrying about a comparison

from distwalk.

Further useful pointers about the "thundering herd" in multi-threaded accept()/select()/epoll() based servers:

• EPOLLEXCLUSIVE in man epoll_ctl (relevant for multiple epollfd waiting on the same bound socket fd, ie, the first solution in bullet list above);
• https://lwn.net/Articles/637435/
• https://lwn.net/Articles/632590/
• https://uwsgi-docs.readthedocs.io/en/latest/articles/SerializingAccept.html
• "accept() scalability on Linux," Univ. Michigan: http://www.citi.umich.edu/projects/linux-scalability/reports/accept.html

from distwalk.

another problem with the current code base, is that buf_alloc() is called always from the same thread, but the allocated memory buffers are later used from the worker threads. In the new solution, we should call buf_alloc() from the various worker threads, so to have some advantage in allocated memory locality on NUMA servers (ideally, with worker threads pinned to physical cores, yet another cmd-line option to add).
(moving this part to a separate issue #25)

from distwalk.

(Note: commit f029e79 doesn't exist anymore due to typo in comment)

from distwalk.

another problem with the current code base, is that buf_alloc() is called always from the same thread, but the allocated memory buffers are later used from the worker threads. In the new solution, we should call buf_alloc() from the various worker threads, so to have some advantage in allocated memory locality on NUMA servers (ideally, with worker threads pinned to physical cores, yet another cmd-line option to add). (moving this part to a separate issue #25)

This problem should be addressed by default in 4d271aa, since I repurposed epoll_main_loop() itself as the thread body

from distwalk.

nothing left here after merging PR #26, closing.

from distwalk.