Code Monkey home page Code Monkey logo

bilitw's Introduction

BILITW (bilibili twemproxy)

twemproxy (pronounced "two-em-proxy"), aka nutcracker is a fast and lightweight proxy for memcached and redis protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipelining and sharding enables you to horizontally scale your distributed caching architecture.

bilitw (bilibili twemproxy), which introduce multi process of twemproxy(one master and mutli worker), is order to get full use of the CPU cores.

Build

To build bilitw from source with debug logs enabled and assertions enabled:

$ git clone 
$ cd bilitw
$ autoreconf -fvi
$ ./configure  CFLAGS="-DGRACEFUL -g -O2" --enable-debug=full
$ make
$ make install
$ nohup bilitw -o /var/log/bilitw.log  -v 3 &

A quick checklist:

  • Use newer version of gcc (older version of gcc has problems)
  • Use CFLAGS="-O1" ./configure && make
  • Use CFLAGS="-O3 -fno-strict-aliasing" ./configure && make
  • autoreconf -fvi && ./configure needs automake and libtool to be installed

Configuration

stats_duration indicates the interval of statistics printout in log file.   
default: 300000 milliseconds -> 5 minutes.

reload_timeout indicates the timeout for graceful reload, if the timeout reached, old workers will close the client connection.  
default: 120000 milliseconds -> 2 minutes

slow_req_duration is the timeout for slow request value
slow request is the response of redis/memcached exceeds slow_req_duration timeout.  
default: not set by default

Default under /etc/nutcracker.yml

Block "global" is for global setting of bilitw.

global:
 stats_duration: 600000
 reload_timeout: 60000
 slow_req_duration: 5000
 stats_file: /tmp/twstats.log

alpha:
 listen: 127.0.0.1:22121
 slow_req_duration: 3000
 hash: fnv1a_64
 distribution: ketama
 auto_eject_hosts: true
 redis: true
 server_retry_timeout: 2000
 server_failure_limit: 1
 servers:
  - 127.0.0.1:6379:1 myserver
  - 127.0.0.1:6380:1 myserver1

Graceful Reload Configuration:

localhost:~:# ps -ef | grep bilitw
root     19521 19227  0 19:22 pts/0    00:00:00 bilitw master
root     19522 19521  0 19:22 pts/0    00:00:00 bilitw worker 0
root     19523 19521  0 19:22 pts/0    00:00:00 bilitw worker 1
kill -SIGHUP 19521

Unit Test

Dependency: yum install python yum install python-redis

For Redis: ./tests/test.py

Help

$ bilitw -h
Options:
-h, --help             : this help
-V, --version          : show version and exit
-t, --test-conf        : test configuration for syntax errors and exit
-d, --daemonize        : run as a daemon
-D, --describe-stats   : print stats description and exit
-v, --verbose=N        : set logging level (default: 5, min: 0, max: 11)
-o, --output=S         : set logging file (default: stderr)
-c, --conf-file=S      : set configuration file (default: /etc/nutcracker.yml)
-s, --stats-port=N     : set stats monitoring port (default: 22223)
-a, --stats-addr=S     : set stats monitoring ip (default: 0.0.0.0)
-i, --stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)
-p, --pid-file=S       : set pid file (default: off)
-m, --mbuf-size=N      : set size of mbuf chunk in bytes (default: 16384 bytes)
-n, --worker-num=N     : set number of workers (default: number of cpu cores)
-M, --core-mask=N      : set cpu core mask that worker process bind to

bilitw -n 2 -M 12    
// 2 means launch 2 workers
// 12 means mask bitmap 0x1100, which tell 2 workers to bind on cpu 2 and cpu 3. 

localhost:~:# ps -ef | grep bilitw
root     19521 19227  0 19:22 pts/0    00:00:00 bilitw master
root     19522 19521  0 19:22 pts/0    00:00:00 bilitw worker 0
root     19523 19521  0 19:22 pts/0    00:00:00 bilitw worker 1

[2015-12-21 19:20:15.247] nc_process.c:256 set worker 0 affinity to cpu core 2
[2015-12-21 19:20:15.247] nc_process.c:256 set worker 1 affinity to cpu core 3

Observability

Observability in bilitw is through logs and stats.

$ bilitw --describe-stats

pool stats:
  client_eof          "# eof on client connections"
  client_err          "# errors on client connections"
  client_connections  "# active client connections"
  server_ejects       "# times backend server was ejected"
  forward_error       "# times we encountered a forwarding error"
  fragments           "# fragments created from a multi-vector request"

server stats:
  server_eof          "# eof on server connections"
  server_err          "# errors on server connections"
  server_timedout     "# timeouts on server connections"
  server_connections  "# active server connections"
  requests            "# requests"
  request_bytes       "total request bytes"
  responses           "# responses"
  response_bytes      "total response bytes"
  in_queue            "# requests in incoming queue"
  in_queue_bytes      "current request bytes in incoming queue"
  out_queue           "# requests in outgoing queue"
  out_queue_bytes     "current request bytes in outgoing queue"

Logging in bilitw is only available when bilitw is built with logging enabled. By default logs are written to stderr. bilitw can also be configured to write logs to a specific file through the -o or --output command-line argument.

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

bilitw's People

Contributors

anewhuahua avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.