Recommendations for Usage about lua-resty-core HOT 14 CLOSED

@splitice Patches welcome :)

from lua-resty-core.

I would if I could ever get dtrace w/ user probes running ;(

from lua-resty-core.

@splitice My suggestion is to always enable lua-resty-core unless it actually slows down the user's OpenResty applications (which is rare though).

from lua-resty-core.

@splitice If you are referring to advanced analysis tools based on dynamic tracing, then the sample-bt, lj-lua-stacks, lj-vm-states tools are the most useful ones:

https://github.com/openresty/nginx-systemtap-toolkit#sample-bt

https://github.com/openresty/stapxx#lj-lua-stacks

https://github.com/openresty/stapxx#lj-vm-states

They can help turn applications that are slower with lua-resty-core to being actually faster with it.

from lua-resty-core.

hi @agentzh
I am using nginx version: openresty/1.7.7.2 on CentOS release 6.6 (Final) with 40 core and 64 GB bare metal server.
I was reading about lua-resty-core and tried it by adding require("resty.core") at first line in my base .lua file the is require by most of my .lua file but surprisingly i found drastic drop in performance. I am curious to know the reason. You have mentioned that use of this library may lead to performance drop in interpreted mode. I want to understand if it is my case.

I have ~300 lua files with ~30K LOC. I have used below command to install OpenResty -

./configure --with-luajit --sbin-path=/usr/sbin --with-http_geoip_module --pid-path=/var/run/nginx.pid -j2 --with-http_stub_status_module

and

[[email protected] ~/platform (master) $] nginx -V
nginx version: openresty/1.7.7.2
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-I/home/platform/installers/ngx_openresty-1.7.7.2/build/luajit-root/usr/local/openresty/luajit/include/luajit-2.1 -O2' --add-module=../ngx_devel_kit-0.2.19 --add-module=../echo-nginx-module-0.57 --add-module=../xss-nginx-module-0.04 --add-module=../ngx_coolkit-0.2rc2 --add-module=../set-misc-nginx-module-0.28 --add-module=../form-input-nginx-module-0.10 --add-module=../encrypted-session-nginx-module-0.03 --add-module=../srcache-nginx-module-0.28 --add-module=../ngx_lua-0.9.14 --add-module=../ngx_lua_upstream-0.02 --add-module=../headers-more-nginx-module-0.25 --add-module=../array-var-nginx-module-0.03 --add-module=../memc-nginx-module-0.15 --add-module=../redis2-nginx-module-0.11 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.13 --add-module=../rds-csv-nginx-module-0.05 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/home/platform/installers/ngx_openresty-1.7.7.2/build/luajit-root/usr/local/openresty/luajit/lib' --sbin-path=/usr/sbin --with-http_geoip_module --pid-path=/var/run/nginx.pid --with-http_stub_status_module --with-http_ssl_module
[[email protected] ~/platform (master) $] lua -v
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

My config file -

server
{
  listen 80;
  listen 443 ssl;
  ssl_certificate     /etc/ssl/certs/test.crt;
  ssl_certificate_key /etc/ssl/certs/test.key;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

  lua_code_cache on;

  server_name ~^([a-z0-9]*).test.com$;

  client_body_in_single_buffer on;
  client_body_buffer_size 24k;
  client_max_body_size 24k;

  keepalive_requests 1000000;
  keepalive_timeout 76s;

  # change server name in header
  more_set_headers 'Server: MyServer/1.0s';

  # log
  set $access_log "";
  log_format ACCESS_LOG $access_log;
  access_log "logs/req-acc.log" ACCESS_LOG buffer=1m flush=5s;
  error_log "logs/production.error.log" error;

  set $CLIENT $1;
  set $CLIENT_REQUEST_URL "http://${CLIENT}.test.com/";
  set $CLIENT_NOTIFY_URL "http://${CLIENT}.test.com/notify";

  location / {
    content_by_lua_file 'code/request.lua';
  }

  location /compressed {
    gzip on;
    gzip_vary on;
    gzip_types 'application/json' 'text/plain';
    content_by_lua_file 'code/request.lua';
  }

  location /html {
    access_log off;
    error_log off;
    content_by_lua "ngx.exit(204)";
  }
}

My server normally handles ~15k request/sec with ~20 ms latency but after adding resty core latency increased to ~250 ms. I receive request with different JSON parameters/value i.e. most of the request are unique and requires separate code path to follow. My clients are set (mostly limited number) of external servers that send us request and each request requires moderate processing at my end.

from lua-resty-core.

@onlynishant Try adding the following line to your init_by_lua* handler:

require "jit".opt.start("minstitch=3")

Enabling LuaJIT's trace stitching can ensure as much Lua code is JIT compiled as possible. The lua-resty-core library heavily uses FFI; those FFI-based Lua code paths would be very slow if being interpreted.

from lua-resty-core.

I tried adding require "jit".opt.start("minstitch=3") in init_by_lua_file and require("resty.core") in content_by_lua_file and ran it for few minutes but latency increased to ~300 ms. Do you think that it will be beneficial in my case?

from lua-resty-core.

@onlynishant No, you should really put require "resty.core" only in your init_by_lua* handler.

from lua-resty-core.

@onlynishant BTW, OpenResty 1.7.7.2 is ancient (released more than 2 years ago) and I don't think its very old lua-resty-core and its very old LuaJIT are in a good shape. You should try the latest OpenResty 1.11.2.4 instead.

from lua-resty-core.

I tried putting require "resty.core" in my init_by_lua_lua handler but didn't work. I am already working on OpenResty 1.11.2.4 up gradation. What are the things that i should be careful about while upgrading from 1.7.7.2 to 1.11.2.4?

from lua-resty-core.

@agentzh I upgraded to 1.11.2.4 and added

require "jit".opt.start("minstitch=3")
require "resty.core"

to init_by_lua_file still no improvement and i am also seeing lot of segfault error (every 1-2 hours, it was with older version also) -
kernel: nginx[40767]: segfault at 7f91ac6cb78e ip 00007f91ac6cb78e sp 00007fff9b149150 error 15

I was expecting it to be fixed by upgrade but unfortunately it didn't. how can i check what's the real reason ?

from lua-resty-core.

@onlynishant Please make sure you are using the lua-resty-core library shipped with that OpenResty. Never use out-of-tree lua-resty-core module versions, which is very likely to be incompatible.

For the segfault itself, you'll have to obtain a gdb full backtrace from a core dump file, otherwise it's not debuggable at all. See

https://www.nginx.com/resources/admin-guide/debug/#backtrace

You may want to use the bt full gdb command instead of backtrace (or bt) as mentioned in this document. Additionally, you may have to configure the core_pattern setting for generating a core dump file properly if you are on Linux. See

https://sigquit.wordpress.com/2009/03/13/the-core-pattern/

Finally, you can try re-building OpenResty with ./configure --with-debug or just use the openresty-debug package from our official Linux package repositories if your operating system is supported. This way we can have internal assertions enabled and hopefully we can make the nginx crash closer to the real culprit if the crash site is far from the real culprit.

from lua-resty-core.

@onlynishant BTW, I believe you should create a dedicated issue for this since it's already OT. Let's stop polluting a closed issue with a different topic.

from lua-resty-core.

@agentzh sure. I will test and start a new issue.

from lua-resty-core.