Nginx requests, memory usage and connections statistics with Munin

Introduction

These three perl scripts are Munin plugins for monitoring the nginx HTTP server.

nginx_status

Based on the one provided in the Debian munin-node package. Which is itself a modified version of the one available at the Munin Plugin Exchange. It shows the connection status for nginx.

Presents the values returned by nginx, using the http_stub_status module, are respectively, the number of:

connections initiated, connections handled, requests served

nginx_connection_request

Presents the number of requests served by connection handled by nginx. The requests-connection ratio handled by the server is the ratio:

requests served / connections handled

This is implemented as a RRDtool datasource of type GAUGE, the default type for Munin.

nginx_request

Based on the one provided in the Debian munin-node package. Which is itself a modified version of the one available at the Munin Plugin Exchange.

It uses a derivative (technically a finite difference) to compute the connection rate value.

nginx_memory

Based on a previous plugin by AkyRhO. It uses ps and awk to get the memory used by the master and worker processes of nginx.

Installation & Configuration

Copy all the scripts to /etc/munin/plugins.
Restart the munin-node daemon: /etc/init.d/munin-node restart.
Done. You should now start to see a new section nginx in your Munin pages with the corresponding graphs.
All scripts have POD documentation. Issue perldoc <script name> to read them.

Environment Settings

Both nginx_request and nginx_status accept two environment variables:

URL: the URL of the nginx status data. The default value is:
http://hostname/nginx_status
UA: the User Agent that the plugin sends in the HTTP header. Since most sites block the libwww-* user agent to get rid of bad bots and mostly up to no good scanners, the scripts send a different User Agent to get around that.

Both variables can be specified in the /etc/munin/plugin-conf.d/munin-node file.

Something like this for altering the URL:

[nginx_status] env.url=http://hostname:8888/nginx-status nginx_status

[nginx_connection_request] env.url=http://hostname:8888/nginx-status nginx_status

[nginx_request] env.url=http://hostname:8888/nginx-status nginx_status

Add env.ua for altering the User Agent.

Don’t forget to restart the munin-node daemon as described in step 2 of the installation instructions above.

Note: the github handling of Org mode markup is broken. Therefore the [nginx_status] [nginx_connection_request] and [nginx_request] lines are separate from the lines specifying the environment variables.

It’s not github’s responsibility but rather a bug in Org-Ruby.

Correct Memory Calculation

Hi Perusio, thank you for your great plugin nginx_memory, but I think there is a problem in calculating the correct memory size of an application.
The plugin accumulates the virtual size for each thread. Here an example why this can't be true:
With your approach all threads for the process php-fpm is accumulated. The total physical memory of this machine is 4GB and you see no swap memory is used and the total used memory is 3540016k (3,376GB).
1063m+1062m+1086m+1069m+1062m+1062m+1062m(root)=7466m=7.29GB
So how can fit 7.29GB fit in memory 4GB RAM without swapping?

The virtual memory is rather a pool which is shared between all subthreads like dynamic/static libraries. Some subthreads can allocate additional memory segments, thats why the virtual memory is not equal for all sub processes.
Instead of the virtual memory VSZ monitor the residential memory RSS.
Changed line:
-my $m = ps u -p \$(pidof nginx) | awk 'NR > 1 {nm += \$5} END {print nm*1024}';
+my $m = ps u -p \$(pidof nginx) | awk 'NR > 1 {nm += \$6} END {print nm*1024}';

`
pstree -p 30138
php-fpm(30138)─┬─php-fpm(30144)
├─php-fpm(30146)
├─php-fpm(30147)
├─php-fpm(30148)
├─php-fpm(30149)
└─php-fpm(30528)

top - 19:25:37 up 6 days, 5:51, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 126 total, 1 running, 125 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3924412k total, 3540016k used, 384396k free, 314576k buffers
Swap: 4063228k total, 240k used, 4062988k free, 378468k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30087 nginx 20 0 162m 6800 1132 S 0.0 0.2 0:00.00 nginx
30088 nginx 20 0 161m 6044 612 S 0.0 0.2 0:00.28 nginx
30144 nginx 20 0 1063m 9532 3700 S 0.0 0.2 0:00.07 php-fpm
30146 nginx 20 0 1062m 5540 1420 S 0.0 0.1 0:00.00 php-fpm
30147 nginx 20 0 1086m 34m 4184 S 0.0 0.9 0:00.81 php-fpm
30148 nginx 20 0 1069m 16m 3776 S 0.0 0.4 0:00.08 php-fpm
30149 nginx 20 0 1062m 5540 1420 S 0.0 0.1 0:00.00 php-fpm
30528 nginx 20 0 1062m 5540 1420 S 0.0 0.1 0:00.00 php-fpm

same with root listed

ps faux|grep php-fpm
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 30138 0.0 0.1 1087632 5872 ? Ss 18:17 0:00 php-fpm: master process (/opt/remi/php54/root/etc/php-fpm.conf)
nginx 30144 0.0 0.2 1088720 9532 ? S 18:17 0:00 _ php-fpm: pool www
nginx 30146 0.0 0.1 1087632 5540 ? S 18:17 0:00 _ php-fpm: pool www
nginx 30147 0.0 0.8 1113012 34852 ? S 18:17 0:00 _ php-fpm: pool www
nginx 30148 0.0 0.4 1095584 16412 ? S 18:17 0:00 _ php-fpm: pool www
nginx 30149 0.0 0.1 1087632 5540 ? S 18:17 0:00 _ php-fpm: pool www
nginx 30528 0.0 0.1 1087632 5540 ? S 18:20 0:00 _ php-fpm: pool www

see whats loaded in the memory:

sudo pmap -d 30138
30138: php-fpm: master process (/opt/remi/php54/root/etc/php-fpm.conf)
Address Kbytes Mode Offset Device Mapping
0000000000400000 3544 r-x-- 0000000000000000 0fd:00000 php-fpm
0000000000975000 588 rw--- 0000000000375000 0fd:00000 php-fpm
0000000000a08000 108 rw--- 0000000000000000 000:00000 [ anon ]
00000000014a3000 2540 rw--- 0000000000000000 000:00000 [ anon ]
00007f69e0c16000 52 r-x-- 0000000000000000 0fd:00000 libnss_files-2.12.so
00007f69e0c23000 2044 ----- 000000000000d000 0fd:00000 libnss_files-2.12.so
00007f69e0e22000 4 r---- 000000000000c000 0fd:00000 libnss_files-2.12.so
00007f69e0e23000 4 rw--- 000000000000d000 0fd:00000 libnss_files-2.12.so
00007f69e0e24000 786432 rw-s- 0000000000000000 0fd:00000 apc.liQKAC (deleted)
00007f6a10e24000 76 r-x-- 0000000000000000 0fd:00000 libzip.so.4.0.0
00007f6a10e37000 2044 ----- 0000000000013000 0fd:00000 libzip.so.4.0.0
00007f6a11036000 4 rw--- 0000000000012000 0fd:00000 libzip.so.4.0.0
00007f6a11037000 52 r-x-- 0000000000000000 0fd:00000 zip.so
00007f6a11044000 2044 ----- 000000000000d000 0fd:00000 zip.so
00007f6a11243000 8 rw--- 000000000000c000 0fd:00000 zip.so

https://linuxconfig.org/ps-output-difference-between-vsz-vs-rss-memory-usage

Cheers,
Paul

perusio / nginx-munin Goto Github PK

nginx-munin's Introduction

Nginx requests, memory usage and connections statistics with Munin

Introduction

nginx_status

nginx_connection_request

nginx_request

nginx_memory

Installation & Configuration

Environment Settings

nginx-munin's People

Contributors

Stargazers

Watchers

Forkers

nginx-munin's Issues

same with root listed

see whats loaded in the memory:

Recommend Projects

Recommend Topics

Recommend Org