Code Monkey home page Code Monkey logo

nginx-sflow-module's Introduction

Copyright (c) 2011-2014 InMon Corp. Licensed under the terms of the InMon sFlow licence:
http://www.inmon.com/technology/sflowlicense.txt

                              nginx-sflow-module
                              ==================

  A binary, random-sampling nginx module designed for:
       lightweight,
        centralized,
         continuous,
          real-time monitoring of very large and very busy web farms.

Installation
============
	First download the nginx sources,  then tell the "./configure" script
	that you want to add this module:

        ./configure ... --add-module=/absolute/path/to/nginx-sflow-module
        make
        make install

Configuration
=============
	The module picks up it's sFlow config from the file /etc/hsflowd.auto,
	so the easiest way to configure it is to install that daemon:

	http://host-sflow.sourceforge.net

	Alternatively,  you could type something like this into /etc/hsflowd.auto:

rev_start=1
sampling=100
header=128
polling=10
agentIP=<ip-address>
collector=127.0.0.1 6343
rev_end=1

Where <ip-address> is the local IP address that best represents this host.

Output
======

  The sFlow output goes to a UDP port on your sFlow collector host.
  There you can examine it using a number of tools,  including the
  freeware "sflowtool", which can be downloaded as source code from:
  
  http://www.inmon.com/technology/sflowTools.php

  For example, to log all the fields in ASCII:
 
  $ sflowtool

  Or to log the HTTP samples in common-log-file format:

  $ sflowtool -H


Example output from sflowtool:

	startDatagram =================================
	datagramSourceIP 10.0.0.150
	datagramSize 192
	unixSecondsUTC 1294168545
	datagramVersion 5
	agentSubId 32576
	agent 10.0.0.150
	packetSequenceNo 7
	sysUpTime 25000
	samplesInPacket 1
	startSample ----------------------
	sampleType_tag 0:1
	sampleType FLOWSAMPLE
	sampleSequenceNo 1
	sourceId 3:65537
	meanSkipCount 400
	samplePool 124
	dropEvents 0
	inputPort 0
	outputPort 1073741823
	flowBlock_tag 0:2100
	extendedType socket4
	socket4_ip_protocol 6
	socket4_local_ip 10.0.0.150
	socket4_remote_ip 10.0.0.70
	socket4_local_port 80
	socket4_remote_port 63023
	flowBlock_tag 0:2201
	flowSampleType http
	http_method 2
	http_protocol 1001
	http_uri /membase.php
	http_host 10.0.0.150
	http_useragent Java/1.6.0_22
	http_bytes 3487
	http_duration_uS 24278
	http_status 200
	endSample   ----------------------
	endDatagram   =================================
	startDatagram =================================
	datagramSourceIP 10.0.0.150
	datagramSize 116
	unixSecondsUTC 1294168501
	datagramVersion 5
	agentSubId 32576
	agent 10.0.0.150
	packetSequenceNo 3
	sysUpTime 42000
	samplesInPacket 1
	startSample ----------------------
	sampleType_tag 0:2
	sampleType COUNTERSSAMPLE
	sampleSequenceNo 3
	sourceId 3:65537
	counterBlock_tag 0:2201
	http_method_option_count 0
	http_method_get_count 113
	http_method_head_count 0
	http_method_post_count 0
	http_method_put_count 0
	http_method_delete_count 0
	http_method_trace_count 0
	http_methd_connect_count 0
	http_method_other_count 0
	http_status_1XX_count 0
	http_status_2XX_count 112
	http_status_3XX_count 0
	http_status_4XX_count 1
	http_status_5XX_count 0
	http_status_other_count 0
	endSample   ----------------------
	endDatagram   =================================

nginx-sflow-module's People

Watchers

 avatar  avatar

Forkers

stroboscope

nginx-sflow-module's Issues

FLOWSAMPLE is not displayed

What steps will reproduce the problem?

1. Nginx (1.4.3) compiled with sflow module nginx-sflow-module-0.9.7
2. host sflow agent hsflowd-1.23.4.tar.gz
3.


What is the expected output? What do you see instead?
I see only counter samples, no flow samples.


All the fields releated to sampleType FLOWSAMPLE is missing.

What version of the product are you using? On what operating system?
I am running on Ubbuntu

Please provide any additional information below.


startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 448
unixSecondsUTC 1388048293
datagramVersion 5
agentSubId 100000
agent 10.0.1.1
packetSequenceNo 283
sysUpTime 8470000
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 283
sourceId 2:1
counterBlock_tag 0:2001
adaptor_0_ifIndex 4
adaptor_0_MACs 1
adaptor_0_MAC_0 0a0027000000
adaptor_1_ifIndex 2
adaptor_1_MACs 1
adaptor_1_MAC_0 28d2441cf152
adaptor_2_ifIndex 1
adaptor_2_MACs 1
adaptor_2_MAC_0 000000000000
adaptor_3_ifIndex 3
adaptor_3_MACs 1
adaptor_3_MAC_0 48d22440bd8c
counterBlock_tag 0:2005
disk_total 98844752896
disk_free 76965779456
disk_partition_max_used 34.44
disk_reads 129884
disk_bytes_read 4901825536
disk_read_time 1542304
disk_writes 45740
disk_bytes_written 2375642112
disk_write_time 599256
counterBlock_tag 0:2004
mem_total 16538497024
mem_free 10718740480
mem_shared 0
mem_buffers 93036544
mem_cached 2841763840
swap_total 4194299904
swap_free 4194299904
page_in 2404447
page_out 1159981
swap_in 0
swap_out 0
counterBlock_tag 0:2003
cpu_load_one 0.710
cpu_load_five 0.400
cpu_load_fifteen 0.350
cpu_proc_run 0
cpu_proc_total 601
cpu_num 8
cpu_speed 2401
cpu_uptime 8650
cpu_user 1542000
cpu_nice 2110
cpu_system 424060
cpu_idle 66219210
cpu_wio 386600
cpuintr 0
cpu_sintr 19110
cpuinterrupts 16062097
cpu_contexts 35214887
counterBlock_tag 0:2006
nio_bytes_in 58593324
nio_pkts_in 65095
nio_errs_in 0
nio_drops_in 0
nio_bytes_out 10417578
nio_pkts_out 54820
nio_errs_out 0
nio_drops_out 0
counterBlock_tag 0:2000
hostname AVNI
UUID 10fcbc9807f4e211bf9428d2441cf152
machine_type 3
os_name 2
os_release 3.8.0-33-generic
endSample   ----------------------
endDatagram   =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample   ----------------------
endDatagram   =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample   ----------------------
endDatagram   =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample   ----------------------
endDatagram   =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048302
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 16
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 15
http_status_3XX_count 1
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample   ----------------------
endDatagram   =================================


Original issue reported on code.google.com by [email protected] on 26 Dec 2013 at 8:59

Not able to work with Ganglia

I am trying to use nginx-sflow module with Ganglia-3.4.0 I cannot get it to 
work. 

Looking at ganglia code it looks like it expects HID to be passed to it. 
However, the nginx-sflow module never passes that. 



Original issue reported on code.google.com by [email protected] on 7 Sep 2012 at 11:52

missing FLOWSAMPLE

Compiled new 0.9.4 version and no more segfaults, etc. Which is great.

BUT:

- i am only getting COUNTERSSAMPLE but i am missing FLOWSAMPLE.

How do i activate nginx to send FLOWSAMPLE, cant figure it out...

Original issue reported on code.google.com by [email protected] on 1 Jun 2011 at 6:54

Not working with nginx 1.0.2

Compiled with nginx 1.0.2.

First if you dont have default configfile on filesystem, you are getting this 
error message from every http request:

[error] 3711#0: *1 stat(/etc/hsflowd.auto) failed while logging request

If you make 0 byte /etc/hsflowd.auto, nginx will segault on every http request

Then if trying with example configuration i am getting error:
[error] 3935#0: *2755 line=rev_start tokc=^@rev_start while logging request

So it seems sflow module (0.9.3) does not work atm. 

Original issue reported on code.google.com by [email protected] on 18 May 2011 at 12:24

Wrong counter record data format

What steps will reproduce the problem?
1. set up centOS 7 minimal on hyper-v environment
2. make install nginx with the mod and dependencies
3. rpm install sflowtool

What is the expected output?
0:0 < counterBlock_tag < 0:5 or 0:1000 < counterBlock_tag < 0:1013

What do you see instead?
1012 < counterBlock_tag

What version of the product are you using? On what operating system?
latest as of Jan 7th 2015 on centOS 7 minimal (upgraded to latest as of Jan 7th 
2015

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 8 Jan 2015 at 5:07

Feature Req: Swap default value of 'sflow' directive

What steps will reproduce the problem?
1. Run an sflow-enabled nginx without an existing, valid /etc/hsflowd.auto
2. Lots of noise to error logs


What is the expected output? What do you see instead?
Less noise :)


What version of the product are you using? On what operating system?
0.9.4 with nginx 1.1.0 and 1.1.1

Please provide any additional information below.
I run a lot of different types of servers with the same nginx build. It's 
really easy to generate a mountain of log entries to nginx error log on boxes 
that don't expect to be running sflow and /etc/hsflowd.auto doesn't exist (and 
if it does exist but is empty, I get piles of errors like "syntax error : 
agentIP=<IP address>|<IPv6 address> (on line 0) while logging request...").

I think it would be less surprising to default to 'sflow off;' and have it 
activated with 'sflow on;', rather than defaulting to on. Or alternatively, if 
/etc/hsflowd.auto exists but is empty, then disable sflow logging.

Love the module btw!

Original issue reported on code.google.com by [email protected] on 31 Aug 2011 at 9:02

sflow module blocks nginx to do graceful shutdown

What steps will reproduce the problem?
1. "nginx -s quit" will put worker into "shutting down" state which never 
finishes
2. "nginx -s stop" stops nginx successfully


What is the expected output? What do you see instead?

I would expect nginx workers shut down. Instead they hang in "shutting down" 
status

What version of the product are you using? On what operating system?

0.9.7, Centos 6

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 5 Jan 2014 at 2:45

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.