sensu-plugins / sensu-plugins-cpu-checks Goto Github PK

sensu-plugins-cpu-checks (0.0.4)

# time /opt/sensu/embedded/bin/metrics-cpu-pcnt-usage.rb | grep real

real    0m1.123s
user    0m0.116s
sys 0m0.006s
# time /opt/sensu/embedded/bin/metrics-cpu-mpstat.rb | grep real

real    0m1.125s
user    0m0.117s
sys 0m0.007s
# time /opt/sensu/embedded/bin/metrics-cpu.rb | grep real

real    0m0.124s
user    0m0.116s
sys 0m0.008s

Ran multiple iterations and the two additional versions seem to always take a second or longer to execute, is the a reason behind this?

metrics-cpu-pcnt-usage.rb only samples CPU usage for a 1 second period, surely this should run for at least 1-5 minutes to provide meaningful data?

The latest gem I can find is 0.0.4 which has a fixed version dependency to sensu-plugin:

sensu-plugin (= 1.2.0)

And this results in the following error after updating sensu to version 0.25.0:

# /opt/sensu/embedded/bin/check-cpu.rb --help
/opt/sensu/embedded/lib/ruby/site_ruby/2.3.0/rubygems/dependency.rb:310:in `to_specs': Could not find 'sensu-plugin' (= 1.2.0) - did find: [sensu-plugin-1.3.0] (Gem::MissingSpecVersionError)

It's already fixed on master:
sensu-plugins-cpu-checks.gemspec#L41

s.add_runtime_dependency 'sensu-plugin', '~> 1.2'

I think there's an issue with the reporting of interrupts from metrics-cpu.rb.

As per https://www.kernel.org/doc/Documentation/filesystems/proc.txt (section 1.8):

The "intr" line gives counts of interrupts serviced since boot time, for each
of the possible system interrupts. The first column is the total of all
interrupts serviced including unnumbered architecture specific interrupts;
each subsequent column is the total for that particular numbered interrupt.

However, in the metrics check

output "#{config[:scheme]}.#{name}", info.last if other_metrics.include? name

This will return the last element of the array for each stat including intr. However, for intr the total is actually the first element. The last column is just the total for a given interrupt (which can actually vary between platform).

Using sensu-plugins-cpu-checks 1.0.0.

Hi,

It appears to me that the metrics-cpu.rb is missing the final column from /proc/stat
This is the guest_nice column.

Here's an example from /proc/stat on a Linux server running kernel version 3.2.0

ben@server:~$ cat /proc/stat |head -n 1
cpu  724635608 1472915 42523717 1190216593 104755408 1555 18433154 58368179 0 0

This has 10 columns.
From https://www.kernel.org/doc/Documentation/filesystems/proc.txt

The meanings of the columns are as follows, from left to right:

• user: normal processes executing in user mode
• nice: niced processes executing in user mode
• system: processes executing in kernel mode
• idle: twiddling thumbs
• iowait: waiting for I/O to complete
• irq: servicing interrupts
• softirq: servicing softirqs
• steal: involuntary wait
• guest: running a normal guest
• guest_nice: running a niced guest

This line of code shows that cpu-metrics.rb is only recording 9 named values:
https://github.com/sensu-plugins/sensu-plugins-cpu-checks/blob/master/bin/metrics-cpu.rb#L45

cpu_metrics = %w(user nice system idle iowait irq softirq steal guest)

The result is that we get an unnamed metric saved, shown by the last line of this output.

ben@server:~$ /opt/sensu/embedded/bin/metrics-cpu.rb |head -n 10
server.cpu.total.user 724691512 1471650046
server.cpu.total.nice 1472915 1471650046
server.cpu.total.system 42529159 1471650046
server.cpu.total.idle 1190347167 1471650046
server.cpu.total.iowait 104771725 1471650046
server.cpu.total.irq 1556 1471650046
server.cpu.total.softirq 18436526 1471650046
server.cpu.total.steal 58377036 1471650046
server.cpu.total.guest 0 1471650046
server.cpu.total. 0 1471650046

The version of sensu-plugin-cpu-checks is 1.0.0

Many thanks for this excellent software.

Hi,

in the Sensu Backend I am getting sh: check-cpu.rb: not found.

I have installed the sensu-plugins-cpu-checks asset with the following manifest:

---
type: Asset
api_version: core/v2
metadata:
  name: sensu-plugins-cpu-checks
spec:
  builds:
  - url: https://s1artproxy:PASSWORD@artifactory:8088/artifactory/remote-sensu-assets/24e5a5d7ff0a5e63b0a6902f1224b7d4592cbdc9/sensu-plugins-cpu-checks_4.1.0_centos7_linux_amd64.tar.gz
    sha512: 8a018622018220323588a0d69ce0ee1e8c3ae72ad708108eb105006e1453a507289445eefb222da838b132106abf86910e2226d2148cc90ae64def9e7d887584
    filters:
    - entity.system.os == 'linux'
    - entity.system.arch == 'amd64'
    - entity.system.platform_family == 'rhel'
    - entity.system.platform_version.split('.')[0] == '7'

But I don't find the check-cpu.rb script on my sensu-agent container. Other plugins install without problems.

How can I check why my installation of sensu-plugins-cpu-checks is failing?

Best regards,
rforberger

Hi, Could MacOS support get added to this since Sensu also has a Mac client now?

I get a dependency error with linux-kstat on MacOS

I have a problem on one of my systems which is showing (according to metrics-cpu-mpstat) nearly 100% cpu utilisation, but according to sar, 75% idle time. The act of measuring the CPU over a second seems to be incorrectly putting up the CPU stats, although not on other identical servers which are 88% idle (according to sar). Running top -d 0.5 shows that the check-rabbitmq checks run at high CPU when they come through. If they coincide with the cpu metrics run, the stats will be skewed. However, running metrics-cpu.rb just takes the counter, and compares it against the value the next time it is run, so gives a true picture of the amount of CPU utilisation over that interval.

The filters debian 9 are too broad, either remove or add debian platform_family and version to filter

- platform: "debian9"
  arch: "amd64"
  asset_filename: "#{repo}_#{version}_debian9_linux_amd64.tar.gz"
  sha_filename: "#{repo}_#{version}_sha512-checksums.txt"
  filter:
  -  "entity.system.os == 'linux'"
  -  "entity.system.arch == 'amd64'"

check-cpu.rb alerts sometimes with total cpu being 100% busy, but the top output captured at the same time as a part of hooks doesn't show any load.

Explanation by @majormoses:

the way that it detects it is by gathering the current and then gathering it again (defaults to 1s) and measuring the difference. As the very act of doing so means that you are inflating the cpu utilization by loading up ruby. In addition because of the short timespan it collects data for the numbers will also not be 100% accurate. You can use the --sleep option to control how long it waits before using another data point, the longer you wait the more accurate it will be. In an ideal world I would opt for changing the default value to 5 as that is what I found was a good compromise for accuracy and being able to still get very granular/frequent checks but the problem is that someone might be setting their thresholds to less than 5 seconds which would cause issues as you'd have multiple instances of the check running at the same time. I am willing to do it as a breaking change as I think its a good move forward, just giving context of where it is and why it was not changed previously.

Change default 1 seconds to something higher to get more accurate data and avoid un-necessary alerts.

Latest release is still showing as 0.0.3, but 0.0.4 has been deployed....

Specify the location to /proc.

An example use case is monitoring the host inside a container. E.g. http://support.sysdigcloud.com/hc/en-us/articles/206770633-Sysdig-Install-Kubernetes-DaemonSet

It looks like we're only linking *.rb executables. Would it be a good idea to also link the .sh versions of plugins here, or are there reasons we'd rather avoid this? https://github.com/sensu-plugins/sensu-plugins-cpu-checks/blob/master/sensu-plugins-cpu-checks.gemspec#L15

The --proc-path long option is being set as a float instead of a string. The default for this option works fine, but when a user supplied value is passed that value is then converted to a float. The value of config[:proc_path] then becomes 0.0.

Assuming a scenario where you are running the Sensu client as a container with the host /proc/ file system mounted at /rootfs/proc, the following output is the error produced:

/opt/sensu/embedded/bin/check-cpu.rb --proc-path=/rootfs/proc
Check failed to run: No such file or directory @ rb_sysopen - 0.0/stat, ["/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-cpu-checks-1.0.0/bin/check-cpu.rb:71:in `initialize'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-cpu-checks-1.0.0/bin/check-cpu.rb:71:in `open'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-cpu-checks-1.0.0/bin/check-cpu.rb:71:in `acquire_cpu_stats'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-cpu-checks-1.0.0/bin/check-cpu.rb:79:in `run'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugin-1.4.4/lib/sensu-plugin/cli.rb:58:in `block in <class:CLI>'"]

In the above output you can see what should be /rootfs/proc/stat is 0.0/stat, because that user input is being converted to a float.

tl;dr: Suggest changing platform to platform_family.

After adding this asset to a Sensu Backend, using sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks --rename sensu-plugins-cpu-checks, the current 4.0.0 release shows this Bonsai metadata (sensuctl asset list --format=yaml):

---
type: Asset
api_version: core/v2
metadata:
  annotations:
    io.sensu.bonsai.api_url: https://bonsai.sensu.io/api/v1/assets/sensu-plugins/sensu-plugins-cpu-checks
    io.sensu.bonsai.name: sensu-plugins-cpu-checks
    io.sensu.bonsai.namespace: sensu-plugins
    io.sensu.bonsai.tags: ruby-runtime-2.4.4
    io.sensu.bonsai.tier: Community
    io.sensu.bonsai.url: https://bonsai.sensu.io/assets/sensu-plugins/sensu-plugins-cpu-checks
    io.sensu.bonsai.version: 4.0.0
  name: sensu-plugins-cpu-checks
  namespace: default
spec:
  builds:
  - filters:
    - entity.system.os == 'linux'
    - entity.system.arch == 'amd64'
    - entity.system.platform == 'debian'
    headers: null
    sha512: da5a183ad1a1f76962561eed659c6184b3e5d6a412432e1ccb4297cce123c41f9a8bb4fdb9ab19663aada978ea862ab2bd5f00bc91e4769ddd87543f5662b3af
    url: https://assets.bonsai.sensu.io/68546e739d96fd695655b77b35b5aabfbabeb056/sensu-plugins-cpu-checks_4.0.0_debian_linux_amd64.tar.gz

(I removed rhel and alpine sections.)

This prevents the asset-managers in sensu agents running on Ubuntu from fetching this asset when running checks. I don't think exluding Ubuntu is intentional?

Thus I suggest changing the metadata from platform to platform_family, and quickly release a 4.0.1.

I feel that this could be invalid, but I'm opening an issue after spending 4 hours trying to find documentation on the subject in the hopes that a quick answer can be provided. I am attempting to upgrade from Sensu 1.x to Sensu 5.x and am having difficulty with this plugin. Previously I would use '--scheme production.:::name:::' to prefix my metrics, and the hostname would be substituted. This does not appear to be the case now as the substitution is not taking place. Is there some other way of handling this under Sensu 5.x?

Hi,
having installed sensu 1.4.2-4 on windows, the command
C:\opt\sensu\embedded\bin\gem install sensu-plugins-cpu-checks fails with error:

ERROR: Could not find a valid gem 'linux-kstat' (= 0.1.3) in any repository
ERROR: Possible alternatives: linux-kstat

When I try to download linux-kstat I see only 0.1.0 using gem install linux-kstat I see only 0.1.0 version available.

Best regards,