Inaccurate CPU usage on multi-threaded processes about pidusage HOT 21 CLOSED

Available in 2.0.7! Thanks to both of you!

from pidusage.

ping @simonepri

from pidusage.

Can you better describe what you mean by "heavily multi-threaded"?

Please provide:

1. Your OS
2. Your CPU
3. Your node version
4. If you can, a screen of the real resource utilization vs the one reported by pidusage.
   Thank You!

from pidusage.

OS : Linux 4.16.12-1-ARCH (Arch Linux with kernel 4.16.12)
CPU : Ryzen 7 1800x (8 cores, 16 threads)
NodeJS Version : v10.3.0

The screenshot is attached below. CPU usage in the Electron program (bottom right) is being read using the pidusage package, real CPU usage is displayed as 15% in the system's task manager (top left).

from pidusage.

Can you please also provide the output of:

ps -o etime,pid,ppid,pcpu,rss,time

Thank you!

from pidusage.

I ran that command with the PID of the process being monitored supplied as an argument (19918), and this is the output :

ELAPSED   PID  PPID %CPU   RSS     TIME
   03:37 19918 19854  240 39956 00:08:41

from pidusage.

Hi @kittyhacker101!
Thank You.
Does the formula 240 / 16 = 15 % return the CPU percentage that you would expect to see?

@soyuka the man page does not talk about multi-threaded process but I guess that the percentage returned is up to vcore * 100.

If that is true for all platforms, the correct formula would be:
(pcpu / vcore)
Where vcore is os.cpus().length and pcpu is the value reported by ps.
https://github.com/soyuka/pidusage/blob/master/lib/ps.js#L101

from pidusage.

@simonepri I guess that the formula there is indeed correct! Do you think that we should patch the library to take this into consideration? Thing is, sometimes you're monitoring a single-threated process (ie javascript thread) therefore it should not use the above formula, or results won't reflect the reality...

from pidusage.

I believe that we need this #57
And then we can return something like that:

{
  cpu: pcpu / os.cpus().length,               // Percentage (0 - 100)
  threads: 5,                                 // Number of threads of the process
  memory: 357306368,                          // bytes
  ppid: 312,                                  // PPID
  pid: 727,                                   // PID
  ctime: 867000,                              // ms user + system time
  elapsed: 6650000,                           // ms since the start of the process
  timestamp: 864000000                        // ms since epoch
}

Then for instance if pidusage returns something like that:

{
  cpu: 25,
  threads: 2,
  [...]
}

And our system has 4 vcore.

We can interpret the reading as:
The process is consuming the 25% of all the computing power. (100/400)
Either by consuming 100% on a single vcore or X% and 100-X% on two different vcore.

from pidusage.

@soyuka
Can you better explain the following?

Thing is, sometimes you're monitoring a single-threated process (ie javascript thread) therefore it should not use the above formula, or results won't reflect the reality...

from pidusage.

I modified the code in the library, replacing the existing CPU usage formula with the one you specified above, and now it does this (process was using 0% CPU in the system's task manager).

from pidusage.

So, I wrote my own snippet to detect CPU usage ( cpu / (os.cpus().length * 100) * 100), and it shows a value much closer to what the system task manager says.

from pidusage.

Yes sorry, distraction error, actually the correct formula should be just:
cpu / os.cpus().length
We will work on this as soon as possible.

I've updated all the comments with the correct formula.

from pidusage.

Oh, okay.

from pidusage.

Can you better explain the following?

It's silly after a second though.

It'd be great to have the thread count as an additional information indeed.. Maybe we could partially support them on OSes that work? Anyway thanks for the report @kittyhacker101 and for looking into this @simonepri !

from pidusage.

@kittyhacker101 can you confirm that the bug is solved?

from pidusage.

The bug in question is solved, but due to this library's averaging of the CPU usage, it takes a while for it to show the correct value (making it less useful).

from pidusage.

Could you better explain what "a while" means? It should be really fast on linux.

from pidusage.

By "a while", I mean it could take 2+ minutes to adjust to the correct CPU usage.

from pidusage.

That's weird.
It should be as fast as your activity monitor.
Under the hood it uses ps so it should be fast and "accurate".
Could you see if the values given by ps matches with the one given by pidusage?

Thank You for the help!

from pidusage.

hi, i hava a querstion about cpu in lib/ps.js.
Why is it like this var cpu = parseFloat(line[3].replace(',', '.'), 10) / os.cpus().length in version 2.0.7, but find code var cpu = parseFloat(line[3].replace(',', '.'), 10) in version 2.0.8

from pidusage.