performancecopilot / parfait Goto Github PK

Several modules (at least 7) use SLF4J instead of Log4J.
However, parfait-agent and one class in partait-core (QuiescentRegistryListener) use Log4J directly.

Therefore either all should use SLF4J or see #71 SLF4J could be replaced with Log4J2 as well and the SLF4J dependency removed.

Parfait currently does not support arrays of any type even though already OpenJDK JVM exposes some JMX metrics as int/long/String arrays. They could perhaps be represented as compound metrics in PCP if supported by Parfait.

It would be nice to allow Micrometer (https://micrometer.io/) to be exposable to PCP via metrics the same way as Dropwizard.
Not only Spring, but several other popular frameworks now support Micrometer.

With Java 8 on RHEL 7 and Parfait agent compiled from git master:

$ cd parfait.git
$ mvn clean package
$ java -javaagent:$(pwd)/parfait-agent/target/parfait-agent-jar-with-dependencies.jar KeyboardReader
[main] INFO io.pcp.parfait.pcp.PcpMonitorBridge - PCP monitoring bridge started for writer [PcpMmvWriter[byteBufferFactory=FileByteBufferFactory[file=/var/lib/pcp/tmp/mmv/KeyboardReader]]]
Enter a line of input

$ pmrep mmv   
Invalid metric mmv.KeyboardReader.java.memorypool.tenured (PM_ERR_INDOM Unknown or illegal instance domain identifier).

Only a very few metrics (e.g., mmv.KeyboardReader.java.jvm.classloader.loaded.*) are actually working.

I think this was reported already earlier but either the report or the patch was lost at some point:

$ parfait --name test --connect localhost:9875
[main] INFO io.pcp.parfait.pcp.PcpMonitorBridge - PCP monitoring bridge started for writer [PcpMmvWriter[byteBufferFactory=FileByteBufferFactory[file=/var/lib/pcp/tmp/mmv/parfait17402]]]

$ pminfo mmv | sort
mmv.control.debug
mmv.control.files
mmv.control.reload
mmv.parfait17402.java.jvm.classloader.loaded.current
...

From [email protected] on February 17, 2011 12:57:41

Based on MSN conversation with Cowan:

10:59
Paul Smith
10:59
old skool, each thread created an event right?

10:59
Paul Cowan
my favourite type of thinking

10:59
Paul Smith
and a master thread wrote that value.
now the new way
each thread writes direct

10:59
Paul Cowan
yes

10:59
Paul Smith
yet that is synchronized on that ByteBuffer or whatever.

10:59
Paul Cowan
yes

10:59
Paul Smith
the former isn't really scalable, because a single thread can't possibly keep up with some level of events
but...
with X threads all competing for that synchronized block any better… ?

11:00
Paul Cowan
yeah, that's what we need to find out
I suspect it's not WORSE
but it would be good to work out

11:00
Paul Smith
the time spent in that critical section is pretty small.

11:00
Paul Cowan
the alternative is to have an ImmutableMap (hence thread-safe) of <MetricName, ByteBuffer>

11:00
Paul Smith
but still, as we add more metrics, and/or add more threads etc it's going to be highly contended.

11:01
Paul Cowan
where each bytebuffer JUST maps the chunk of ram for that metric

11:01
Paul Smith
a slice of the original buffer

11:01
Paul Cowan
WHOAH I JUST BLEW MY OWN MIND

11:01
Paul Smith
that should be easy.

11:01
Paul Cowan
yep

11:01
Paul Smith
that needs an issue to capture..
that's awesome.
on 'start' the original buffer is sliced up into that map.

Original issue: http://code.google.com/p/parfait/issues/detail?id=9

From [email protected] on September 05, 2014 15:47:08

What steps will reproduce the problem? mvn install -DskipTests
executing the above command will reproduce the problem What is the expected output? What do you see instead? jar will be created. But it threw error What version of the product are you using? On what operating system? parfait-0.3.0.RC1
CentOS release 6.3 (Final) Please provide any additional information below.

Attachment: err.txt

Original issue: http://code.google.com/p/parfait/issues/detail?id=13

From [email protected] on August 28, 2010 07:18:17

This is an ER, but right now it seems, GCode doesn't distinguish those yet (e.g. via tags ?;-)

I know there's already a base JDBC monitor, so I simply want to raise the question, if persistence tools like Hibernate, iBatis, JPA, JDO or others
may have additional monitoring requirements?

Original issue: http://code.google.com/p/parfait/issues/detail?id=1

Parfait already requires Java 8 which introduced LongAdder. From https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAdder.html:

This class is usually preferable to AtomicLong when multiple threads update a common sum that is used for purposes such as collecting statistics, not for fine-grained synchronization control. Under low update contention, the two classes have similar characteristics. But under high contention, expected throughput of this class is significantly higher, at the expense of higher space consumption.

Counters implemented with AtomicInteger should be migrated to LongAdded as appropriate.

Unless I am mistaken, there is no mechanism for removing a Monitorable from the MonitorableRegistry.

Could removal of a Monitorable be implemented?

I am trying to integrate Parfait with PCP, using the code based on PcpMonitorBridgeIntegrationTest.java.
Here is the .mmv producing code:

public static void main(String[] args) throws InterruptedException, IOException {
    final FileByteBufferFactory bufferFactory = new FileByteBufferFactory(new File("/tmp/filebuff.mmv"));
    final PcpWriter writer = new PcpMmvWriter(bufferFactory, IdentifierSourceSet.DEFAULT_SET);
    final PcpMonitorBridge pcpMonitorBridge = new PcpMonitorBridge(writer);
    final MonitorableRegistry registry = new MonitorableRegistry();
    final DynamicMonitoringView dynamicallyModifiedView = new DynamicMonitoringView(registry, pcpMonitorBridge, 1000);

    final DummyMonitorable monitorable = new DummyMonitorable("simplestudy.testmetric");
    registry.register(monitorable);

    dynamicallyModifiedView.start();
    Thread.sleep(5000);
    dynamicallyModifiedView.stop();
}

private static class DummyMonitorable extends AbstractMonitorable<String> {
    protected DummyMonitorable(String name) {
        super(name, "Description of " + name, String.class, Unit.ONE, ValueSemantics.CONSTANT);
    }

    @Override
    public String get() {
        return "This is just a test";
    }
}

My .mmv file is accepted by mmv, but during the processing something goes wrong. The only unusual thing that I see in logs is:

__pmUnpinPDUBuf(0x25372d0) -> fails

However it doesn't give me any clue on how to fix it. Any help would be greatly appreciated!

Here is the mmv.log (recording started right before mmv.reload set):

pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 28, last read 28, still need 0
[18091]pmGetPDU: PMNS_NAMES fd=0 len=40 from=2
000:       28     700e        2  b000000        0  1000000  a000000 2e766d6d 
008: 6f6c6572 7e7e6461 
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: Received PDU_PMNS_NAMES
__pmDecodeNameList
namelist dump: numnames = 1
  name[0]: "mmv.reload"
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: MMV: pmdammv: reloading
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: MMV: create_client_stat: filebuff.mmv, /var/lib/pcp/tmp/mmv/filebuff.mmv
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: pmdaRehash: PMDA pmdammv: successful rebuild
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: MMV: pmdammv: 2 metrics and 0 indoms after reload
__pmSendIDList
IDlist dump: numids = 1
  PMID[0]: 0x11800000 70.0.0
__pmFindPDUBuf(24) -> 0x25372b0
   pinned pdubuf[size](pincnt): 0x25372b0...0x25372c7[24](1) 0x2537880...0x2537c7f[1024](1)
[18091]pmXmitPDU: PMNS_IDS fd=1 len=24
000:       18     700d        0        0  1000000     8011 
__pmUnpinPDUBuf(0x25372b0) -> pdubuf=0x25372b0, pincnt=0
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)
pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 4, last read 4, still need 0
[18091]pmGetPDU: DESC_REQ fd=0 len=16 from=2
000:       10     7004        2     8011 
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: Received PDU_DESC_REQ
__pmFindPDUBuf(32) -> 0x2537310
   pinned pdubuf[size](pincnt): 0x2537310...0x253732f[32](1) 0x2537880...0x2537c7f[1024](1)
[18091]pmXmitPDU: DESC fd=1 len=32
000:       20     7005        0     8011        0 ffffffff  3000000        0 
__pmUnpinPDUBuf(0x2537310) -> pdubuf=0x2537310, pincnt=0
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)
pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 16, last read 16, still need 0
[18091]pmGetPDU: PROFILE fd=0 len=28 from=2
000:       1c     7002        2        0        0        0        0 
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: Received PDU_PROFILE
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)
pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 20, last read 20, still need 0
[18091]pmGetPDU: FETCH fd=0 len=32 from=2
000:       20     7003        2        0        0        0  1000000     8011 
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: Received PDU_FETCH
__pmPinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=2
__pmUnpinPDUBuf(0x253789c) -> pdubuf=0x2537880, pincnt=1
pmResult dump from 0x25365b0 timestamp: 0.000000 00:00:00.000 numpmid: 1
  70.0.0 (<noname>): numval: 1 valfmt: 0 vlist[]:
   value 0 0 0x0
__pmFindPDUBuf(48) -> 0x2537310
   pinned pdubuf[size](pincnt): 0x2537310...0x253733f[48](1) 0x2537880...0x2537c7f[1024](1)
[18091]pmXmitPDU: RESULT fd=1 len=44
000:       2c     7001        0        0        0  1000000     8011  1000000 
008:        0 ffffffff        0 
__pmUnpinPDUBuf(0x2537310) -> pdubuf=0x2537310, pincnt=0
__pmFreeResultValues(0x25365b0) numpmid=1
__pmUnpinPDUBuf(0x25372a0) -> fails
   pinned pdubuf[size](pincnt): 0x2537880...0x2537c7f[1024](1)
free(0x25372a0) vset pmid=70.0.0
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)
pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 32, last read 32, still need 0
[18091]pmGetPDU: RESULT fd=0 len=44 from=2
000:       2c     7001        2 f8fe1a95 9087fd96  1000000     8011  1000000 
008:        0 ffffffff  1000000 
pmgetconfig: PCP_TMPFILE_DIR=/var/tmp
[Tue Oct 27 14:13:53] pmdammv(18091) Debug: Received PDU_RESULT
vlist[0] pmid: 70.0.0 numval: 1 valfmt: insitu
need: 32 vsize: 20 nvsize: 32 vbsize: 0 offset: 44 hdr.len: 44 pduend: 0x25378ac vsplit: 0x25378ac (diff 0) pdubuf: 0x2537880 (diff 44)
__pmFindPDUBuf(32) -> 0x2537310
   pinned pdubuf[size](pincnt): 0x2537310...0x253732f[32](1) 0x2537880...0x2537c7f[1024](1)
new vlist[0] pmid: 70.0.0 numval: 1 valfmt: insitu value: 1
pmResult dump from 0x25372a0 timestamp: -1793392904.-1761769584 09:46:32.2533197 numpmid: 1
  70.0.0 (<noname>): numval: 1 valfmt: 0 vlist[]:
   value 1 1.4012985e-45 0x1
pmExtractValue:  1 [32] -> 1 [32]
__pmFindPDUBuf(16) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537310...0x253732f[32](1) 0x2537460...0x253746f[16](1) 0x2537880...0x2537c7f[1024](1)
__pmSendError: sending error PDU (code=0, toversion=2)
[18091]pmXmitPDU: ERROR fd=1 len=16
000:       10     7000        0        0 
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pmFreeResult(0x25372a0)
__pmFreeResultValues(0x25372a0) numpmid=1
__pmUnpinPDUBuf(0x2537310) -> pdubuf=0x2537310, pincnt=0
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)
pduread(0, ...): have 12, last read 12, still need 0
__pmFindPDUBuf(1024) -> 0x2537880
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1) 0x2537880...0x2537c7f[1024](1)
__pmUnpinPDUBuf(0x2537460) -> pdubuf=0x2537460, pincnt=0
pduread(0, ...): have 4, last read 4, still need 0
[18091]pmGetPDU: ERROR fd=0 len=16 from=2
000:       10     7000        2 99cfffff 
__pmDecodeError: got error PDU (code=-12391, fromversion=2)
__pmUnpinPDUBuf(0x2537880) -> pdubuf=0x2537880, pincnt=0
__pmFindPDUBuf(1024) -> 0x2537460
   pinned pdubuf[size](pincnt): 0x2537460...0x253785f[1024](1)

We have tried to leverage TravisCI to build & test Parfait against the latest public PCP version (which might not be in the OS distros).

However, due to Python dependency hell, this is not working. This issue aims to bring this support back. For the moment we are building Parfait against the latest PCP version available within the Ubuntu bionic distro (which was PCP v4.x at this point).

This issue stems from the discovery many moons ago that for older Java+Linux Kernal combinations some of the Thread CPU metric collections were INSANELY SLOW.

To work around this, an evilly hidden JVM option was discovered: -XX:+UseLinuxPosixThreadCPUClocks

Which when active, drastically improved performance.

From [email protected] on January 07, 2011 14:18:47

See this discussion with Cowan: http://groups.google.com/group/parfait-dev/browse_thread/thread/e2698c671a303c97

Original issue: http://code.google.com/p/parfait/issues/detail?id=7

Parfait currently does not support non-primitive number types (Integer etc) even though those should be straightforward to support in addition to primitive number types (int etc). Encountering such attributes leads to a cryptic error message like:

java.lang.RuntimeException: Metric foo.bar is not optional but has bad attribute [bar]

In practice, an MBean attribute created with

MBeanAttributeInfo("test", "java.lang.Integer", "", true, false, false);

works finely on JConsole and other such tools but fails with Parfait. While changing this to

MBeanAttributeInfo("test", int.class.getName(), "", true, false, false);

would solve the issue, it's really up to Parfait to support these kind of common types.

From [email protected] on January 31, 2011 11:13:52

From psmith:

One useful feature would be a mechanism to automatically expose all
metrics via a restful API.

If one wrapped a CXF or jersey or restlet interface around a registry
and allow the highernlevek discovery restful Apis then that would be
cool.

From me:

+100. This has been on my todo list for very long time.

We've already got parfait-cxf which has cxf dependencies -- it's kind of the opposite of what you're talking about (it's stuff for taking CXF services and exporting metrics) but there's a precedent with parfait-spring and parfait-jmx, both of which mix 'capture metrics from x' and 'expose metrics via x'.

Very very simple to do I think.

Original issue: http://code.google.com/p/parfait/issues/detail?id=8

On linux, parfait appears to leak TimerTask objects, which in turn leak mmap files.

We are using parfait in our JVM as an agent. It starts and runs normally, we can add instances to it as we need to.

We observe that when we add an instance, the JVM seems to hold onto old versions of the memory mapped file:

root@host:~# grep /var/lib/pcp/tmp/mmv/com /proc/$(cat /var/run/PID)/maps
7f2754067000-7f275406d000 rw-s 00000000 00:14 50972 /var/lib/pcp/tmp/mmv/com
7f2764002000-7f2764007000 rw-s 00000000 00:14 50292 /var/lib/pcp/tmp/mmv/com (deleted)
7f276402e000-7f2764032000 rw-s 00000000 00:14 49529 /var/lib/pcp/tmp/mmv/com (deleted)

We also see that this increases over time, as we add new instances.

We took heap dumps before and after noticing this behavior, and it appears that:

• TimerTask is an anonymous function, which QuiescentRegistryListeners create as necessary
• Each TimerTask holds a DynamicMonitoringViews
• Each DynamicMonitoringViews holds a DirectByteBuffer
• Each DirectByteBuffer corresponds to an entry in the maps file

When we changed QuiescentRegistryListener to:

• Track the previously created TimerTask
• When it makes a new TimerTask, cancel the old one
  Then, the JVM would GC the memory mapped files.

I'm attaching the patch we created. I offer this contribution subject to the apache 2.0 license that governs parfait itself.

Thanks for all the work you and your team do on parfait, we really like PCP, it does a great job for us.

leak-minimal.txt

From [email protected] on December 20, 2010 13:31:18

To help people not familiar with Parfait, it would be great to have a Ganglia export format for Parfait, this way people could transition to parfait without giving up their Ganglia infrastructure if they like.

The Hadoop Ganglia MetricContext can be used as a great starting point for getting the basic idea.

Original issue: http://code.google.com/p/parfait/issues/detail?id=4

The default configuration causes exception on startup with newer Java versions:

# java -version
openjdk version "1.8.0_161"
OpenJDK Runtime Environment (build 1.8.0_161-b14)
OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)
# java -javaagent:/parfait-agent-jar-with-dependencies.jar KeyboardReader
194 [main] INFO io.pcp.parfait.pcp.PcpMonitorBridge - PCP monitoring bridge started for writer [PcpMmvWriter[byteBufferFactory=FileByteBufferFactory[file=/var/lib/pcp/tmp/mmv/KeyboardReader]]]
Enter a line of input

# java -version
openjdk version "9.0.4"
OpenJDK Runtime Environment (build 9.0.4+11)
OpenJDK 64-Bit Server VM (build 9.0.4+11, mixed mode)
# java -javaagent:/parfait-agent-jar-with-dependencies.jar KeyboardReader
[main] ERROR io.pcp.parfait.ParfaitAgent  - Stopping Parfait agent [KeyboardReader]
java.lang.RuntimeException: Metric java.memorypool.eden[init] is not optional but has bad attribute [Usage]
	at io.pcp.parfait.AgentMonitoringView.createMonitorable(AgentMonitoringView.java:108)
	at io.pcp.parfait.AgentMonitoringView.register(AgentMonitoringView.java:92)
	at io.pcp.parfait.ParfaitAgent.startView(ParfaitAgent.java:119)
...

Tested on Fedora 27.

This is based on a conversation with Nick Tindall.

One often confusing element between DropWizard metrics and PCP is around Counters & rates. DropWizard has a Counter object, but it's not really the same as a Counter in PCP. In DW Metrics, Counters can go backwards, so really are 'free running' semantics, so they are not suitable as use as PCP Counters (which model rates).

We do have the MeteredAdapter which uses DW Metrics Meters which are rates, but what this gives us is just 1/5/15 minute sliding window averages and an overall average rate.

In PCP, these values are useful at times, but in all honesty the "mean" sub-metric is a bit useless, because if it's the average over a long time, overnight low activity will skew the value severly, and it's not giving anyone much of a picture.

However DW Metric Meters do expose a 'getCount()' method which looks to expose the underlying.... Counter.. :)

If we change the MeteredAdaptor to include a new 'raw' value that takes this value and exports it to PCP, then all of a sudden we have the ability to track rates properly in PCP.

something like:

diff --git a/parfait-dropwizard/src/main/java/com/custardsource/parfait/dropwizard/metricadapters/MeteredAdapter.java b/parfait-dropwizard/src/main/java/com/custardsource/parfait/dropwizard/metricadapters/MeteredAdapter.java
index 9c994e5..ad36af2 100644
--- a/parfait-dropwizard/src/main/java/com/custardsource/parfait/dropwizard/metricadapters/MeteredAdapter.java
+++ b/parfait-dropwizard/src/main/java/com/custardsource/parfait/dropwizard/metricadapters/MeteredAdapter.java
@@ -1,10 +1,5 @@
 package com.custardsource.parfait.dropwizard.metricadapters;

-import static com.codahale.metrics.MetricRegistry.name;
-
-import javax.measure.unit.Unit;
-import java.util.Set;
-
 import com.codahale.metrics.Metered;
 import com.custardsource.parfait.Monitorable;
 import com.custardsource.parfait.ValueSemantics;
@@ -12,6 +7,11 @@ import com.custardsource.parfait.dropwizard.MetricAdapter;
 import com.custardsource.parfait.dropwizard.NonSelfRegisteringSettableValue;
 import com.google.common.collect.Sets;

+import javax.measure.unit.Unit;
+import java.util.Set;
+
+import static com.codahale.metrics.MetricRegistry.name;
+
 public class MeteredAdapter implements MetricAdapter {

     private final Metered metered;
@@ -19,6 +19,7 @@ public class MeteredAdapter implements MetricAdapter {
     private final NonSelfRegisteringSettableValue<Double> oneMinuteRate;
     private final NonSelfRegisteringSettableValue<Double> meanRate;
     private final NonSelfRegisteringSettableValue<Double> fifteenMinuteRate;
+    private final NonSelfRegisteringSettableValue<Long> rawCounter;

     public MeteredAdapter(Metered metered, String name, String description) {
         this.metered = metered;
@@ -26,6 +27,7 @@ public class MeteredAdapter implements MetricAdapter {
         this.fiveMinuteRate = new NonSelfRegisteringSettableValue<>(name(name, "five_minute_rate"), description + " - Five minute rate", Unit.ONE, metered.getFiveMinuteRate(), ValueSemantics.FREE_RUNNING);
         this.oneMinuteRate = new NonSelfRegisteringSettableValue<>(name(name, "one_minute_rate"), description + " - One minute rate", Unit.ONE, metered.getOneMinuteRate(), ValueSemantics.FREE_RUNNING);
         this.meanRate = new NonSelfRegisteringSettableValue<>(name(name, "mean_rate"), description + " - Mean rate", Unit.ONE, metered.getMeanRate(), ValueSemantics.FREE_RUNNING);
+        this.rawCounter = new NonSelfRegisteringSettableValue<>(name, description + " - raw", Unit.ONE, metered.getCount(),ValueSemantics.MONOTONICALLY_INCREASING);
     }

     @Override

I have been trying to figure out how to use parfait and I thought I had it all figured out but I can't get my custom jmx mbeans to be registered. I am currently running parfait using the javaagent argument when calling my app. After lots of messing around I think the issue is that parfait is being loaded before my app initializes so the JMX mbeans do not exist at the time time parfait is initialized. What am I doing wrong?

I also tried to figure out how the JMXconnector works as I thought that might be a better way to go but I can't figure out how to configure the proxy.xml to point it at a specific java app.

Any help would be appreciated.

Hi Guys,

I have Parfait 1.1.1 built, PCP 5.3.1 installed on my CentOS 7 box. I placed parfait-agent-jar-with-dependencies.jar in /usr/lib64/parfait.jar. It works with the ACME examples, but I tried to use it with Tomcat by adding -javaagent:/usr/lib64/parfait.jar to JAVA_OPTS in /etc/tomcat/tomcat.conf and no metrics appeared under mmv after Tomcat was started. I tried to generate some debug logs from Tomcat, but even with the logging level set to FINEST I only get this:

INFO: Command line argument: -javaagent:/usr/lib64/parfait.jar

I also tried to plug it into Apache's Cassandra. It also did not work, but generated more logs:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-cassandra-3.11.10/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib64/parfait.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
INFO  [main] 2021-07-05 11:51:18,109 ParfaitAgent.java:146 - Starting Parfait agent [cassandra]
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
	at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
	at io.pcp.parfait.AgentMonitoringView.checkCompositeDataItem(AgentMonitoringView.java:186)
	at io.pcp.parfait.AgentMonitoringView.createMonitorable(AgentMonitoringView.java:104)
	at io.pcp.parfait.AgentMonitoringView.register(AgentMonitoringView.java:92)
	at io.pcp.parfait.ParfaitAgent.startView(ParfaitAgent.java:119)
	at io.pcp.parfait.ParfaitAgent.startLocal(ParfaitAgent.java:128)
	at io.pcp.parfait.ParfaitAgent.premain(ParfaitAgent.java:148)
	... 6 more
FATAL ERROR in native method: processing of -javaagent failed

Is there something else required for this to work? If anyone has suggestions please share. I can do some more tests and checks if needed.

Regards,
MS

From [email protected] on December 20, 2010 13:37:41

Cowan had a nice idea for an AggregateCounter that does in built totalizing of a set of ‘Instance Domains":

Counter bulkCounter = AggregatedCounters.aggregate(registry, "elasticsearch.index.bulk", "elasticsearch.index.buik.shard" + shardId, Unit.ONE, ...);

with something like:

CompositeCounter aggregate(MonitorableRegistry registry, String topLevel, String subLevel, Unit...) { Monitorable existingCounter = registry.getMonitorable(topLevel);
if (existingCounter == null) {
// Create the counter
} else if (! existingCounter instanceof Counter) {
// Uh-oh, there's an existing metric that's not actually a counter. What idiot made that?
throw new IdiotException();
}

  Counter newCounter = new MonitoredCounter(subLevel, registry, unit....);
  return CompositeCounter.of((Counter) existingCounter, newCounter);
}

? It's not THAT clean but I think it works fine in this case -- just specify the 'bulk' counter at 'sub' counter creation time, but if it already exists reuse it.

Original issue: http://code.google.com/p/parfait/issues/detail?id=5

JSR 385 has been Final at JCP.org for some weeks now, and the API has been stable 2.0 since early July.

Would it be possible for Parfait to upgrate? Happy to help with a PR as well.

Calling io.pcp.parfait.dxm.BasePcpWriter#reset clears metricData but there are several references held on to PcpMetricInfos, InstanceDomains, PcpStrings and PcpValueInfos (PcpValueInfos are cleared in metricData map but there is a reference held in the perMetricByteBuffers map).

Maps/arrays/stores are as follows:

io.pcp.parfait.dxm.BasePcpWriter#metricInfoStore
io.pcp.parfait.dxm.BasePcpWriter#instanceDomainStore
io.pcp.parfait.dxm.PcpMmvWriter#stringInfo
io.pcp.parfait.dxm.BasePcpWriter#perMetricByteBuffers

These data structures should also be cleared out when #reset is called. I'm more just making a note of this at the moment. I noticed this when working on MMV v2 support but haven't tackled this object leak as part of that issue. The patch set is already fairly large.

From [email protected] on December 16, 2010 13:07:46

Using log4j is a bit of a needless imposition, a switch to slf4j would be extremely nice-to-have

Original issue: http://code.google.com/p/parfait/issues/detail?id=2

I'm going to be adding a couple of extra metrics to the Parfait-DropWizard extension for Timers to expose the histogram percentile's for request latency measurements. Currently it only exposes the request rate counter, and 1/5/15 minute rates.

On the request latency it provides an average, median, min/max,stddev, but that's over the lifetime of the process which isn't that useful for very long running webapps with often long durations of 'less busy'.

The existing DropWizard metrics provides histogram decaying reservoir percentiles for 'recent' requests in the classic, 75th/95th/98th/99/99.9th Percentile which exposes much more useful 'recent' distribution information about requests.

These will appear as additional sub-metrics alongside the existing.

I'll be doing this in a new v3 branch, and merge back to master (Babylon is not yet migrated to the new v4 namespaces as yet).

This issue tracks changes for general dependency upgrades of the Parfait library.

The intent is to:

• Ensure all dependencies with vulnerabilities are fixed - see https://github.com/performancecopilot/parfait/security
• Generally all dependencies are the latest version available
• Upgrade Maven Plugin dependencies

I saw, Parfait 0.4 defines its own quantity types Information or InformationRate. While there's nothing wrong with that and exchanging data with other JSR-363 implementations or libraries in the same JVM may not happen much, could the small (4kB) Systems Quantity library be used instead, too?
https://github.com/unitsofmeasurement/uom-systems/tree/master/quantity

It is on jcenter http://jcenter.bintray.com/systems/uom/systems-quantity/ but if that makes it easier, we're happy to publish it on MavenCentral, too.

Log4j1.2 is now end of life. There are some key spots where Parfait uses Log4j's MDC context which will need to be migrated during the journey to Log4j2.

There are a few areas that we need to consider, and this might warrant separating these into a separate module (due to library dependencies). We could also decide whether to continue to support log4j1.2 at all.

Areas affected:

• io.pcp.parfait.timing.ThreadContext.MdcBridge
• General use of log4j1.2 logging (parfait-agent, and QuiescentRegistryListener)

From https://groups.io/g/pcp/message/17850:

If you start a Java application and use parfait --connect to get metrics from it, after the Java application is restarted, Parfait gets stuck in an error loop and eats up almost 20% of CPU on my laptop and provides no more metrics for the Java app.

Parfait should provide a helper utility to allow flexibly attaching Parfait agent to an already running VM. This is possible with the Attach API which is an integral part of JVM since Java 9 (this would work also on Java 8 as long as the tools.jar (which is not needed with Java 9 and later) is available).

This approach would allow gathering metrics with Parfait from an earlier started VM without the need for an additional JVM process once the helper has loaded the agent.

From [email protected] on December 20, 2010 13:29:57

The Log4jSink (now name changed, forget what it is now) output in tab delimited format. This was presumably thought to be useful, but real world uses I've struggled a lot mining the data.

Given python has very nice JSON support built in, and it generally being a very human readable format, I would think it best to at least have it as an option to be output in JSON format (perhaps leaving the default as is).

Original issue: http://code.google.com/p/parfait/issues/detail?id=3

From [email protected] on December 20, 2010 16:54:35

From a discussion with psmith:

public class SamplingEventSink implements StepMeasurementSink {
public SamplingEventSink(StepMeasurementSink delegate, float sampleFrequency) {
... pretty self-evident from here...

Original issue: http://code.google.com/p/parfait/issues/detail?id=6

From [email protected] on June 26, 2014 07:36:41

We currently use JSR 275 for units of measure, but that JSR has been rejected. Instead we should migrate to the JSR 363. On advice from the spec lead:

275 has been rejected 4 years ago. It was always feature-complete, e.g. GeoAPI has also since added it to an official OGC standard, but the JSR was non-final from a JCP point.
We currently see more momentum around 363 also from JUGs and the EC, so this time fingers crossed it'll be final, and at leas then certainly supersedes 275 or the Unit-API 0.6 (which we published under unitsofmeasurement.org so far)

Please add www.unitsofmeasurement.org. The domain host doesn't allow a wildcard, I also heard something similar from a host for a domain of myself, but this one was registered by fellow Spec Lead Jean-Marie Dautelle. unitsofmeasurement.github.io does not require a "www" in fact it likely won't work, but the .org domain currently does.

Cheers,
Werner

Original issue: http://code.google.com/p/parfait/issues/detail?id=12

Using https://github.com/myllynen/mbean2json we saw:

With the newly generated jvm.json the number of available metrics jumps from 18 (that are available with the default jvm.json) to 147. Metric names and descriptions are based on a simple generator so if there's anyone wanting to put this in, all metrics names and descriptions should be reviewed and adjusted as needed.

Issue #46 and #47 are relevant as well when updating the default jvm.json.

Thanks.

At least string metrics have wrong units:

$ pminfo -dmtT mmv.test.java.runtime_name

mmv.test.java.runtime_name PMID: 70.3154.734 [An internal Java process identifier]
    Data Type: string  InDom: PM_INDOM_NULL 0xffffffff
    Semantics: discrete  Units: count
Help:
An internal Java process identifier
$ 

There was also discussion whether using unsigned integers/longs should be considered at for counters and other metrics which can only be positive; with Java 8 and Guava this should be possible but it's debatable whether this kind of change would be worth the effort (unlike with the above string example where units should certainly be corrected).

It would be nice to allow DropWizard Metric HealthChecks (https://dropwizard.github.io/metrics/3.1.0/manual/healthchecks/) to be exposable to PCP via metrics.

The rationale here is that when a health check fails, it would be nice to have it correlated within the PCP metric space to align with whatever the failure was.

Perhaps ideally these Health Check failures (or recovery) would be better represented as PCP Events, but since MMV doesn't yet support placing events yet, for now it might be ok to have a Monitored Value exposed matching a "OK" or "NOT OK" style (or numerical equivalent) to align in the PCP timeline.

Parfait currently does not support TabularData type even though already OpenJDK JVM exposes at least SystemProperties as TabularData. Individual TabularData/SystemProperties items (which include sometimes very helpful information (at least in global projects) like locale) could be represented as string values in PCP if supported by Parfait.

From [email protected] on March 10, 2011 16:48:57

ThreadContext.clear()

doesn't actually do the inner MDC.clear()
that's because MDC doesn't HAVE a clear..
but it's not clearing...
we really need to iterate over the MDC keys and remove.
because otherwise the MDC becomes tainted.

Original issue: http://code.google.com/p/parfait/issues/detail?id=11