signalfx / signalfx-java Goto Github PK

View Code? Open in Web Editor NEW

38.0 68.0 42.0 1.53 MB

Java client libraries for SignalFx

Home Page: https://signalfx.com/

License: Apache License 2.0

Shell 0.73% Java 97.57% Python 1.69%

signalfx java codahale-metrics yammer-metrics metrics monitoring

signalfx-java's Introduction

ℹ️ SignalFx was acquired by Splunk in October 2019. See Splunk SignalFx for more information.

SignalFx client libraries

⚠️ This repository and its published libraries are deprecated

This repository contains legacy libraries for reporting metrics to Splunk Observability Cloud (formerly SignalFx). The only commits that will be made to this repo are organizational or security related patches. No additional features will be added, and the repository will be archived and the final versions published on or prior to February 1, 2025.

⚠️This repo will be archived February 1st 2025.

Splunk has adopted OpenTelemetry. Use the OpenTelemetry Java SDK or the Splunk Distribution of OpenTelemetry Java Instrumentation agent to send telemetry data to Splunk. Do not integrate signalfx-java into new services.

Do not use this library in your project

You should not use this library in new services. Use OpenTelemetry to send data to Splunk instead.

To send data to Splunk, you need a Splunk Observability Cloud account and organization API token. For more information on Observability Cloud and to create an account, go to Splunk Observability.

Legacy documentation is still here for posterity: legacy-usage.md.

Executing SignalFlow computations

Learn more about using SignalFlow here.

License

signalfx-java's People

Contributors

Stargazers

Watchers

Forkers

codeaudit hgschmie erikdw lganzzzo alexanderkm choang anandsoft gnufied m1strall jefersonm brandoncurrie tlisonbee adranjan jhaber heltonmarx mark43 ericlemieux sunset3000 timvlaer tschiman wxy325 twilliamson sachpat mreilly82 isabella232 carldea mbc1990 mbalchcrystal ybjosephson asuresh4 rajeshalane laurit jtrzebiatowski atlassian-forks lenin-jaganathan excaleo agoallikmaa orange-brooch codelion nishantagarwal1990 maulikp-splunk worldsoldier

signalfx-java's Issues

Twist Lock Scan vulnerability with <jackson.databind.version>2.10.3</jackson.databind.version>

@dloucasfx Please bump the version from 2.10.3 to 2.11.0 or the latest Jackson databind verison. It is blocking us.
exclusion is not working.
<jackson.databind.version>2.11.0</jackson.data-bind.version>

as it contains this vulnerability - https://nvd.nist.gov/vuln/detail/CVE-2020-25649
From: Finicity

Flow client can't be reused after connection lost.

I use flow client (lib version 1.0.3) to subscribe on metrics using code pretty close to one described in readme:

while (true) {
    val computation = flow.execute(STATUS_QUERY)
    try {
        for (message in computation) {
            handleRecievedMessage(message)
        }
    } catch (ex: SignalFlowException) {
        logger.error(ex) { "Flow subscription stopped, will resubscribe after timeout $DELAY_BETWEEN_FLOW_SUBSCRIPTIONS" }
        delay(DELAY_BETWEEN_FLOW_SUBSCRIPTIONS.toMillis())
    }
}

Here flow is a client that created once.

Turned out that if exception occurs during polling, (usually it looks like c.s.s.WebSocketTransport$TransportConnection - Lost WebSocket connection with stream.signalfx.com/[ipadress-here]:443 (1006).) I can't repeat query on first line of cycle.

Instead of correct result I got NPE:

Caused by: java.lang.NullPointerException
at com.signalfx.signalflow.WebSocketTransport$TransportConnection.sendMessage(WebSocketTransport.java:535)

I'm not sure if it's a bug or not. If not, please improve readme by specifying that on lost connection, client should be recreated or restarted somehow.

Handle expired-tsid stream message

https://developers.signalfx.com/docs/stream-messages-specification#expired-tsid-messages

See signalfx/signalfx-python@5782c9d for equivalent Python implementation.

Bump to slf4j major version with patch version of lib itself?

Hi, I see that 1.0.25 was bumped to slf4j 2.x version. While api is backward compatible from code perspective, it is still very questionable decision.

I'd like to propose you to consider roll this back or update your versioning at least in minor part.

Twistlock vulnerability for io.netty_netty-codec

Please bump up the latest version for io.netty_netty-codec it is flagged as high Twistlock vulnerability.

Thanks, Sandeep From Finicity

NPE with null defaultSourceName

I accidentally passed a null defaultSourceName which resulted in the following error at runtime:

java.lang.NullPointerException: use Optional.orNull() instead of Optional.or(null)
at com.signalfx.shaded.google.common.base.Preconditions.checkNotNull (Preconditions.java:787)
at com.signalfx.shaded.google.common.base.Absent.or (Absent.java:50)
at com.signalfx.codahale.reporter.AggregateMetricSenderSessionWrapper.addMetric (AggregateMetricSenderSessionWrapper.java:187)
at com.signalfx.codahale.reporter.AggregateMetricSenderSessionWrapper.addMetric (AggregateMetricSenderSessionWrapper.java:151)
at com.signalfx.codahale.reporter.SignalFxReporter.report (SignalFxReporter.java:84)
at com.codahale.metrics.ScheduledReporter.report (ScheduledReporter.java:237)
at com.codahale.metrics.ScheduledReporter.lambda$start$0 (ScheduledReporter.java:177)

Probably should validate defaultSourceName is not null in the constructor (or fall back to the SourceNameHelper)

Actualize scm section in pom files and url tag

Right now pom.xml of parent contains invalid scm parameters:

<scm>
    <connection>scm:git:[email protected]:signalfx/signalfx-clients.git</connection>
    <developerConnection>scm:git:[email protected]:signalfx/signalfx-clients.git
    </developerConnection>
    <url>[email protected]:signalfx/signalfx-clients.git</url>
  </scm>

url should point to https://github.com/signalfx/signalfx-java
connection and developer connection should point to scm:git:[email protected]:signalfx/signalfx-java.git

Also it'd be great to configure this section explicitly in child pom files so that tools like https://docs.renovatebot.com/ can extract and generate proper links to project sites. Right now for example https://github.com/signalfx/signalfx-java/blob/master/signalfx-codahale/pom.xml contains <url>http://www.signalfx.com</url> (maybe point to github?) and also links to [email protected]:signalfx/signalfx-clients.git instead of signalfx-java repo.

Just in case you can take a look at https://www.mojohaus.org/flatten-maven-plugin/ if you don't like idea to managing all these manually

Change primary branch name from `master` to `main`

This is newer, less troubling naming convention and we should migrate this repo.

Unshaded Dependency Classes in Jar

Starting with version 0.0.40, unshaded dependency classes are showing up in the JAR. In particular, we hit problems due to SLF4J, but here's all the paths I see not under com/signalfx:

org/slf4j
google/protobuf/
com/github/os72/protobuf351

Yammer Metrics Setup Example Compilation Errors & Typo in README.md

In the Set up Yammer metrics section, the metric registry is set up as:

final MetricRegistry metricsRegistry = new MetricRegistry();

It should instead be the following (note MetricRegistry was changed to MetricsRegistry):

final MetricsRegistry metricsRegistry = new MetricsRegistry();

Also there's a typo in the signalfxReporter where it should be metricsRegistry instead of metricsRegistery.

Java 11 warning for protobuf illegal reflective access

We've doing a migration of our application to Java 11 and Spring Boot 2.1.0 and we get these warning messages in our logs:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.github.os72.protobuf351.UnsafeUtil (file:/Users/user/.m2/repository/com/signalfx/public/signalfx-java/0.0.48/signalfx-java-0.0.48.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.github.os72.protobuf351.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Not sure if this repository is the right one to reise the issue but I considered that we are using signalfx library and not protobuf so here should be the best place to report the issue.

Add Builder to AbstractHttpDataPointProtobufReceiverConnection

The AbstractHttpDataPointProtobufReceiverConnection class and its two implementations are getting a little constructor heavy and could benefit from the builder pattern.

The inheritance will make this a bit tricky. At worst, we should add builders to the child classes and leave AbstractHttpDataPointProtobufReceiverConnection alone. Tho maybe all 3 can share a non-inner builder, I haven't tried to tell yet.

This should really only be done after #268 is merged.

Codahale Examples For Adding Default Dimensions Don't Work With Specified Versions in README

Adding default dimensions using the SignalFxReporter does not work with the version of the codahale library referenced in the README. The addDimension and addDimensions methods did not exist on the Builder in version 0.0.23

Timestamp sent using SignalFxProtocolBuffers is inaccurate on the chart.

I ran the example from the Sending metrics without using Codahale section in the README, and the metric was recorded with a few seconds delay on the chart. Is this expected? Shouldn't the metric be plotted at the supplied timestamp exactly? Am I missing something?

Remove Duplicate Classes

We're beginning our integration with signalfx. One rule we have in place for our builds is to avoid duplicate classes in our classpath. This is enforced with the maven enforcer plugin. Here is an example of a class being duplicated:

From the signalfx jar:

→ jar -tvf signalfx-codahale-0.0.45.jar | grep com.codahale.metrics.MetricRegistry.class
 14822 Mon Mar 26 14:41:08 MST 2018 com/codahale/metrics/MetricRegistry.class

From the dropwizard jar:

→ jar -tvf metrics-core-3.1.5.jar | grep com.codahale.metrics.MetricRegistry.class
 12076 Fri Jun 02 19:59:20 MST 2017 com/codahale/metrics/MetricRegistry.class

Is it possible to remove these duplicate classes and rely on maven dependencies instead? Either that or namespace them under a different package to avoid this conflict?

Current code/integration is depricated?

Good day! I've got the letter from SignalFX support:

As SFx Java agent is deprecated please find the below link to our Splunk Otel Java instrumentation.
https://github.com/signalfx/splunk-otel-java

That is the documentation you are looking for.

I suppose we can add disclaimer to avoid misunderstanding from SFX community. I reached this signalfx-java project yesterday via google and waste the time to build a half-workable code. And after I fount Opentelemetry approach. WDYT?

SignalFx reporter is not converting durations according to reporter/metric config

I've recently tried to use signalfx-codahale libs to send metrics directly to SignalFx ingest endpoint.
Previously we were using Graphite Reporter from io.dropwizard.metrics:metrics-graphite + metrics proxy or Sl4j log reporters.
What I observed is that metrics that being sent through SignalFx reporter showed in dashboard Nanoseconds (that is how codahale records them) and not duration unit that I've specified (milliseconds).
Adjusting duration units in reporter/reporter factory config does not help.

I've looked up source code and discovered that all reporters (CSV, SL4J, GRAPHITE) inherit Scheduled Reporter and doing following conversion before sending a metric:
graphite.send(prefix(name, "mean"), format(convertDuration(snapshot.getMean())), timestamp);
or
loggerProxy.log(marker, convertDuration(snapshot.getMin()), convertDuration(snapshot.getMax()), convertDuration(snapshot.getMean()),
So duration units get converted before being send. Btw convertDuration is a method of base class of ScheduledReporter.

SignalFx reporter ignores that and sends raw values directly.

If this is expected behavior it's very confusing one.

Signalfx-codahale-1.0.19.jar jetty vulnerability

https://nvd.nist.gov/vuln/detail/CVE-2022-2048
We found a vulnerability with the severity HIGH in signalfx-codahale-1.0.19.jar.

signalfx-yammer-0.0.21 not found

The README suggests yammer 2.0.x is supported through the signalfx-yammer jar file, but that file is not found at:
http://mvnrepository.com/artifact/com.signalfx.public

Nor is it built by

git clone [email protected]:signalfx/signalfx-java
cd signalfx-java
git checkout 0.0.21
mvn clean package

Provide unshaded version or relocate additional dependencies

When looking at the latest signalfx-java 1.0.14 artifact, I show these dependencies as shaded without relocation (causing duplicate classes on the classpath). If an unshaded version is made available, teams could ensure consistent versions across an application. Alternatively, all of these dependencies should be relocated to minimize conflicts on the classpath.

com/google/errorprone/**/*.class
com/google/j2objc/**/*.class
google/protobuf/**/*.proto
javax/annotation/**/*.class
mozilla/public-suffix-list.txt (I believe this comes with Guava?)
org/checkerframework/**/*.class

We encountered this because of running security checks which found vulnerabilities in shaded Jackson and other libraries in an older version. If this artifact was available unshaded, we'd have been able to manage the versions of dependencies across all components.

Allow timezone param for SignalFlowClient

The SignaFlow client for java does not have a way to specify the timezone while the signalfx-python lib does. We would like to have similar capabilities.

Possible connection issue in AbstractHttpDataPointProtobufReceiverConnection when entity.getContent().close() is called

Upon moving to SpringBoot 2.4.2, I’ve encountered what seems to be a connection issue with signalfx-java. SpringBoot 2.4.2 uses signalfx-java v1.0.5, but this may have been introduced with v1.0.4.

This can reproduced by modifying the test testHttpConnection as follows:

    public void testHttpConnection() throws Exception {
        Server server = new Server(0);
        server.setHandler(new AddPointsHandler());
        server.start();
        URI uri = server.getURI();
        DataPointReceiver dpr = new HttpDataPointProtobufReceiverFactory(
                new SignalFxEndpoint(uri.getScheme(), uri.getHost(), uri.getPort()))
                .createDataPointReceiver();
        dpr.addDataPoints(AUTH_TOKEN, Collections.singletonList(
                SignalFxProtocolBuffers.DataPoint.newBuilder().setSource("source").build()));
        
        Thread.sleep(30000);

        dpr.addDataPoints(AUTH_TOKEN, Collections.singletonList(
                SignalFxProtocolBuffers.DataPoint.newBuilder().setSource("source").build()));
        server.stop();
    }

Notice that it calls twice and sleeps 30 sec in between calls. Running the test results in in the following exception:

com.signalfx.metrics.SignalFxMetricsException: Exception posting to addDataPoints
	at com.signalfx.metrics.connection.AbstractHttpDataPointProtobufReceiverConnection.addDataPoints(AbstractHttpDataPointProtobufReceiverConnection.java:68)
	at com.signalfx.metrics.connection.HttpDataPointProtobufReceiverConnectionTest.testHttpConnection(HttpDataPointProtobufReceiverConnectionTest.java:50)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)
Caused by: org.apache.http.NoHttpResponseException: 192.168.0.12:51987 failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
	at com.signalfx.connection.AbstractHttpReceiverConnection.postToEndpoint(AbstractHttpReceiverConnection.java:72)
	at com.signalfx.metrics.connection.AbstractHttpDataPointProtobufReceiverConnection.addDataPoints(AbstractHttpDataPointProtobufReceiverConnection.java:48)
	... 27 more

Debugging the issue took me into AbstractHttpDataPointProtobufReceiverConnection.java here. This change was introduced with signalfx-java v1.0.4 as part of this PR. When I comment out entity.getContent().close();, I do not see the exception.

NPE on non-Yammer SignalFlow example on HEAD + JDK11

Getting this from HEAD:

Exception in thread "main" com.signalfx.signalflow.SignalFlowException: failed to send message for channel channel-KW3PKDAD
    at com.signalfx.signalflow.WebSocketTransport$TransportConnection.sendMessage(WebSocketTransport.java:525)
    at com.signalfx.signalflow.WebSocketTransport.execute(WebSocketTransport.java:122)
    at com.signalfx.signalflow.Computation.execute(Computation.java:185)
    at com.signalfx.signalflow.Computation.<init>(Computation.java:67)
    at com.signalfx.signalflow.SignalFlowClient.execute(SignalFlowClient.java:53)
    at com.signalfx.jvmtoy.App.main(App.java:19)
Caused by: java.lang.NullPointerException
    at com.signalfx.signalflow.WebSocketTransport$TransportConnection.sendMessage(WebSocketTransport.java:522)
    ... 5 more

Seems to be something related to @mpetazzoni's upgrade of Jetty in here.

$ java -version
java version "11.0.2" 2019-01-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)

java.lang.NoClassDefFoundError: com/codahale/metrics/ThreadLocalRandomProxy updating resetting timer

Attempting to use a resetting timer from https://github.com/signalfx/signalfx-java is yielding a ClassNotFoundException:

java.lang.ClassNotFoundException: com.codahale.metrics.ThreadLocalRandomProxy
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[na:na]
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[na:na]
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[na:na]
    at com.codahale.metrics.ResettingExponentiallyDecayingReservoir.update(ResettingExponentiallyDecayingReservoir.java:104) ~[signalfx-codahale-1.0.5.jar:4.1.12.1]
    at com.codahale.metrics.ResettingExponentiallyDecayingReservoir.update(ResettingExponentiallyDecayingReservoir.java:89) ~[signalfx-codahale-1.0.5.jar:4.1.12.1]
    at com.codahale.metrics.Histogram.update(Histogram.java:41) ~[metrics-core-4.1.12.1.jar:4.1.12.1]
    at com.codahale.metrics.Timer.update(Timer.java:199) ~[metrics-core-4.1.12.1.jar:4.1.12.1]
    at com.codahale.metrics.Timer.update(Timer.java:94) ~[metrics-core-4.1.12.1.jar:4.1.12.1]

Pom contains:

<dependency>
      <groupId>com.signalfx.public</groupId>
      <artifactId>signalfx-codahale</artifactId>
      <version>1.0.5</version>
    </dependency>

I'm unable to find ThreadLocalRandomProxy in https://github.com/dropwizard/dropwizard or anywhere else, any hints?

SignalFxReporter doesn't maintain the state of metrics when we invoke report() method

I am trying to log my application metrics to SignalFx using SignalFxReporter class. I'm initializing the metric registry and SignalFxReporter at the time of app start and set SignalFxReporter to report every 1 minute.

final String signalfxAuthToken = configuration.getString("signalfx.authToken");
final SignalFxReporter signalfxReporter = new SignalFxReporter.Builder(
      METRIC_REGISTRY,
      new StaticAuthToken(signalfxAuthToken),
 ).addDimension("env", "prod").build();
metricMetadata = signalfxReporter.getMetricMetadata();

signalfxReporter.start(1, TimeUnit.MINUTES);

I see that its reporting the same metrics repeatedly. I just gonethrough the signalfxReporter class impl and I don't see that its maintaining the state of the metrics when it reports. It just reports all the available metrics in MetricRegistry all the time.

Is it expected behavior? I am actually looking for a non blocking way of SFxReporting from java application where it reports only new or changed metrics.

Twist Lock Scan vulnerability with <jackson.databind.version>2.9.10.3</jackson.databind.version>

Please bump the version to 2.9.10.4 or the latest Jackson databind verison. It is blocking us.
exclusion is not working.
<jackson.databind.version>2.9.10.3</jackson.databind.version>

From: Finicity

Jackson dependency has security vulnerability, twistlock will not build without an update.

In my release process we use Twistlock to scan for libraries with known vulnerabilities. Jackson 2.9.10 has vulnerabilities that were resolved in 2.9.10.1 and 2.9.10.2 see: link. Twistlock is forcing an upgrade to 2.9.10.2 to resolve these issues. Please update the library with the updated version of jackson so we can pass our security vulnerably scans.

What we are experiencing is that excluding the jackson dependencies from the project is not sufficient to change the version. We think this is happening because of the shading used in the project for fasterxml.

Currently using version 0.2.1.

README.md example for adding dimensions with yammer 2.0 incorrect

Adding Dimensions and SignalFx metadata to Codahale metrics -> Yammer Metrics 2.0.x
The example is copy-pasted from above, and does not reflect the Queue metric as shown in the 3.0 code.

Support for MultiToken clients

Are there any plans of supporting multi-token client initialisation similar to https://github.com/signalfx/golib/blob/master/sfxclient/multitokensink.go?