Code Monkey home page Code Monkey logo

java-util's Introduction

Build Status

com.metamx.common

Utility code for java and jvm-based languages.

com.metamx.http - HTTP client

This is an async HTTP client library that provides two basic functions:

  1. a channel caching pool for Netty channels that works well for inter-service communication
  2. the absolute minimum abstractions on top of Netty's HTTP support to make it a little easier to issue requests

com.metamx.emitter - Event Emitter

Emitter creation

The easiest way to create an emmiter instance is via com.metamx.emitter.core.Emitters.create method. This method will create one of the predefined emitters or your own implementation based on the properties provided. com.metamx.emitter.type property value defines what Emitter should be created.

Emitter types

Logging emitter

com.metamx.emitter.type = logging will create LoggingEmitter. For more details on LoggingEmitter configuration please refer com.metamx.emitter.core.LoggingEmitterConfig

Http emitter

com.metamx.emitter.type = http will create HttpPostEmitter. The only required parameter is com.metamx.emitter.recipientBaseUrl that is a url where all the events will be sent too. For more details on HttpPostEmitter configuration please refer com.metamx.emitter.core.HttpEmitterConfig

Parametrized URI http emitter

com.metamx.emitter.type = parametrized will create ParametrizedUriEmitter. ParametrizedUriEmitter is a http emitter that can be used when events should be posted to different url based on event data. The URI pattern is defined via com.metamx.emitter.recipientBaseUrlPattern property. For instance: com.metamx.emitter.recipientBaseUrlPattern=http://example.com/{feed} will make it send events to different endpoints according to event.getFeed value. com.metamx.emitter.recipientBaseUrlPattern=http://example.com/{key1}/{key2} requires that key1 and key2 are defined in event map.

Custom emitter

To create your own emitter you need to implement EmitterFactory and add it as registered type to ObjectMapper that is used to call Emitters.create. com.metamx.emitter.type property should be set to the type name of registered subclass that is usually set with an annotation @JsonTypeName("my_custom_emitter") You can refer to com.metamx.emitter.core.CustomEmitterFactoryTest for an example of custom emitter creation. All properties with com.metamx.emitter.* prefix will be translated into configuration json used to create Emitter. Consider the following example:

com.metamx.emitter.type = my_custom_emitter
com.metamx.emitter.key1 = val1
com.metamx.emitter.outer.inner1 = inner_val1
com.metamx.emitter.outer.inner2 = inner_val2

will be translated into:

{
  "type": "my_custom_emitter",
  "key1": "val1",
  "outer":
    {
      "inner1": "inner_val1",
      "inner2": "inner_val2"
    }
}

com.metamx.metrics - Server Metrics

A library for emitting server-metrics

java-util's People

Contributors

akashdw avatar cheddar avatar drcrallen avatar esevastyanov avatar fjy avatar gianm avatar himanshug avatar isimonenko avatar jdanbrown avatar jeffwong avatar jon-wei avatar leventov avatar logarithm avatar navis avatar nishantmonu51 avatar tjgiuli avatar xanec avatar xdralex avatar xvrl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

java-util's Issues

Uncontrolled build up of buffers in HttpPostEmitter

Currently, in HttpPostEmitter, when a Batch object is sealed, it will call onSealExclusive in HttpPostEmitter:

void onSealExclusive(Batch batch)
{
addBatchToEmitQueue(batch);
wakeUpEmittingThread();
if (!isTerminated()) {
int nextBatchNumber = EmittedBatchCounter.nextBatchNumber(batch.batchNumber);
if (!concurrentBatch.compareAndSet(batch, new Batch(this, acquireBuffer(), nextBatchNumber))) {
// If compareAndSet failed, the service is closed concurrently.
Preconditions.checkState(isTerminated());
}
}
}

Notice that this will automatically create a new Batch object as the concurrentBatch. What this may result, and what we have observed, is that there may be an uncontrolled build up of these objects especially when the EmittingThread does not emit fast enough and result in OOME.

Without altering the existing behavior (e.g., additional block elsewhere in the code), I suggest that we can put in additional checks on queue length in emitAndReturnBatch:

while (true) {
Batch batch = concurrentBatch.get();
if (batch == null) {
throw new RejectedExecutionException("Service is closed.");
}
if (batch.tryAddEvent(eventBytes)) {
return batch;
}
// Spin loop, until the thread calling onSealExclusive() updates the concurrentBatch. This update becomes visible
// eventually, because concurrentBatch.get() is a volatile read.
}
}

While this is not strictly and directly linked to the queue but it should allow some degree of control in the queue length (i.e., not adding new events to the Batch when there are already plenty to clear) and at the same, the blocking is expected in this method call.

TrustManagerFactory relies on default

I am reaching out to you as we conducted an empirical study to understand the nature of cryptographic misuses in enterprise-driven projects on GitHub. During our study, we randomly inspected a few of the misuses. One of the misuses for which we could confirm the finding of the analysis, CogniCryptSAST is within this project.

  • HttpClientInit: The implementation in line 143 calls the method TrustManagerFactory.getDefaultAlgorithm() that relies on defaults and can be altered at runtime JCA. Thus, analyses like CogniCryptSAST consider these usages as insecure as the used TrustManager may be insecure.

We hope that this report helps you and would be glad to get your thoughts on this issue.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.