tyagihas / java_nats Goto Github PK

I've created a gist with the test code: https://gist.github.com/adamhonen/04070d59159e09174658

Here's the setup:

I have 4 t2.micro machines named dev-stream-01 to dev-stream-04
dev-stream-01 is used to run the loader program.using the following command:

java -cp stream.jar NatsLoader 100000 1000 > out.log 2>&1 &

(stream.jar is of course just an arbitrary name I've picked)

dev-stream-02 to dev-stream-04 are configured to form a cluster.
They all point to each other in configuration.

So, for example, this is dev-stream-02's configuration file:

port: 4242
http_port: 8222

cluster {
host: 'dev-stream-02' # host/net interface
port: 4244 # port for inbound route connections

routes = [
nats-route://dev-stream-03:4244
nats-route://dev-stream-04:4244
]
}

And the other two are similar, with the expected changes.

I launch the java process and then check which server is being used to publish messages and which is sending them back to us.
I do this using iftop from dev-stream-01, although there might be better ways. I just happened to be using it anyway in order to see how much data was being moved around for various similar products.

Anyway, I then kill the gnatsd process on the producer server and immediately bring it back up.
In some rare cases, re-connection happens.
More often, I just see that no more messages are being sent.

The java process remains alive, so I run kill -3 on it in order to see the stacktrace.
The results vary a lot.
Before extracting the test code from my entire POC application (which does a whole lot of stuff that are irrelevant to nats) I saw that either the producer thread, or one of the MsgProcessor threads, was holding the sendBuffer lock and stuck on read while other threads were waiting to lock it.
Now with the extracted code, I mainly see StackOverFlow errors.
It still gets stuck sometimes, but the thread dump still shows the signs of a StackOverFlow error, which repeats the following:

at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)
at org.nats.Connection.access$300(Connection.java:20)
at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)
at org.nats.Connection.access$300(Connection.java:20)
at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)
at org.nats.Connection.access$300(Connection.java:20)
at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)
at org.nats.Connection.access$300(Connection.java:20)
at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)
at org.nats.Connection.access$300(Connection.java:20)
at org.nats.Connection$MsgProcessor.run(Connection.java:703)
at org.nats.Connection.reconnect(Connection.java:638)

I assume the difference in how reconnecting failed between my larger application and this smaller one, is that the larger application also had other threads doing work (e.g. tracking various metrics), so the nats client threads and the producer thread were getting scheduled differently.

Hello,

I am trying to use this library (through it's scala api, but I think this problem is caused by the java api), and I am running into some problems when there are some exceptions, for example when the server goes down.

java_nats indeed tries to reconnect, but if the reconnection fails, it just logs the problem and never throws an exception, and never tries again either. This is a problem because the code calling this library cannot act on the error as it never receives an indication that there is an error, like an exception or an error callback.

Is this intended?

Thanks,
Simao

I'm struggling with the same issue as the author of this mail:
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/GPvXpyNCW7A/-DBpY_EFi24J
Basically we can't consume large messages fast enough without having a BufferUnderflowException:

Exception in thread "Thread-0" java.nio.BufferUnderflowException
    at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:265)
    at org.nats.Connection$MsgProcessor.processMessage(Connection.java:751)
    at org.nats.Connection$MsgProcessor.run(Connection.java:698)

My message handler looks like this (sorry for the Scala):

val suid = conn.subscribe("events",
  new MsgHandler {
    override def execute(msg: String) {
      println(s"Received update : $msg")
    }
  }
)

Any ideas?

my compony's nats server need authorized, witch means i need add username and pass to NATS when connected.
just like this:
nats://nats:[email protected]:4222/

can this java-nats-client support this?

Is there a way to publish/receive binary data (byte[]) instead of strings? Any intention of adding this functionality?

Just tried the embedded server.
It always reports "Address already in use", but I cant find anything running on my 4222 port.

This Server has no streaming implemented: https://nats.io/documentation/streaming/nats-streaming-intro/