wurstmeister / storm-docker Goto Github PK

I have started getting the following error when running StormDocker. It was working fine, now i've started getting the following error in StormUI. It works intermittently if i refresh the page a couple of times.

I'm seeing similar errors in the supervisor log file.

I've tried re-pulling all of the images down and restarting the docker service but this issue keeps coming and going whatever I do.

I'm running on Ubuntu 12.04 64bit. Any ideas?

org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.ui.core$cluster_summary.invoke(core.clj:455)
at backtype.storm.ui.core$fn__8223.invoke(core.clj:789)
at compojure.core$make_route$fn__3365.invoke(core.clj:93)
at compojure.core$if_route$fn__3353.invoke(core.clj:39)
at compojure.core$if_method$fn__3346.invoke(core.clj:24)
at compojure.core$routing$fn__3371.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__3375.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__7540.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8268.invoke(core.clj:858)
at ring.middleware.keyword_params$wrap_keyword_params$fn__4029.invoke(keyword_params.clj:27)
at ring.middleware.nested_params$wrap_nested_params$fn__4068.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__4001.invoke(params.clj:55)
at ring.middleware.multipart_params$wrap_multipart_params$fn__4096.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__4277.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__4266.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__4197.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7179.invoke(jetty.clj:16)
at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
at java.net.Socket.connect(Socket.java:546)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)

I tried to use fig start to start the cluster. But it looks like some nodes are missing or failed to start. But I still can go to storm ui and I see following exception:

In this file: https://github.com/wurstmeister/storm-docker/blob/master/storm/start-supervisor.sh

you use these two ENVs: $ZK_PORT_2181_TCP_ADDR and $NIMBUS_PORT_6627_TCP_ADDR

Where are they coming from? I never defined them and this just works.

Hi

I did 'fig up'
Everything seems to be working from the console

I'm using boot2docker.
boot2docker up = 192.168.59.103

Opening 192.168.59.103:8080 --> 'Web page is not available'.
I see that storm UI exposes 8080.

Am I missing something?

Hi!
Can you show an example how can I deploy a topology? I started all image (up and running), but how can I deploy a topology (without install the storm to local machine) ?

Thanks
b0c1

I got this error message while deploying my topology? I can't seems to figure it out, why ?

development/storm/apache-storm-0.9.3/bin -Dstorm.jar=storm-starter-0.0.1-SNAPSHOT.jar storm.starter.ExclamationTopology ExTopo
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.TApplicationException: Invalid method name: 'getClusterInfo'
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:211)
        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:117)
        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:70)
        at storm.starter.ExclamationTopology.main(ExclamationTopology.java:59)
Caused by: org.apache.thrift7.TApplicationException: Invalid method name: 'getClusterInfo'
        at org.apache.thrift7.TApplicationException.read(TApplicationException.java:108)
        at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:71)
        at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:468)
        at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:456)
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:202)
        ... 3 more

This is awesome. I got everything working so quickly!

However, I was curious how I could enable multilang support with this? Specifically using node.js. Just need a nudge in the right direction.

Thanks!

-Matt

Hi, the link that points to fig installation page refers that "Fig has been replaced by Docker Compose, and is now deprecated. "

Is it necessary to change any configuration to run it with compose or it's still the same ?

Thanks

Unable to start STORM and NIMBUS with following error... any thoughts?

ui_1 | Setting up Kafka clients
zookeeper_1 | 2018-01-14 00:46:00,032 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /172.17.0.6:49746
zookeeper_1 | 2018-01-14 00:46:00,037 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@882] - Connection request from old client /172.17.0.6:49746; will be dropped if server is in r-o mode
zookeeper_1 | 2018-01-14 00:46:00,037 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@928] - Client attempting to establish new session at /172.17.0.6:49746
zookeeper_1 | 2018-01-14 00:46:00,050 [myid:] - INFO [SyncThread:0:ZooKeeperServer@673] - Established session 0x160f22047fb0001 with negotiated timeout 30000 for client /172.17.0.6:49746
ui_1 | Offset data: {"speedd-traffic-actions":{"0":[545]},"speedd-traffic-out-events":{"0":[21421]}}
ui_1 | Actions offset: 545
ui_1 | Events offset: 21421
ui_1 | Setting up Consumer on-message event

nimbus_1 | 2018-01-14 00:47:09,704 INFO exited: storm-nimbus (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:47:10,709 INFO spawned: 'storm-nimbus' with pid 148
supervisor_1 | 2018-01-14 00:47:10,741 INFO exited: storm-supervisor (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:47:11,712 INFO success: storm-nimbus entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
supervisor_1 | 2018-01-14 00:47:11,743 INFO spawned: 'storm-supervisor' with pid 142
supervisor_1 | 2018-01-14 00:47:12,746 INFO success: storm-supervisor entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
nimbus_1 | 2018-01-14 00:49:00,851 INFO exited: storm-nimbus (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:49:01,856 INFO spawned: 'storm-nimbus' with pid 218
supervisor_1 | 2018-01-14 00:49:01,946 INFO exited: storm-supervisor (exit status 13; not expected)

Hello!
The documentation for fig http://orchardup.github.io/fig/install.html was moved to http://www.fig.sh/
Please update your README file.
Thanks.
Florin

This is perfect and saved me a lot of time. I owe you a beer.

Hi

I'm using boot2docker.
fig up.
Storm UI working @ http://192.168.59.103:49080/
I tried:
storm jar target/my.jar my.poackage.Topology my-topology -c nimbus.host=192.168.59.103
--> connection refused
storm jar target/my.jar my.poackage.Topology my-topology -c nimbus.host=192.168.59.103:49627
--> UnknownHostException
storm jar target/my.jar my.poackage.Topology my-topology 192.168.59.103
--> Connection refused

I'm obviously doing something wrong, but what? :-)

PS I looked at: https://github.com/wurstmeister/storm-kafka-0.8-plus-test
I don't understand what you mean by - and how it relates to the default storm-docker (without kafka). Is it the IP:port of the zookeeper?

I must say I checked the storm documentation as well:
http://storm.incubator.apache.org/documentation/Command-line-client.html
which just states : Syntax: storm jar topology-jar-path class ...
without explaining what '...' includes

Thank you very much for your patience.

Hi,
When I run the command "storm jar /package/hub-0.0.1-SNAPSHOT.jar com.phoenix.hub.HubApplication phoenixTopology", it get "Could not find or load main class " exception,
how to fix this

Any intention to support jdk 1.8 here?
Thanks!
Tyson

From XXX:latest，for some reason i need to set other parameter,so we should use LATEST for each Dockerfile header.

Ignoring exception while trying to get leader nimbus info from localhost. will retry with a different seed host.
org.apache.storm.thrift.protocol.TProtocolException: Required field 'nimbuses' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:172.17.0.5, uptime_secs:1775, num_workers:4, num_used_workers:0, supervisor_id:abc2e7a9-7948-41c8-9228-a9e8f4c51a91, version:VERSION_NOT_PROVIDED)], nimbus_uptime_secs:1781, topologies:[], nimbuses:null)
at org.apache.storm.generated.ClusterSummary.validate(ClusterSummary.java:622) ~[storm-core-1.0.1.jar:1.0.1]

Hello!
Can you please give an example where do you have more than one worker node? I can see in the fig.yml file that there is only one supervisor. But if I need more how to proceed?
Thanks.
Florin

Hi. The official documentation for Storm says that the following processes must be run under Monit or deamontools (or another "supervisor" process) so that if they crash they will be automatically restarted:

• Numbus
• Supervisors
• Zookeeper
• Storm UI

It also says that cron must be configured in order to compact Zookeeper's data and transaction logs.

Are these things arranged for by wurstmeister/storm-docker, or is this something that I would have to configure myself?

Thanks.

|>oug