SSH launch failing on slave ran with supervisor about docker-plugin HOT 4 CLOSED

Yes, I observe this too :-(

What I think happens is Jenkins launches the slave, then immediately tries to connect to it (we defer this to the SSH Launcher, which is equivalent to you ssh'ing from the console). This is all in DockerComputerLauncher (which defers to SSHLauncher, which is part of jenkins).

If that connection happens very quickly, I think sometimes the sshd in the container isn't ready yet, and the launch fails. It seems a shame that the SSHLauncher doesn't retry (it does wait 60 seconds to get that connection).

If this is true, we might be able to fix by retrying the SSH launch.

from docker-plugin.

That is a shame that it doesn't retry. :(

I'll look into seeing if that is the problem then!

from docker-plugin.

Hmm. I don't know if it's related to these fixes, but my Jenkins now eventually dies from having too many threads. It seems dead SSH connections leave a reader lying around:

"reader" daemon prio=3 tid=0x000000000ae59000 nid=0x2063f runnable [0xfffffd7f44d1a000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at net.schmizz.sshj.transport.Reader.run(Reader.java:68)

from docker-plugin.

@magnayn I'm not sure. My jenkins has been running for about a week straight without dying and I am spawning slaves at a pretty good rate.

from docker-plugin.