sshtools / maverick-synergy Goto Github PK

I attached both of the logs and its becoming a head scratcher. Is it a possibility the server is requesting a key that isnt supported in Maverick?

As per usual, your help is greatly appreciate.

MaverickLog.txt
WinScpLog.txt

Thank you!
-J

Hi,
i'm using:

SshKeyPair pair = SshKeyPairGenerator.generateKeyPair(SshKeyPairGenerator.ED25519);
SshPrivateKeyFile sshPrivateKeyFile = SshPrivateKeyFileFactory.create(pair, passphrase, SshPrivateKeyFileFactory.OPENSSH_FORMAT);
SshPublicKey sshPublicKey = pair.getPublicKey();
SshPublicKeyFile sshPublicKeyFile = SshPublicKeyFileFactory.create(sshPublicKey, comment, SshPrivateKeyFileFactory.OPENSSH_FORMAT);

and generated keys not working with eg. GitHub, but if i change version to 3.0.7 or 3.0.6 it works.
Version 3.0.9 is also broken

The library can create an infinite loop if the ssh server is shutdown twice due to the shutdown method using the getExecutorService() method which will create a new executor service and add a new shutdown hook after the server has been shutdown once.

resolved by #23

I am using v3.0.3-FINAL and its SshPrivateKeyFileFactory to extract a private key from a key pair like so:

SshKeyPair pair = SshKeyPairGenerator.generateKeyPair("ssh-rsa", 2048);

SshPrivateKeyFile prvfile = SshPrivateKeyFileFactory.create(pair, "", SshPrivateKeyFileFactory.OPENSSL_FORMAT);

It fails with a message saying: java.lang.ClassNotFoundException: com.sshtools.common.publickey.OpenSSHPrivateKeyFileBC

I have included the com.sshtools:maverick-bc library in my imports.

Looking at the code, it looks like the tryBc method on line 176 of the SshPrivateKeyFileFactory is looking for a class named:
"com.sshtools.common.publickey.OpenSSHPrivateKeyFileBC"
but the maverick-bc class is at:
"com.sshtools.common.publickey.bc.OpenSSHPrivateKeyFileBC"

The missing "bc" in the package path seems to be the problem.

How do you set the SecurityLevel to indicate the allowed key ciphers when starting the SshServer? Is there a way to do it after the server is created?

Getting this exception with bouncy castle 1.69. Probably not supported yet.

java.security.spec.InvalidKeySpecException: encoded key spec not recognized: 'buf' must have length 32
with bouncy castle 1.69 version. The work around is to go back to 1.68.

org.bouncycastle
bcprov-jdk15on
1.69

Here is a simple test case

import com.sshtools.common.publickey.*;
import com.sshtools.common.ssh.components.SshKeyPair;

import java.io.IOException;
import java.nio.charset.StandardCharsets;

public class TestCase {

private static final String EXAMPLE_PRIVATE_KEY = "-----BEGIN OPENSSH PRIVATE KEY-----\n" +
        "b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABBg\n" +
        "cV+MWCsqs95/2gU8qPWwAAAAEAAAAAEAAAAzAAAAC3NzaC1lZDI1NTE5AAAAIAxP\n" +
        "t6ivXsQjygWAMO7TvzmTKdNo/0gLd5XBw6ILSa6QAAAAkHQmYjPlbGWnwlBjeb1T\n" +
        "ATBpThm4mY9iJ5I5sustQVqJF/meE+eygDbuLFrCjXpvargV2WtHbrOVQB8kbUPd\n" +
        "jB8HFjnvEP8dL75q5Hngab2V2uVt8vJ0+d8vdrTwqCasFW/ASvzudrDJtFV+KGxv\n" +
        "7m2LvH+Xgyc8ChxyZfIBB6Jtoy1KkQyA2V4d8ILB4qLFmA==\n" +
        "-----END OPENSSH PRIVATE KEY-----\n";
private static final String EXAMPLE_PRIVATE_KEY_PASSPHRASE = "bbUNtjXPtsCA75!!";

public static String testGeneratePublicKey() throws InvalidPassphraseException, IOException {
    SshKeyPair pair = SshKeyUtils.getPrivateKey(EXAMPLE_PRIVATE_KEY, EXAMPLE_PRIVATE_KEY_PASSPHRASE);
    SshPublicKeyFile kf =
            SshPublicKeyFileFactory.create(
                    pair.getPublicKey(), "Comment Test", SshPrivateKeyFileFactory.OPENSSH_FORMAT);
    return new String(kf.getFormattedKey(), StandardCharsets.UTF_8);
}


public static void main(String[] args) {
    try {
        System.out.println(testGeneratePublicKey());
    } catch (Throwable e) {
        e.printStackTrace();
    }
}

}

Using latest release 3.0.9

We have to connect to a third-party sftp-server, wich is none of the common ones (eg open-ssh).
On channel open they transmit an initial window-size of 2^31 wich causes an IllegalArgumentException:

com.sshtools.common.ssh.SshException: java.lang.IllegalArgumentException
	at com.sshtools.client.sftp.SftpChannel.initializeSftp(SftpChannel.java:327)
	at com.sshtools.client.sftp.SftpChannel.<init>(SftpChannel.java:202)
	at com.sshtools.client.sftp.SftpClient.<init>(SftpClient.java:131)
	at com.sshtools.client.sftp.SftpClient.<init>(SftpClient.java:123)
	at com.sshtools.client.sftp.SftpClient.<init>(SftpClient.java:135)
	...
Caused by: java.lang.IllegalArgumentException
	at java.nio.Buffer.limit(Buffer.java:275)
	at com.sshtools.synergy.ssh.ChannelNG.sendChannelDataAndBlock(ChannelNG.java:524)
	at com.sshtools.synergy.ssh.ChannelNG.sendChannelDataAndBlock(ChannelNG.java:476)
	at com.sshtools.synergy.ssh.ChannelNG.sendChannelDataAndBlock(ChannelNG.java:462)
	at com.sshtools.client.tasks.AbstractSubsystem.sendMessage(AbstractSubsystem.java:137)
	at com.sshtools.client.sftp.SftpChannel.initializeSftp(SftpChannel.java:239)
	... 18 more

This is because of invalid handling of window-sizes > 2^31-1 :

In Section 5.2 of rfc 4254 it is stated, that

Implementations MUST correctly handle window sizes of up to 2^32 - 1 bytes.

But as 'int' is used as datatype in ChannelDataWindow.java and referencing places it can't be handled in java because of wrap-around (only signed integer is supported).
I suggest to migrate to long as datatype for maximumWindowSpace, minimumWindowSpace, maximumPacketSize and initialWindowSpace.

As hotfix a size-check in ChannelDataWindow can be used as workaround.
I've attached a patch to this issue.

Greetz,
Karsten

ChannelDataWindow.zip

When i use the Client API (see code below at the bottom) as mentioned in the API docs, default SshClientContext creates and initializes the SShEngine.DefaultInstance() which does the startup of the instance.

During startup, 1 acceptor thread, 1 connector thread and 2 transfer threads are created. The following properties control the number of threads and I can change the thread count.

maverick.config.connect.threads (default 1 thread)
maverick.config.transfer.threads (default 2 threads)
maverick.config.accept.threads (default 1 thread)

When using the only as a client, is it safe to turn off acceptor threads by setting the system property maverick.config.accept.threads to 0. I don't see a need to spin this thread.

Also, what are connect and transfer threads used for. Any guidance on setting the appropriate values for them.

try(SshClient ssh = new SshClient("localhost", 22, "lee", "xxxxxx".toCharArray())) {
runTask(new SftpClientTask(ssh) {
protected void doSftp() {
}
});
}

I have to connect to an ssh-server, wich is not under my control.
The protocol-version that is used is '5'.

I get following exception, when processing supported features:

java.io.IOException: Unexpected length of 1936744803 bytes exceeds available data of 72 bytes
	at com.sshtools.common.util.ByteArrayReader.checkLength(ByteArrayReader.java:99)
	at com.sshtools.common.util.ByteArrayReader.readString(ByteArrayReader.java:232)
	at com.sshtools.common.util.ByteArrayReader.readString(ByteArrayReader.java:220)
	at com.sshtools.client.sftp.SftpChannel.processSupported(SftpChannel.java:424)
	at com.sshtools.client.sftp.SftpChannel.initializeSftp(SftpChannel.java:285)
	... 19 more

I've checked the sftp-documentation and it states that the supported-structure is as follows:

string "supported"
        string supported-structure
           uint32 supported-attribute-mask
           uint32 supported-attribute-bits
           uint32 supported-open-flags
           uint32 supported-access-mask
           uint32 max-read-size
           string extension-names[0..n]

in SftpChannel the data is processed in this way:

if(supportedStructure.available() >= 4) {
	int count = (int) supportedStructure.readInt();
		for (int i = 0; i < count; i++) {
			String ext = supportedStructure.readString();
			if(Log.isTraceEnabled()) {
				Log.trace("Server supports '" + ext
						+ "' extension");
			}
			supportedExtensions.add(ext);
		}
	}

So if there is data available it determines the number of extension an than reads as many strings.
i don't know, if this is correct, but it doesn't work.
i think the documentation has to be read as follows: as long as data is available read string.

if changing the if-block quoted above to

while (supportedStructure.available() >= 4) {
	String ext = supportedStructure.readString();
	if(Log.isTraceEnabled()) {
		Log.trace("Server supports '" + ext
				+ "' extension");
	}
	supportedExtensions.add(ext);
}

it works fine.

In the log following messages show up:

Processed extension 'supported'
Server supports 'space-available' extension
Server supports '[email protected]' extension
Server supports '[email protected]' extension

wich seems to make sense...

Greetz,
Karsten

In the com.sshtools.common.files.vfs.VirtualMappedFile#toActualPath (

To replicate this, create a virtual mount that has a mount with a leading slash and no trailing slash (e.g., /readonly on ram://root/read ). Then query a virtual path for that mount (e.g., /readonly/file.txt).

The toActualPath method will execute line 218:

str = FileUtils.addTrailingSlash(parentMount.getRoot())
					+ virtualPath.substring(parentMount.getMount().length());

The virtualPath substring call will result in /file.txt, which is appended to the root with trailing slash ending up with an actual path of: ram://root/read//file.txt which has a double slash.

Since VirutalMountTemplate strips the trailing slashes, I am not sure how to work around this problem.

Observation

Currently the Signing CA: RSA uses ssh-rsa which is deprecated.

From #14, certificate signing with github extension produces:

        Type: [email protected] user certificate
        Public key: RSA-CERT SHA256:JLm/c5O3ZbLd2/7pD8Z/PsHM8TiXJjx7jDHeJ4If1QA
        Signing CA: RSA SHA256:UpGjC+XHqiqmky+maxVXUrH3ek/+PS5K6ZjEWLRKOZU (using ssh-rsa)
        Key ID: "KEY-IDENTITY"
        Serial: 0
        Valid: from 2021-05-19T19:00:00 to 2021-08-17T19:00:00
        Principals: (none)
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc
                [email protected] UNKNOWN OPTION (len 16)

This fails to authenticate because its not rsa-sha2-***.

RSA keys: Add support for rsa-sha2-256 and rsa-sha2-512

Looks like this ->

Signing CA: RSA SHA256:rzmG+gyHbxsvaD1LkehnMjsIrzOAavGtlExpq9MOhc4 (using rsa-sha2-512)

RFC: https://tools.ietf.org/rfc/rfc8332.txt

This memo updates RFCs 4252 and 4253 to define new public key
   algorithms for use of RSA keys with SHA-256 and SHA-512 for server
   and client authentication in SSH connections.

New RSA Public Key Algorithms

This memo adopts the style and conventions of [RFC4253] in specifying
how use of a public key algorithm is indicated in SSH.

The following new public key algorithms are defined:

 rsa-sha2-256        RECOMMENDED    sign    Raw RSA key
 rsa-sha2-512        OPTIONAL       sign    Raw RSA key

These algorithms are suitable for use both in the SSH transport layer
[RFC4253] for server authentication and in the authentication layer
[RFC4252] for client authentication.

Since RSA keys are not dependent on the choice of hash function, the
new public key algorithms reuse the "ssh-rsa" public key format as
defined in [RFC4253]:

string "ssh-rsa"
mpint e
mpint n

All aspects of the "ssh-rsa" format are kept, including the encoded
string "ssh-rsa". This allows existing RSA keys to be used with the
new public key algorithms, without requiring re-encoding or affecting
already trusted key fingerprints.

Signing and verifying using these algorithms is performed according
to the RSASSA-PKCS1-v1_5 scheme in [RFC8017] using SHA-2 [SHS] as
hash.

For the algorithm "rsa-sha2-256", the hash used is SHA-256.
For the algorithm "rsa-sha2-512", the hash used is SHA-512.

Observation

When CA signed user certificate generation is attempted with extensions, I notice that the produced signed certificate has no extensions.

Investigation

The extensions are not being written and processed correctly. It also seems that key value extensions only handle key (flag) based.

In OpenSshCertificate.java extensions are read in assuming that it has no key value pairs:

for(String ext : extensions) {
	this.extensions.put(ext, "");
}

This will fail for my case where Github requires an extension that needs a value. In my code I have something like this for ssh-keygen process generation: String.format("extension:%s=%s", key, value).

In the #sign() the ByteArrayWriter (or some other writer) never writes the extensions so when #decodeCertificate(reader) is called we have:

tmp = new ByteArrayReader(reader.readBinaryString());
this.extensions = new HashMap<>();
this.extensionOrder = new ArrayList<>();
while (tmp.available() > 0) {
	String name = tmp.readString().trim();
	extensionOrder.add(name);
	extensions.put(name, tmp.readString());
}
tmp.close();

The extensions that had values in it are now removed from the state and tmp is empty at this stage since no extensions were written. Whats interesting with the above code is that this could work with key value pairs if they were written in the first place.

When the OpenSshCertificate is generated, the extension map is empty.

Ask

Can extensions be encoded correctly with the certificate?
Can extensions support key / value pairs ie. Github Ssh Certificates?

Update blog article with key value pair example: https://www.jadaptive.com/openssh-certificate-cheat-sheet/

Update API docs:
https://www.jadaptive.com/app/manpage/en/article/2895616

Consider creating a new overloaded method that takes in a Map of Extensions instead of a List since you end up trying to convert it anyway in the code.

Notes

I would be happy to make time to work with the team to help code this if there is not enough staffing on the JAdaptive side to handle this.

I can also help contribute to the cheat sheet by providing a Github SSH Signing Example.

version 3.0.10
I'm getting this error:

Caused by: java.io.IOException: Task did not succeed and did not report an error
at com.sshtools.client.SshClient.doTask(SshClient.java:288)

This error message stinks. The reason it happens is because the throwable is getting saved here:
com.sshtools.common.ssh.ConnectionAwareTask#lastError
but the error is getting read from here:
com.sshtools.client.tasks.AbstractSessionTask#lastError

The documentation for enabling per-connection logging (https://www.jadaptive.com/app/manpage/en/article/1564950) suggests that the user and ident properties are optional filters to reduce the logs. However, the system is not producing any log files if I do not provide values for those properties.

The isLoggingUser and isLoggingIdentifier methods in ConnectionLoggingContext seem to have the bug in the first 4 lines of their code:

      private boolean isLoggingUser(Connection<?> con) {
			
		String v = getProperty(".user", "");
		if("".equals(v)) {         // <-- The property value defaults to the blank value so the returned value is false if a value is not provided
			return false;
		}
		Set<String> users = new HashSet<String>(Arrays.asList(v.split(",")));
		if(!Objects.isNull(con.getUsername())) {
			return users.contains(con.getUsername());
		} else {
			return users.isEmpty();
		}
	}

I am getting FileNotFound Exception in put(String local,String remote) method when using the below dependency in maven file.

com.sshtools
maverick-synergy-client
3.0.0-SNAPSHOT

and the code snippet as below
ssh.runTask(new SftpClientTask(ssh) {
@OverRide
protected void doSftp() {
try {
log.info("File to Copy:{}", fileToCopy.getAbsolutePath());
put(fileToCopy.getAbsolutePath(), remoteDir);
log.info("SFTP File put operation done successfully");
isSftpSuccessful = Boolean.TRUE;
} catch (FileNotFoundException e) {
e.printStackTrace();
message = "Seems invalid file is being uploaded through SFTP";
isSftpSuccessful = Boolean.FALSE;
} catch (SftpStatusException e) {
e.printStackTrace();
if (e.getStatus() == SftpStatusException.SSH_FX_PERMISSION_DENIED) {
message = "Failed to create file in remote directory as the configured user don't have permission to create one";
}
isSftpSuccessful = Boolean.FALSE;
} catch (SshException e) {
e.printStackTrace();
isSftpSuccessful = Boolean.FALSE;
} catch (TransferCancelledException e) {
e.printStackTrace();
isSftpSuccessful = Boolean.FALSE;
} catch (Exception e) {
e.printStackTrace();
isSftpSuccessful = Boolean.FALSE;
}
}
});

This was working fine 1 week back when I had the maverick-synergy-client-3.0.0-20200225.155328-75.jar in my .m2 repository.
After I build my code again yesterday,I got some new dependencies downloaded in my .m2 repo and the maverick-synergy-client-3.0.0-20200405.143821-96.jar.Since then I an getting below exception

File to Copy:C:\Users\PSEZWAR\WelcomeSFTP.txt
java.io.FileNotFoundException: C:\Users\PSEZWAR\C:\Users\PSEZWAR\WelcomeSFTP.txt (The filename, directory name, or volume label syntax is incorrect)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at com.sshtools.common.files.direct.AbstractDirectFile.getInputStream(AbstractDirectFile.java:109)
at com.sshtools.client.sftp.SftpClient.put(SftpClient.java:1778)
at com.sshtools.client.sftp.SftpClient.put(SftpClient.java:1892)
at com.sshtools.client.sftp.SftpClientTask.put(SftpClientTask.java:976)
at com.xylem.owareportservice.util.SFTPUtil$1.doSftp(SFTPUtil.java:129)
at com.sshtools.client.sftp.SftpClientTask.doTask(SftpClientTask.java:55)
at com.sshtools.common.ssh.ConnectionAwareTask.run(ConnectionAwareTask.java:45)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I have my local file at C:\Users\PSEZWAR\WelcomeSFTP.txt but the code is appending the C:\Users\PSEZWAR\ to my file location resulting the FileNotFoundException.
Please look into this issue as this was not existing in previous revision.Please help me to resolve the issue.

MODULE:
This issue was discovered when implementing SFTP server using maverick-synergy-server API with maverick-virtual-filesystem.

ISSUE:
If you use get to transfer the servers own log file and have connection logging on the file transfer will eventually throw StackOverFlowError that has following cycle in stack (Bottom to top is the call order).
at com.sshtools.common.logger.FileLoggingContext.logToFile(FileLoggingContext.java:84)
at com.sshtools.common.logger.FileLoggingContext.log(FileLoggingContext.java:79)
at com.sshtools.common.logger.FileLoggingContext.createLogFile(FileLoggingContext.java:68)
at com.sshtools.common.logger.FileLoggingContext.checkRollingLog(FileLoggingContext.java:109)
at com.sshtools.common.logger.FileLoggingContext.logToFile(FileLoggingContext.java:84)

This can happen even if you don't have connection logging on but then it doesn't happen every time.

There are at least these issues:

1. The log rotate fails because the file is the file we are transferring.
2. The transfer hangs in client side waiting data that will never arrive because the transfer thread dies.
3. The transfer tries to send the ever growing file completely but it fails because the file grows faster than it can be sent.

My initial reasoning is that because the log rotate can never rename the original log file because it is locked for transfer.
When it thinks log rotate is done it tries to do the initial logging at the very beginning of the new log file telling the file it now logs on to.
But because the log rename fails (This is not checked btw) this initial logging is actually tried on the old log file.
It does log rotate check and sees that the file is bigger than maximum size so it tries to log rotate... and here we go in the merry go round until we run out of stack space.

STEPS TO REPRODUCE:
Create SFTP server so that you set the logging settings as follows:

1. Set the server log file path to be inside the SFTP servers virtual file system.
2. Put connection logging level to TRACE.
3. Connect to this server and use 'get' to download the server logfile.

This new error has been happening, it seems that the files are downloading but our job deletes them after because of the error. What could be causing this and how can we resolve it? Thanks as always

"[Invalid message length in SFTP protocol [40], Invalid message length in SFTP protocol [1694498825], Invalid message length in SFTP protocol [1426063360], Invalid message length in SFTP protocol [0], Invalid message length in SFTP protocol [391407717], Invalid message length in SFTP protocol [544174181], Invalid message length in SFTP protocol [1918989417]

I'm using the ls command in the SftpClientTask but I get an exception that it is not a directory. Which is fine, as I was currently checking if its a file or not. What is the alternative to check for a file if it exists?

Thank you!!

I'm trying to use the 3.0.9 client jar, and I am getting...

Exception in thread "main" java.lang.NoClassDefFoundError: com/sshtools/common/util/IOUtils
at com.sshtools.common.publickey.SshKeyUtils.getPrivateKey(SshKeyUtils.java:80)
at com.sshtools.client.SshClient.(SshClient.java:80)
at com.sshtools.client.SshClient.(SshClient.java:76)

I can't find IOUtils in any of the jars in the 3.0.9 release.

Greetings!

I ran into an issue and I am not sure how to go about it or what I need to do on my end to fix it. Below is some of the log, if you want I can post the complete DEBUG log. I looked at the source and see that it's a less than but the value of p is actually equal to, is there a way to go around that? I may be completely off base since I am still learning.

[ pool-1-thread-1] DEBUG - Lang:
[ pool-1-thread-1] DEBUG - First Packet Follows: false
[ pool-1-thread-1] DEBUG - Local Key Exchanges: diffie-hellman-group-exchange-sha256,[email protected],curve25519-sha256,diffie-hellman-group18-sha512,diffie-hellman-group17-sha512,diffie-hellman-group16-sha512,diffie-hellman-group15-sha512,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,rsa2048-sha256,diffie-hellman-group14-sha1
[ pool-1-thread-1] DEBUG - Local Public Keys: ecdsa-sha2-nistp256,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,rsa-sha2-512,rsa-sha2-256,[email protected],[email protected],[email protected],ssh-rsa,[email protected]
[ pool-1-thread-1] DEBUG - Local Ciphers CS: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr,aes256-cbc,3des-cbc,aes128-cbc
[ pool-1-thread-1] DEBUG - Local Ciphers SC: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr,aes256-cbc,3des-cbc,aes128-cbc
[ pool-1-thread-1] DEBUG - Local Macs SC: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
[ pool-1-thread-1] DEBUG - Local Macs CS: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
[ pool-1-thread-1] DEBUG - Local Compression CS: none
[ pool-1-thread-1] DEBUG - Local Compression SC: none
[ pool-1-thread-1] DEBUG - Firing EVENT_KEY_EXCHANGE_INIT success=true
[ pool-1-thread-1] DEBUG - Lang:
[ pool-1-thread-1] DEBUG - First Packet Follows: false
[ pool-1-thread-1] DEBUG - Local Key Exchanges: diffie-hellman-group-exchange-sha256,[email protected],curve25519-sha256,diffie-hellman-group18-sha512,diffie-hellman-group17-sha512,diffie-hellman-group16-sha512,diffie-hellman-group15-sha512,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,rsa2048-sha256,diffie-hellman-group14-sha1
[ pool-1-thread-1] DEBUG - Local Public Keys: ecdsa-sha2-nistp256,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,rsa-sha2-512,rsa-sha2-256,[email protected],[email protected],[email protected],ssh-rsa,[email protected]
[ pool-1-thread-1] DEBUG - Local Ciphers CS: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr,aes256-cbc,3des-cbc,aes128-cbc
[ pool-1-thread-1] DEBUG - Local Ciphers SC: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr,aes256-cbc,3des-cbc,aes128-cbc
[ pool-1-thread-1] DEBUG - Local Macs CS: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
[ pool-1-thread-1] DEBUG - Local Macs SC: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
[ pool-1-thread-1] DEBUG - Local Compression CS: none
[ pool-1-thread-1] DEBUG - Local Compression SC: none
[ pool-1-thread-1] DEBUG - Firing EVENT_KEY_EXCHANGE_INIT success=true
[ pool-1-thread-1] INFO - Using BC for DH; prime range is 1024 to 8192 bits
[ pool-1-thread-1] DEBUG - Negotiated Key Exchange: diffie-hellman-group-exchange-sha256
[ pool-1-thread-1] DEBUG - Negotiated Public Key: ssh-rsa
[ pool-1-thread-1] DEBUG - Negotiated Cipher CS: aes256-ctr
[ pool-1-thread-1] DEBUG - Negotiated Cipher SC: aes256-ctr
[ pool-1-thread-1] DEBUG - Negotiated Mac CS: hmac-sha1
[ pool-1-thread-1] DEBUG - Negotiated Mac SC: hmac-sha1
[ pool-1-thread-1] DEBUG - Negotiated Compression CS: none
[ pool-1-thread-1] DEBUG - Negotiated Compression SC: none
[ pool-1-thread-1] DEBUG - Minimum DH prime size is 2048
[ pool-1-thread-1] DEBUG - Preferred DH prime size is 2048
[ pool-1-thread-1] DEBUG - Maximum DH prime size is 8192
[ pool-1-thread-1] DEBUG - Sent SSH_MSG_KEY_DH_GEX_REQUEST
com.sshtools.common.ssh.SshException: Minimum DH p value not provided [1024]
at com.sshtools.client.components.DiffieHellmanGroupExchange.exchangeGroup(DiffieHellmanGroupExchange.java:270)
at com.sshtools.client.components.DiffieHellmanGroupExchange.processMessage(DiffieHellmanGroupExchange.java:405)
at com.sshtools.synergy.ssh.TransportProtocol.processMessage(TransportProtocol.java:1992)
at com.sshtools.synergy.ssh.TransportProtocol.processBinaryPackets(TransportProtocol.java:627)
at com.sshtools.synergy.ssh.TransportProtocol.onSocketRead(TransportProtocol.java:457)
at com.sshtools.client.TransportProtocolClient.onSocketRead(TransportProtocolClient.java:132)
at com.sshtools.synergy.nio.SocketConnection.processReadEvent(SocketConnection.java:294)
at com.sshtools.synergy.nio.SshEngine$SocketReadWriteTask.doTask(SshEngine.java:675)
at com.sshtools.common.ssh.ConnectionAwareTask.run(ConnectionAwareTask.java:45)
at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.executeAllTasks(ExecutorOperationSupport.java:128)[ pool-1-thread-1] INFO - Using BC for DH; prime range is 1024 to 8192 bits
[ pool-1-thread-1] DEBUG - Negotiated Key Exchange: diffie-hellman-group-exchange-sha256
[ pool-1-thread-1] DEBUG - Negotiated Public Key: ssh-rsa
[ pool-1-thread-1] DEBUG - Negotiated Cipher CS: aes256-ctr
[ pool-1-thread-1] DEBUG - Negotiated Cipher SC: aes256-ctr
[ pool-1-thread-1] DEBUG - Negotiated Mac CS: hmac-sha1
[ pool-1-thread-1] DEBUG - Negotiated Mac SC: hmac-sha1
[ pool-1-thread-1] DEBUG - Negotiated Compression CS: none
[ pool-1-thread-1] DEBUG - Negotiated Compression SC: none
[ pool-1-thread-1] DEBUG - Minimum DH prime size is 2048
[ pool-1-thread-1] DEBUG - Preferred DH prime size is 2048
[ pool-1-thread-1] DEBUG - Maximum DH prime size is 8192
[ pool-1-thread-1] DEBUG - Sent SSH_MSG_KEY_DH_GEX_REQUEST
com.sshtools.common.ssh.SshException: Minimum DH p value not provided [1024]
at com.sshtools.client.components.DiffieHellmanGroupExchange.exchangeGroup(DiffieHellmanGroupExchange.java:270)
at com.sshtools.client.components.DiffieHellmanGroupExchange.processMessage(DiffieHellmanGroupExchange.java:405)
at com.sshtools.synergy.ssh.TransportProtocol.processMessage(TransportProtocol.java:1992)
at com.sshtools.synergy.ssh.TransportProtocol.processBinaryPackets(TransportProtocol.java:627)
at com.sshtools.synergy.ssh.TransportProtocol.onSocketRead(TransportProtocol.java:457)
at com.sshtools.client.TransportProtocolClient.onSocketRead(TransportProtocolClient.java:132)
at com.sshtools.synergy.nio.SocketConnection.processReadEvent(SocketConnection.java:294)
at com.sshtools.synergy.nio.SshEngine$SocketReadWriteTask.doTask(SshEngine.java:675)
at com.sshtools.common.ssh.ConnectionAwareTask.run(ConnectionAwareTask.java:45)
at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.executeAllTasks(ExecutorOperationSupport.java:128)

I'd like to say thanks for this awesome library! It's solved so many issues for us and I am learning more about SSH connectivity.

Any help would be greatly appreciated. Thank you!

Getting this error while running. It seams you have compiled it with JDK higher than 8. When running the jar with version 8 throws this error.
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;|

at com.sshtools.synergy.nio.SocketConnection.processWriteEvent(SocketConnection.java:358)|

at com.sshtools.synergy.nio.SshEngine$SocketReadWriteTask.doTask(SshEngine.java:979)|

at com.sshtools.common.ssh.ConnectionAwareTask.run(ConnectionAwareTask.java:45)|

at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.executeAllTasks(ExecutorOperationSupport.java:127)|

at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.run(ExecutorOperationSupport.java:84)|

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)|

at java.util.concurrent.FutureTask.run(FutureTask.java:266)|

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)|

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)|

at java.lang.Thread.run(Thread.java:750)|

Here (https://www.jadaptive.com/app/manpage/en/article/1579899) documentation says that using OpenSSH Certificate Authenticator I can configure the SSH server to accept CA-signed public certificates.

I know that using ssh-keygen utility I can sign client public certificate with server private key, but how can I do this operation with maverick API?

Best regards,
Lorenzo.

When using putLocalDirectory(sourceFile.getAbsolutePath(), deployment.getDestinationDirectoryName(), true, true, new FileTransferProgress() {...})
with sync true, then the files copied will be deleted.
This is because when the SftpClient asks the DirectoryOperation if it contains a specific local file it will never say true.
AbstractDirectFile needs to override the equals method in order to be found.

Hi guys, I am running into an issue when trying to gracefully handle an SSH server being unreachable. When I initialize the client as follows

try (SshClient ssh = new SshClient(hostname, port, username, 5_000L, password.toCharArray())) {
    ...
} catch (IOException | SshException e) {
    log.error("Failed to connect: {}", e.getMessage());
}

I would expect the catch clause to fire if no connection can be established within 5 seconds. However, this is not the case. What happens when the timeout has elapsed?

Thanks!

Hello,

We are using maverick-synergy version 3.1.1 with maverick-bc module.

We have been trying to get RSA SHA2 certificate authentication working but it's failing. ECDSA and ED25519 certificate authentication is working fine.

I have created certificates for server and client using following commands. My SSH version was OpenSSH_9.6p1, LibreSSL 3.3.6

ssh-keygen -t rsa-sha2-256 -b 4096 -f ssh_ca -C ssh_ca

ssh-keygen -f ssh_host_rsa_key -N '' -b 4096 -t rsa-sha2-256
ssh-keygen -s ssh_ca -t rsa-sha2-256 -I SERVER -h ssh_host_rsa_key.pub

ssh-keygen -f ssh_client_rsa_key -N '' -b 4096 -t rsa-sha2-256
ssh-keygen -s ssh_ca -t rsa-sha2-256 -I CLIENT -n testuser ssh_client_rsa_key.pub
ssh-keygen -t rsa-sha2-256 -o -p -f ssh_client_rsa_key

This creates following certificates with certificate type of [email protected], we'll return to this later.

ssh-keygen -L -f ssh_host_rsa_key-cert.pub                                         
ssh_host_rsa_key-cert.pub:
        Type: [email protected] host certificate
        Public key: RSA-CERT SHA256:GRoyO8CKePz0v9X35Vxhjacwfx3aBD9zd0NoFMUQ/5I
        Signing CA: RSA SHA256:GUeYV9RkU3SVZEUH3ULi2p0LocWB1dJmlTwTZrUuNVw (using rsa-sha2-256)
        Key ID: "SERVER"
        Serial: 0
        Valid: forever
        Principals: (none)
        Critical Options: (none)
        Extensions: (none)

ssh-keygen -L -f ssh_client_rsa_key-cert.pub 
ssh_client_rsa_key-cert.pub:
        Type: [email protected] user certificate
        Public key: RSA-CERT SHA256:L4fS3d6ZPP9fTLJHsteDOQ3jpS5+0IVhywdjsurKL+E
        Signing CA: RSA SHA256:KqSXZ9vqV6q0mMy5vdpqPH/84ZXie+fMbsrfkuOqkJc (using rsa-sha2-256)
        Key ID: "CLIENT"
        Serial: 0
        Valid: forever
        Principals: 
                testuser
        Critical Options: (none)
        Extensions: 
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

First issue that we ran into was in Class com.sshtools.client.PublicKeyAuthenticator (https://github.com/sshtools/maverick-synergy/blob/master/maverick-synergy-client/src/main/java/com/sshtools/client/PublicKeyAuthenticator.java) where method private boolean setupNextKey() throws IOException, SshException {}. It seems that currentKey is both instance of SshRsaPublicKey and instance of OpenSshRsaCertificate and first condition currentKey instanceof SshRsaPublicKey matches also in certificate use case and the else if block is never visited even when certificate is used. Client then connects to the server using normal public key instead of using a certificate.

if(currentKey instanceof SshRsaPublicKey && currentKey.getBitLength() >= 1024) {
	//handling of public keys
}else if(currentKey instanceof OpenSshRsaCertificate && currentKey.getBitLength() >= 1024)
{
	//handling of certificates
}

We think that the correction could be to swap the conditions, so that OpenSshRsaCertificate is checked first. Normal public keys shouldn't match OpenSshRsaCertificate condition.

if(currentKey instanceof OpenSshRsaCertificate && currentKey.getBitLength() >= 1024) {
        //handling of certificates
}else if(currentKey instanceof SshRsaPublicKey && currentKey.getBitLength() >= 1024)
{
	//handling of public keys
}

Second issue was also in the Class com.sshtools.client.PublicKeyAuthenticator (https://github.com/sshtools/maverick-synergy/blob/master/maverick-synergy-client/src/main/java/com/sshtools/client/PublicKeyAuthenticator.java). Code checks if server supports SHA512 signatures but then currentKey is populated with new OpenSshRsaSha256Certificate and signed with SHA512. This will result in invalid signature which is rejected by SSH server.

if(policy.getSupportedSignatures().contains(SshContext.PUBLIC_KEY_RSA_SHA512)) {
	signingAlgorithm = SshContext.PUBLIC_KEY_RSA_SHA512;
	currentKey = new OpenSshRsaSha256Certificate().init(currentKey.getEncoded());
} else if(policy.getSupportedSignatures().contains(SshContext.PUBLIC_KEY_RSA_SHA256)) {
	signingAlgorithm = SshContext.PUBLIC_KEY_RSA_SHA256;
	currentKey =  new OpenSshRsaSha256Certificate().init(currentKey.getEncoded());
} else {
	Log.debug("Server does not support {} signature for key {}",
			currentKey.getSigningAlgorithm(),
			SshKeyUtils.getOpenSSHFormattedKey(currentKey));
	continue;
}

Accepted certificate ID "CLIENT" (serial 0) signed by RSA CA SHA256:KqSXZ9vqV6q0mMy5vdpqPH/84ZXie+fMbsrfkuOqkJc via /etc/ssh/ssa_ca.pub
debug3: mm_answer_keyallowed: publickey authentication: RSA-CERT key is allowed
debug3: mm_request_send: entering, type 23
debug3: mm_sshkey_verify: entering [preauth]
debug3: mm_request_send: entering, type 24 [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 24
debug3: mm_answer_keyverify: publickey RSA-CERT signature unverified: incorrect signature
debug1: auth_activate_options: setting new authentication options
debug3: mm_request_send: entering, type 25
Failed publickey for testuser from 10.20.30.40 port 54306 ssh2: RSA-CERT SHA256:L4fS3d6ZPP9fTLJHsteDOQ3jpS5+0IVhywdjsurKL+E ID CLIENT (serial 0) CA RSA SHA256:KqSXZ9vqV6q0mMy5vdpqPH/84ZXie+fMbsrfkuOqkJc

This can be fixed buy using OpenSshRsaSha512Certificate for the SHA512 signature

if(currentKey instanceof OpenSshRsaCertificate && currentKey.getBitLength() >= 1024) {
	if(policy.getSupportedSignatures().contains(SshContext.PUBLIC_KEY_RSA_SHA512)) {
		signingAlgorithm = SshContext.PUBLIC_KEY_RSA_SHA512;
		currentKey = new OpenSshRsaSha512Certificate().init(currentKey.getEncoded());
	} else if(policy.getSupportedSignatures().contains(SshContext.PUBLIC_KEY_RSA_SHA256)) {
		signingAlgorithm = SshContext.PUBLIC_KEY_RSA_SHA256;
		currentKey =  new OpenSshRsaSha256Certificate().init(currentKey.getEncoded());
	} else {
		Log.debug("Server does not support {} signature for key {}",
				currentKey.getSigningAlgorithm(),
				SshKeyUtils.getOpenSSHFormattedKey(currentKey));
		continue;
	}	 
}

Final issue was in class com.sshtools.common.publickey.OpenSshCertificate (https://github.com/sshtools/maverick-synergy/blob/master/maverick-base/src/main/java/com/sshtools/common/publickey/OpenSshCertificate.java). Method public SshPublicKey init(byte[] blob, int start, int len) throws SshException.

There is a following check which compares header to algorithm.

if (!header.equals(getAlgorithm())) {
	throw new SshException("The encoded key is not DSA",
			SshException.INTERNAL_ERROR);
}

We made a debug print from the situation which returned following

Log.info("read header = "+header);
Log.info("read algorithm = "+getAlgorithm());

INFO - read header = [email protected]
INFO - read algorithm = [email protected]

It seems to us that the code is trying to compare certificate type ([email protected]) to used algorithm ([email protected]) which can never match since only supported certificate type seems to be [email protected] (See cert creation above). We managed to get this work by bypassing this check completely.

Unlike ECDSA where certificate type contains the key length, there only seems to be one type for RSA certificates. SHA256 and SHA512 keys still use the [email protected] as certificate type.

ssh -Q key |grep cert

[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

For one of our use case where we are trying to connect to a sftp server we can see some weird behavior. In this particular case SFTP server allows cd command only with a '/' at end

1. cd dirA/dirB/ - works
2. cd dirA/dirB - does not work
   When we try calling sftpClient.cd("dirA/dirB/") it attempts to call cd dirA/dirB and fails.
   We tried to look into your code and it seems like '/' is removed intentionally.
   Can someone explain what is the reason for that and can we have some workaround to cd/list with '/' at the end

Line 962 in file https://github.com/sshtools/maverick-synergy/blob/master/maverick-synergy-client/src/main/java/com/sshtools/client/sftp/SftpChannel.java

Notice a division by zero exception when log level is set to Info.

if(Log.isInfoEnabled()) {
Log.info("Optimized write to {} took {} seconds at {}", filename, seconds, IOUtils.toByteSize(transfered / seconds, 1));
}

Greetings again!

I am trying to connect to a site using our proxy. I have tested the proxy in another instance (WinSCP) so connectivity is not an issue. The code seems pretty simple

sshClientContext.enableHTTPProxy(PROXY_HOSTNAME, PROXY_PORT, HOST, PORT, PROXY_USERNAME, PROXY_PASSWORD, null, null);

Also tried:
sshClientContext.enableHTTPProxy(PROXY_HOSTNAME, PROXY_PORT, HOST, PORT, PROXY_USERNAME, PROXY_PASSWORD, "", Map.of());

I should mention we have two proxies one with username/password and one without and I have not been able to get either to work.

sshClientContext.enableHTTPProxy(PROXY_HOSTNAME, PROXY_PORT, HOST, PORT, null, null, "", Map.of());

And then:

 try (SshClient ssh = new SshClient(host, port, username, sshClientContext,
                30000,
                password.toCharArray(),
                identities)) {
}

However, there is no action. I tested both the SFTP I am trying to connect to without the proxy and it passes. Then I tested with the proxy and it fails. I tested the proxy itself via WinSCP (as mentioned) and it works.

Here is the logging with the proxy on, I didnt include everything because its the same with the proxy code off:

14 Feb 2022 17:58:51,610 [                main]  DEBUG -    ecdh-sha2-nistp521 (client) will be supported using JCE Provider SunEC
14 Feb 2022 17:58:51,612 [                main]  DEBUG -    rsa2048-sha256 (client) will not be supported: Cannot find any provider supporting RSA/None/OAEPWithSHA256AndMGF1Padding
Using Proxy: proxy.corp.net:9119
Private Key Path: /home/theKey
14 Feb 2022 17:58:51,762 [                main]  TRACE - Adding registration request to queue
14 Feb 2022 17:58:51,762 [     SSHD-TRANSFER-1]  TRACE - Registering channel with interested ops 5
14 Feb 2022 17:58:51,762 [     SSHD-TRANSFER-1]  TRACE - Channel is open
14 Feb 2022 17:58:51,763 [     SSHD-TRANSFER-1]  TRACE - Channel is registered
14 Feb 2022 17:58:51,770 [     SSHD-TRANSFER-1]  DEBUG - Posting message com.sshtools.client.TransportProtocolClient$1 to queue
14 Feb 2022 17:58:51,771 [     SSHD-TRANSFER-1]  TRACE - Registration complete
14 Feb 2022 17:58:51,771 [     SSHD-TRANSFER-1]  TRACE - Flag selector as READ/WRITE
14 Feb 2022 17:58:51,771 [     SSHD-TRANSFER-1]  TRACE - Selected key
14 Feb 2022 17:58:51,771 [     SSHD-TRANSFER-1]  TRACE - Processing transport-client WRITE
14 Feb 2022 17:58:51,772 [     SSHD-TRANSFER-1]  ERROR - Selector thread encountered an error
java.lang.IllegalArgumentException
	at com.sshtools.common.ssh.ConnectionAwareTask.<init>(ConnectionAwareTask.java:32)
	at com.sshtools.synergy.nio.SshEngine$SocketReadWriteTask.<init>(SshEngine.java:656)
	at com.sshtools.synergy.nio.SshEngine$TransferSelectorThread.processSelectionKey(SshEngine.java:641)
	at com.sshtools.synergy.nio.SelectorThread.run(SelectorThread.java:492)

Exception: IOException
Exception message: Failed to connect to sftp.can.com:22
End of SFTP Download

This one I am really stumped on so any help would be greatly appreciated!

after invoking
Log.getDefaultContext().enableConsole( Level.DEBUG );
when stopping tomcat, infamous message appears:

18-Aug-2020 17:06:17.482 WARNING [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [bla] appears to have started a thread named [Thread-5] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: [email protected]/jdk.internal.misc.Unsafe.park(Native Method) [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) [email protected]/java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) [email protected]/java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) [email protected]/sun.nio.fs.AbstractWatchService.poll(AbstractWatchService.java:108) com.sshtools.common.logger.DefaultLoggerContext$FileWatcher.run(DefaultLoggerContext.java:227)

but also:

[FileSystemWatchService] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: [email protected]/sun.nio.fs.LinuxWatchService.poll(Native Method) [email protected]/sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:316) [email protected]/java.lang.Thread.run(Thread.java:834)

It seems that ./sources/maverick-base/src/main/java/com/sshtools/common/logger/DefaultLoggerContext.java starts a FileWatcher extends Thread thread, that itself starts a WatchService which starts actually another underlying thread. There seems no API to stop them. Maybe a simple:

public synchronized void shutdown() { watcher.stopThread(); }

would suffice for the created thread, and improve stopThread to call close on the underlying watcher WatchService object (maybe using two different local names not watcher twice for different objects would also be better).

What is the best way to shutdown the client library when used in a web application ?
Will you be able to provide an API for that ?
Some threads are left running when the web application is redeployed in a container like tomcat.

I had to do the following

SshEngine.getDefaultInstance().shutdownNow(true, 2000);
Log.getDefaultContext().shutdown(); //this can potentially throw a null pointer exception as the DefaultLoggerContext.watcher can be null

I still have the following thread running

Name: SSHD-ACCEPT-1
State: RUNNABLE
Total blocked: 0 Total waited: 0
Stack trace:
java.base@15/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
java.base@15/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:357)
java.base@15/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:182)
java.base@15/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)

• locked sun.nio.ch.Util$2@7ee85529
• locked sun.nio.ch.WindowsSelectorImpl@66affdba
  java.base@15/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  com.sshtools.synergy.nio.SelectorThread.run(SelectorThread.java:415)

Greetings again!

I am currently running into an issue where I'd like to get information about a file or set of files where the file path has a wildcard. For example I tried the following:

SftpFileAttributes sftpFileAttribute = stat("/home/user/MyFile_2022_06_01*.csv");

In the directory I have a file

/home/user/MyFile_2022_06_01_Daily.csv

I expect that file to be returned in the attributes but I am getting no file found.

Is there an alternative way to do this or am I doing something wrong with the above? Im aware that stat returns attributes of a file or a directory, but I'd like to get something working using the wildcard.

As always, any information is greatly appreciated!

EDIT

I wrote a work around, but I wonder if there's a more efficient way to do this:

if (fileName.contains("*")) {
    List < SftpFile > filesToDownload = new ArrayList < > ();
    Pattern filePattern = Pattern.compile(StringUtils.replace(fileName, "*", ".*"));
    SftpFile[] filesFound = ls(folderPath);
    for (SftpFile sftpFile: filesFound) {
        System.out.println("File found: " + sftpFile.getAbsolutePath());
        Matcher matcher = filePattern.matcher(sftpFile.getFilename());
        if (matcher.matches()) {
            filesToDownload.add(sftpFile);
        }
    }
    System.out.println("Files found: " + filesToDownload.size());
}

I've looked all over for javadocs, and I can't find any. Do they exist?

Also, the README has a link to community forums, but I get a 503 error when I hit the link. Are the forums going to be back?

I am experiencing an issue where listing the contents of the directory is producing multiple of the same folder. I tracked the problem to line 94 of the VirtualMappedFile class. The getChildren method produces duplicate folders when there are mounts to subfolders. For example, I have mounts defined at these folders:

1. /
2. /folder_a
3. /folder_b
4. /folder_b/subfolder_a
5. /folder_b/subfolder_b

When running getChildren from "/", the result shows 4 files:
folder_a
folder_b
folder_b
folder_b

I would expect the result to contain just folder_a and folder_b.

The getChildren method in VirtualMountFile has similar code producing the duplicately named files as well starting on line 99.

When connecting to SFTP using the OpenSSH command line client, the client hangs indefinitely after issuing the "exit" command. This affects many production scripts that automate file transfers.

Jeffs-MacBook-Pro:myfiles jthorn$ ssh -V
OpenSSH_7.9p1, LibreSSL 2.7.3
Jeffs-MacBook-Pro:myfiles jthorn$ sftp [email protected]
Connected to [email protected].
sftp> exit

The client never returns after the exit command.

The static executor field on SshContext can be shutdown and then reused by another SshClient, resulting in an exception when trying to connect to the server:

java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1e3a0815[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@2fdf07c1[Wrapped task = com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask@6049443b]] rejected from java.util.concurrent.ThreadPoolExecutor@5785f3a0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2965]

As seen in the exception, the ThreadPoolExecutor has a Terminated status.

As a workaround, I am using reflection to set the executor to null so a new one is created when necessary. I think the SshContext.shutdown function should explicitly set the executor to null so the terminated executor is not reused.

When running a protocol fuzzer against the maverick library, we found an issue where the server will keep a connection open indefinitely during the DH group exchange negotiation process. An attacker could use this behavior to exhaust the server's connections and deny service to users. This file has the trace log files for the connection that show the server is waiting in a state of 'Transport protocol is expecting another packet':
2021-06-09-16-03-58-143__1202fbee-d9aa-4a20-86fd-9c98ae9c2267_.log

The code that disregards the fuzzed packet and decides to wait forever seems to be here:

Ideally, the connection would close after a configurable amount of time when it is left in this state. If that is not feasible, can you think of a way my application code can monitor this process to shut down connection when appropriate?

Thank you in advance.

I have an interesting issue uploading and downloading data to an Axway secure transport server
I created a small project isolating the issue. When running the test n times it usually starts failing after 2 to 3 iterations with Authentication Failed Error
Interestingly, if I configure maverick to log in DEBUG mode, the error is gone and all connections are successful.
Using Loglevel TRACE I was able to reproduce the error a few times getting some logging information. Attached is the log from a run with a successful upload and a Authentication Failed error while trying to download the file.

synergy.log

The error happens after waiting for 30 seconds at 08:30:38

14 Dez. 2023 08:30:38,104 [     pool-3-thread-1]  TRACE - transport-protocol: No more tasks, will wait for a few more seconds before completing task
14 Dez. 2023 08:30:38,105 [     pool-3-thread-1]  TRACE - {}: Operation task has ended
14 Dez. 2023 08:30:38,105 [     SSHD-TRANSFER-1]  TRACE - transport-client has state ops=1 READ
14 Dez. 2023 08:31:08,111 [                main]  DEBUG - Authentication failed
14 Dez. 2023 08:31:08,111 [                main]   INFO - Disconnect host/ip:8022 By Application
14 Dez. 2023 08:31:08,111 [                main]  DEBUG - Posting message com.sshtools.synergy.ssh.TransportProtocol$DisconnectMessage to queue

I'm using Maverick in the version 3.0.11
The Axway host software has version: 5.5-20231130 (3201)

  static void uploadFile(String source, String fileName, String destination) throws Exception {
    Logger.info("Creating SSH Client... Connecting to host");

    try (SshClient ssh = new SshClient("host", 8022, "user", "pass")) {
      Logger.info("Creating SFTP Client...");
      SftpClient client = new SftpClient(ssh);
      client.cd(destination);
      client.put(source, fileName);
      client.quit();
      ssh.disconnect();
      await().forever().until(() -> !ssh.isConnected());

    } catch (PermissionDeniedException
        | SshException
        | IOException
        | TransferCancelledException
        | SftpStatusException e) {

      Logger.error(e.getMessage());
      throw new Exception();
    }
  }

  static void downloadFile(String source, String fileName, String destination) throws Exception {
    Logger.info("Creating SSH Client... Connecting to host");

    try (SshClient ssh = new SshClient("host", 8022, "user", "pass")) {
      Logger.info("Creating SFTP Client...");
      SftpClient client = new SftpClient(ssh);
      client.cd(source);
      client.get(fileName, destination);
      client.rm(fileName);
      client.quit();
      ssh.disconnect();
      Logger.info("Disconnecting ssh client");
      await().forever().until(() -> !ssh.isConnected());

    } catch (PermissionDeniedException
        | SshException
        | IOException
        | TransferCancelledException
        | SftpStatusException e) {

      Logger.error(e.getMessage());
      throw new Exception();
    }
  }

Hi guys, I'm trying to ssh into a cluster node by using sshclient
Ex: SshClient ssh = new SshClient(hostname, port, username, password.toCharArray()

Exception:
java.io.IOException: Failed to negotiate a transport component from {} and {}
java.io.IOException: Failed to connect to :22

and unable to connect to the cluster.
I'm using maverick jar version 3.6 -maverick-synergy-client-3.0.6-all.jar
Kindly refer the screenshot attached. Can someone help what needs to be done?

Thanks and Regards,
Radhesh

Trying to unload a task on a remote macOS machine. The thread hangs forever on the shellprocess drain method.
Logs:
DEBUG - There are now 2 active connections on client connection manager
DEBUG - Firing EVENT_CONNECTED success=true
DEBUG - Sent local identification string SSH-2.0-MaverickSynergy
DEBUG - Remote client version OK
DEBUG - Firing EVENT_NEGOTIATED_PROTOCOL success=true
DEBUG - Posting SSH_MSG_KEX_INIT
DEBUG - Sent SSH_MSG_KEX_INIT
DEBUG - Received SSH_MSG_KEX_INIT
DEBUG - Remote Key Exchanges: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
DEBUG - Remote Public Keys: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
DEBUG - Remote Ciphers CS: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
DEBUG - Remote Ciphers SC: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
DEBUG - Remote Macs CS: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
DEBUG - Remote Macs SC: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
DEBUG - Remote Compression CS: none,[email protected]
DEBUG - Remote Compression SC: none,[email protected]
DEBUG - Lang:
DEBUG - First Packet Follows: false
DEBUG - Local Key Exchanges: diffie-hellman-group-exchange-sha256,[email protected],curve25519-sha256,diffie-hellman-group18-sha512,diffie-hellman-group17-sha512,diffie-hellman-group16-sha512,diffie-hellman-group15-sha512,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,rsa2048-sha256,diffie-hellman-group14-sha1
DEBUG - Local Public Keys: ecdsa-sha2-nistp256,x509v3-ecdsa-sha2-nistp384,x509v3-ecdsa-sha2-nistp256,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,x509v3-ecdsa-sha2-nistp521,rsa-sha2-512,rsa-sha2-256,[email protected],[email protected],[email protected],x509v3-sign-rsa-sha1,ssh-rsa,x509v3-sign-rsa,x509v3-ssh-rsa,x509v3-rsa2048-sha256,[email protected],x509v3-ssh-dss,x509v3-sign-dss
DEBUG - Local Ciphers CS: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr
DEBUG - Local Ciphers SC: aes256-ctr,[email protected],[email protected],[email protected],aes192-ctr,aes128-ctr,3des-ctr
DEBUG - Local Macs CS: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
DEBUG - Local Macs SC: [email protected],hmac-sha2-512,hmac-sha2-512,hmac-sha2-512,hmac-sha2-512-96,[email protected],hmac-sha2-256,[email protected],hmac-sha2-256,hmac-sha2-256-96,[email protected],hmac-sha1,hmac-sha1-96
DEBUG - Local Compression CS: none,zlib,[email protected]
DEBUG - Local Compression SC: none,zlib,[email protected]
DEBUG - Firing EVENT_KEY_EXCHANGE_INIT success=true
DEBUG - Negotiated Key Exchange: diffie-hellman-group-exchange-sha256
DEBUG - Negotiated Public Key: ecdsa-sha2-nistp256
DEBUG - Negotiated Cipher CS: aes256-ctr
DEBUG - Negotiated Cipher SC: aes256-ctr
DEBUG - Negotiated Mac CS: [email protected]
DEBUG - Negotiated Mac SC: [email protected]
DEBUG - Negotiated Compression CS: none
DEBUG - Negotiated Compression SC: none
DEBUG - Minimum DH prime size is 2048
DEBUG - Preferred DH prime size is 2048
DEBUG - Maximum DH prime size is 8192
DEBUG - Sent SSH_MSG_KEY_DH_GEX_REQUEST
DEBUG - Received 2048 bit DH prime with group 2
DEBUG - Sent SSH_MSG_KEXDH_INIT
DEBUG - Verifying server DH parameters
DEBUG - Verified DH parameters. Performing DH calculations
DEBUG - Verifying calculated DH parameters
DEBUG - Calculating exchange hash
DEBUG - Completed key exchange calculations
DEBUG - Received SSH_MSG_NEWKEYS
DEBUG - Sent SSH_MSG_NEWKEYS
DEBUG - Firing EVENT_HOSTKEY_ACCEPTED success=true
DEBUG - Firing EVENT_KEY_EXCHANGE_COMPLETE success=true
DEBUG - Sent SSH_MSG_SERVICE_REQUEST ssh-userauth
DEBUG - Starting Authentication Protocol
DEBUG - Starting none authentication
DEBUG - SSH_MSG_USERAUTH_FAILURE received auths=publickey,password,keyboard-interactive
DEBUG - none authentication failed
DEBUG - Authenticating with password
DEBUG - Adding password authentication
DEBUG - We prefer keyboard-interactive over password so injecting keyboard-interactive authenticator
DEBUG - Starting keyboard-interactive authentication
DEBUG - SSH_MSG_USERAUTH_INFO_REQUEST received
DEBUG - SSH_MSG_USERAUTH_INFO_RESPONSE sent
DEBUG - SSH_MSG_USERAUTH_INFO_REQUEST received
DEBUG - SSH_MSG_USERAUTH_INFO_RESPONSE sent
DEBUG - SSH_MSG_USERAUTH_SUCCESS received
DEBUG - Initialized MaxChannels=100
DEBUG - Stopping Authentication Protocol
DEBUG - keyboard-interactive authentication succeeded
DEBUG - Authentication succeeded
DEBUG - Sent SSH_MSG_CHANNEL_OPEN channel=0 channelType=session
DEBUG - Received SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=false
DEBUG - Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION channel=0 remote=0 remotepacket=32768 remotewindow=0
DEBUG - Starting session task
DEBUG - Creating session for interactive shell
DEBUG - Sent SSH_MSG_CHANNEL_REQUEST request=pty-req wantReply=true channel=0 remote=0 localWindow=1024000 remoteWindow=0
DEBUG - Sent SSH_MSG_CHANNEL_REQUEST request=shell wantReply=true channel=0 remote=0 localWindow=1024000 remoteWindow=0
DEBUG - Received SSH_MSG_CHANNEL_SUCCESS channel=0 remote=0 localWindow=1024000 remoteWindow=0
DEBUG - Received SSH_MSG_CHANNEL_WINDOW_ADJUST channel=0 remote=0 adjust=2097152
DEBUG - Received SSH_MSG_CHANNEL_SUCCESS channel=0 remote=0 localWindow=1024000 remoteWindow=2097152
DEBUG - Received SSH_MSG_CHANNEL_DATA len=57 channel=0 remote=0 localWindow=1023943 remoteWindow=2097152
DEBUG - Received SSH_MSG_CHANNEL_DATA len=183 channel=0 remote=0 localWindow=1023760 remoteWindow=2097152
DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023734 remoteWindow=2097152
DEBUG - Performing marker test: echo "---BEGIN---" ; echo $?
DEBUG - Sent SSH_MSG_CHANNEL_DATA seq=12 len=30 channel=0 remote=0 localWindow=1023734 remoteWindow=2097122
DEBUG - Session creation complete
DEBUG - Checking state of startup controller
DEBUG - Shell still in startup mode, draining startup output
DEBUG - Shell startup (read): Last login: Sat Jun 24 11:00:53 2023 from 192.168.1.18

DEBUG - Shell startup (read):
DEBUG - Shell startup (read): The default interactive shell is now zsh.
DEBUG - Shell startup (read): To update your account to use zsh, please run chsh -s /bin/zsh.
DEBUG - Shell startup (read): For more details, please visit https://support.apple.com/kb/HT208050.
DEBUG - Received SSH_MSG_CHANNEL_DATA len=10 channel=0 remote=0 localWindow=1023724 remoteWindow=2097122
DEBUG - Received SSH_MSG_CHANNEL_DATA len=6 channel=0 remote=0 localWindow=1023718 remoteWindow=2097122
DEBUG - Received SSH_MSG_CHANNEL_DATA len=8 channel=0 remote=0 localWindow=1023710 remoteWindow=2097122
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023706 remoteWindow=2097122
DEBUG - Potentially found test marker [<machine_name>:~ akshay$ echo "---BEGIN---]
DEBUG - Detected echo of test marker command since we did not find LF at end of marker ch=34 currentLine=<machine_name>:~ akshay$ echo "---BEGIN---
DEBUG - Received SSH_MSG_CHANNEL_DATA len=2 channel=0 remote=0 localWindow=1023704 remoteWindow=2097122
DEBUG - Received SSH_MSG_CHANNEL_DATA len=16 channel=0 remote=0 localWindow=1023688 remoteWindow=2097122
DEBUG - Shell startup (read): <machine_name>:~ akshay$ echo "---BEGIN---" ; echo $?
DEBUG - Potentially found test marker [---BEGIN---]
DEBUG - Looking good, found CR
DEBUG - Found test marker
DEBUG - Detecting shell settings
DEBUG - Shell startup (detect): 0
DEBUG - This looks like a *nix type machine, setting EOL to CR only and exit code variable to $?
DEBUG - Checking state of startup controller
DEBUG - Shell is ready for command
DEBUG - Executing command: uname
DEBUG - Executing raw command: echo "---BEGIN---"; uname; echo "---END---;PROCESS=1687584933178;EXITCODE=$?"

DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023662 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=2 channel=0 remote=0 localWindow=1023660 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023634 remoteWindow=2097044
DEBUG - Sent SSH_MSG_CHANNEL_DATA seq=13 len=78 channel=0 remote=0 localWindow=1023634 remoteWindow=2097044
DEBUG - uname: Expecting begin marker
DEBUG - Received SSH_MSG_CHANNEL_DATA len=9 channel=0 remote=0 localWindow=1023625 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=5 channel=0 remote=0 localWindow=1023620 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=7 channel=0 remote=0 localWindow=1023613 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023609 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023606 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023603 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023599 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023596 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023592 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023589 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023585 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023581 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023578 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023575 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023572 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023568 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023565 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023561 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023558 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023555 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=13 channel=0 remote=0 localWindow=1023542 remoteWindow=2097044
DEBUG - Received SSH_MSG_CHANNEL_DATA len=8 channel=0 remote=0 localWindow=1023534 remoteWindow=2097044
DEBUG - uname: Found begin marker
DEBUG - Received SSH_MSG_CHANNEL_DATA len=44 channel=0 remote=0 localWindow=1023490 remoteWindow=2097044
DEBUG - uname: ---END---;PROCESS=1687584933178;EXITCODE=
DEBUG - uname: Found end marker
DEBUG - uname: Looking for exit code
DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023464 remoteWindow=2097044
DEBUG - uname: Exit code is 0
DEBUG - Remote side reported it is Darwin
DEBUG - Setting default sudo prompt
DEBUG - Checking state of startup controller
DEBUG - Shell is ready for command
DEBUG - Executing command: export SUDO_PROMPT=Password:
DEBUG - Executing raw command: echo "---BEGIN---"; export SUDO_PROMPT=Password:; echo "---END---;PROCESS=1687584933190;EXITCODE=$?"

DEBUG - Sent SSH_MSG_CHANNEL_DATA seq=14 len=101 channel=0 remote=0 localWindow=1023464 remoteWindow=2096943
DEBUG - export SUDO_PROMPT=Password:: Expecting begin marker
DEBUG - Received SSH_MSG_CHANNEL_DATA len=7 channel=0 remote=0 localWindow=1023457 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=5 channel=0 remote=0 localWindow=1023452 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=7 channel=0 remote=0 localWindow=1023445 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023441 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023437 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023434 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023431 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023428 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023425 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023422 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023418 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023415 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023412 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023408 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=9 channel=0 remote=0 localWindow=1023399 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=11 channel=0 remote=0 localWindow=1023388 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023384 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023381 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023378 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023375 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023372 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023369 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023366 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023362 remoteWindow=2096943
DEBUG - Received SSH_MSG_CHANNEL_DATA len=57 channel=0 remote=0 localWindow=1023305 remoteWindow=2096943
DEBUG - export SUDO_PROMPT=Password:: Found begin marker
DEBUG - export SUDO_PROMPT=Password:: ---END---;PROCESS=1687584933190;EXITCODE=
DEBUG - export SUDO_PROMPT=Password:: Found end marker
DEBUG - export SUDO_PROMPT=Password:: Looking for exit code
DEBUG - export SUDO_PROMPT=Password:: Exit code is 0
DEBUG - Shell initialized
DEBUG - Shell is ready for command
DEBUG - Executing command: sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>
DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023279 remoteWindow=2096943
DEBUG - Executing raw command: echo "---BEGIN---"; sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>; echo "---END---;PROCESS=1687584933197;EXITCODE=$?"

DEBUG - Sent SSH_MSG_CHANNEL_DATA seq=15 len=173 channel=0 remote=0 localWindow=1023279 remoteWindow=2096770
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: Expecting begin marker
DEBUG - Received SSH_MSG_CHANNEL_DATA len=9 channel=0 remote=0 localWindow=1023270 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=5 channel=0 remote=0 localWindow=1023265 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=7 channel=0 remote=0 localWindow=1023258 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023255 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023251 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023248 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023244 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023241 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023238 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023235 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023231 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023228 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023225 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=2 channel=0 remote=0 localWindow=1023223 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023219 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023216 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023212 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023209 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023206 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023202 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023199 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023195 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023191 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=8 channel=0 remote=0 localWindow=1023183 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=6 channel=0 remote=0 localWindow=1023177 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023174 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023171 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023167 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023164 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023161 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023158 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023155 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023151 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023148 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023145 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023142 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023139 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023136 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023133 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023130 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023127 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023124 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=4 channel=0 remote=0 localWindow=1023120 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023117 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023114 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023111 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023108 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=3 channel=0 remote=0 localWindow=1023105 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=13 channel=0 remote=0 localWindow=1023092 remoteWindow=2096770
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: Found begin marker
DEBUG - Received SSH_MSG_CHANNEL_DATA len=44 channel=0 remote=0 localWindow=1023048 remoteWindow=2096770
DEBUG - Received SSH_MSG_CHANNEL_DATA len=26 channel=0 remote=0 localWindow=1023022 remoteWindow=2096770
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: ---END---;PROCESS=1687584933197;EXITCODE=
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: Found end marker
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: Looking for exit code
DEBUG - sudo launchctl unload -wF /Library/LaunchDaemons/<plist_file_path>: Exit code is 0
DEBUG - sudo password expression not matched
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]
DEBUG - There are 1 channels currently open
DEBUG - Sending SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Sent SSH_MSG_GLOBAL_REQUEST request=[email protected] wantReply=true
DEBUG - Received SSH_MSG_GLOBAL_REQUEST_FAILURE for [email protected]

Thread Dump
at java.lang.Object.wait(Native Method)
at com.sshtools.synergy.ssh.CachingDataWindow.waitFor(CachingDataWindow.java:159)
at com.sshtools.synergy.ssh.ChannelNG$ChannelInputStream.read(ChannelNG.java:1365)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at com.sshtools.client.shell.ShellInputStream.read(ShellInputStream.java:145)
at com.sshtools.client.shell.ShellProcess.drain(ShellProcess.java:82)

I am trying to test connecting various private key formats to the server and I am running into an occasional issue that says that publickey authentication failed.

I've dug into the code and have found that this is caused by the server sending a response to an attempt at "none" authentication, but the client code thinks the failure message is from the publickey authenticator.

I have traced this to a race condition in the SshClient constructor. This only happens occasionally but the pattern for the failed cases is:

1. none authentication is attempted via the AuthenticationProtocolClient class' start() method.
2. public key authentication is attempted, which sets the currentAuthenticator in AuthenticationProtocolClient to the public key authenticator
3. A SSH_MSG_USERAUTH_FAILURE message is received from the server for the none authentication and processed in AuthenticationProtocolClient.processMessage. Since the "currentAuthenticator" is set to the public key authenticator, the client reports the failure incorrectly on the publickey.

I have been trying to find a way to configure the client to not send the NoneAuthentication but I think it is not possible.

Hi. We're looking at replacing our current, mature SSH implementation based on Jsch with Maverick Synergy, and so far, it looks very promising. (Jsch is lacking in recent cipher support).

I look forward to integrating with Maverick Synergy, and appreciate all of the hard work that's obviously gone into it.

That said, I ran into a few issues trying to build from source, the initial objective being to get complete Javadocs for the API, since they don't seem to be online. Admittedly, I'm a Maven novice (we mostly use Ant), so some of this may just be my learning curve - take this with that grain of salt.

1. In the main pom.xml, Maven (3.6.0) now seems to want the URL in the scm element to be prefixed by "scm:git:...", e.g.:

    <scm>
            <url>https://github.com/sshtools/maverick-synergy/</url>
            <connection>scm:git:https://github.com/sshtools/maverick-synergy.git</connection>
    </scm>
   

2. It seems the Java 8 compiler must be used, not just Java 8 compatibility. Post-Java 8, Thread.destroy() has been removed from java.lang.Thread. Because Maven wants to bring in Java 11 on Ubuntu 18.04, I think the Java 11 compiler had issues with use of Thread.destroy(). The fix was to force the Java 8 compiler to be used instead of Java 11 in pom.xml:

                    <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-compiler-plugin</artifactId>
                            <version>3.1</version>
                            <configuration>
                                    <source>1.8</source>
                                    <target>1.8</target>
                                    <executable>/usr/lib/jvm/java-8-openjdk-amd64/bin/javac</executable>
                                    <compilerVersion>1.8</compilerVersion>
                                    <fork>true</fork>
                            </configuration>
                    </plugin>
   

...though I think this can be avoided by setting JAVA_HOME to point to the Java 8 JDK. To be clear, I didn't want Java 11, but Maven did (via apt).

3. In maverick-synergy-fuse-drive/pom.xml, the repository URL has to be https://... these days, apparently, not http://...

    <repositories>
            <repository>
                    <id>central</id>
                    <name>bintray</name>
                    <url>https://jcenter.bintray.com</url>
            </repository>
    </repositories>
   

...symptom was a 403/Forbidden when trying to download dependencies.

4. Some tests would fail, so I had to specify -DskipTests. Perhaps some other set up is needed in order to run tests? It looks like some paths may be hard-coded in some of the tests (/home/user/test/...).

5. I also had to disable jar signing (more set up needed?) and license checking (-Dlicense.skip=true).

6. It would be nice to have aggregated Javadocs. As mentioned, my goal was to get a complete set of Javadocs (aggregated). This proved to be difficult, as pom.xml isn't really set up to support the site lifecycle needed for this. I was finally able to get what I wanted, more or less, by hacking-in maven-site-plugin, though that wasn't 100% successful - I had to do "mvn javadoc:aggregate-jar" to get Javadocs aggregated. It would be nice to have complete Javadocs if needed, vs. having to dig around individual projects. To be clear, Javadocs were generated for all projects - just not aggregated to the site target.

Anyway, thanks again for all the hard work. Hope this helps the next person getting started from scratch.

Unable to generate Certificate using rsa-sha2-512 algorithm. Tried debugging it in latest version1.7.52. There no support for rsa-sha2-512 version in OpenSshCertificate implementations.

SshKeyPair pair = SshKeyUtils.makeRSAWithSHA256Signature(SshKeyPairGenerator.generateKeyPair(SshKeyPairGenerator.SSH2_RSA));
SshKeyPair ca = SshKeyUtils.makeRSAWithSHA256Signature(SshKeyPairGenerator.generateKeyPair(SshKeyPairGenerator.SSH2_RSA));
SshCertificate cert = SshCertificateAuthority.generateCertificate(pair, 0L,OpenSshCertificate.SSH_CERT_TYPE_USER, "KEY-IDENTITY","john", 365,new CertificateExtension.Builder().defaultExtensions().build(), ca);
SshKeyUtils.savePrivateKey(pair, "", "User SHA2", new File("test"));
SshKeyUtils.saveCertificate(cert, "", "Cert SHA2", new File("test"));

When tried this, getting below exception.
Unsupported certificate type rsa-sha2-256 generated an error at com.sshtools.publickey.SshCertificateAuthority.generateCertificate(SshCertificateAuthority.java:137)

I know, it's quite old, but sadly still widely used :-(

I'm trying to upgrade from 3.0.9 to 3.0.10, but it doesn't seem to be compatible to java 8 anymore. I don't know, if this is intended?!

The problem has to do with the flip-method of the java.nio.ByteBuffer class.

In java-versions <= 1.8 ByteBuffer extends java.nio.Buffer and therefor inhertis the flip method with following signature:

public final Buffer flip()

the method is final.
In java > 1.8 the method is not final anymore and it's overriden in ByteBuffer changing the signature to

public ByteBuffer flip()

so returning a more specific type.
Although your SocketConnection-Class doesn't make use of the returned value, it leads to problems at runtime:

java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer; at com.sshtools.synergy.nio.SocketConnection.processWriteEvent(SocketConnection.java:358) at com.sshtools.synergy.nio.SshEngine$SocketReadWriteTask.doTask(SshEngine.java:979) at com.sshtools.common.ssh.ConnectionAwareTask.run(ConnectionAwareTask.java:45) at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.executeAllTasks(ExecutorOperationSupport.java:127) at com.sshtools.common.ssh.ExecutorOperationSupport$OperationTask.run(ExecutorOperationSupport.java:84) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Is it possible to make it usable with java 8 anymore? That would be nice!

Thanks for your efford
and greetz,

Karsten

Hi,

When trying to configure the virtual file system on the server from the docs here:
https://www.jadaptive.com/app/manpage/en/article/1587734

server.setFileFactory(new VirtualFileFactory(new VFSFileFactory()));

When using the latest 3.0.0-SNAPSHOT version a FileFactory is expected instead of an AbstractFileFactory. What would be the correct way to configure the server with a virtualFileFactory?

Thanks

Tim

Hi ya,
The ssh client command throws an IllegalArgumentException due to the following line

The orElse is evaluated before returning the connection, which in turn gets evaluated to the orElseThrow since clientSupplier is empty
It needs to be converted to an orElseGet like

super(builder.connection.orElseGet(() -> builder.clientSupplier.orElseThrow(() -> new IllegalArgumentException("No connection or client supplied.")).apply(0).getConnection()));

Greetings!

I have ran into a new problem and I tried searching the documentation but no avail. I am trying to use stat on a file that exists in the remote server however the file name itself has spaces. The file in question looks like this

/home/test/test/1.31.22 TEST File.csv

When I use the stat command I get SftpStatusException: No such file

When I log into the server manually I am able to see the file if I run the following command

ls "/home/test/test/1.31.22 TEST File.csv"

When logging into the remote server via maverick it informs me I am at the root when I use the pwd() command, however I don't think this should be an issue since I am passing in the full path when I do:

stat("/home/test/test/1.31.22 TEST File.csv")

I've tried wrapping both the file name and the whole file path in quotes but I still get the same error. I am running out of ideas as to what I can do to get the information for this file so that I can download it. The exception doesn't tell me much except the file is not found. Is there another function that I may have to use? Do I need to encode the String so that it allows me to use spaces? Any help would be greatly appreciated!

Thank you!

Hi,

Is it possible to add a custom listener that would be invoked when specific sftp operations occur for auditing purposes? For example; execute some custom method when file is put or deleted via sftp? I'm keen to migrate from apache mina that supports this feature via their 'ftplet' (https://mina.apache.org/ftpserver-project/ftplet.html)

Tim