fracpete / rsync4j Goto Github PK
View Code? Open in Web Editor NEWSimple Java wrapper for rsync for Linux, OSX and Windows.
Home Page: https://fracpete.github.io/rsync4j/
License: GNU General Public License v3.0
Simple Java wrapper for rsync for Linux, OSX and Windows.
Home Page: https://fracpete.github.io/rsync4j/
License: GNU General Public License v3.0
RSYNC fails when run from the user home directory with an apostrophe in the user name. Something like O'Reilly. This is on windows. Workaround is to change RSYNC4J_HOME env.
OpenSSH 9.6 was released Dec 18th, 2023. Once cygwin makes new binaries available, upgrade bundled ones:
https://ucmirror.canterbury.ac.nz/cygwin/x86_64/release/openssh/
I'm trying to run rsync command with runtime in java.
In my A.class file
@autowired
B bclass;
Public void testFileMove()
{
String srcPath="src/main/resources/tmp/"
String destPath="src/main/resources/destTemp"
ReflectionUtils.setField(bclass, "tempSrc", srcPath) ;
ReflectionUtils.setField(bclass, "tempDest", destPath) ;
bclass.execute();
}
In my B.class
String tempSrc;
String tempDest;
Public void execute()
{
String runtimeCmd= "rsync -avR --remove-source-files "+tempSrc+"/ "+tempDest
Process process =Runtime.getRunyime.exec(runtimeCmd) ;
Int exitCode=process.waitFor() ;
Logger.Info(" Exit code is :"+exitCode) ;
}
I always get error code as 23 for above code.
When i give source and destination path as full path it works well.
Eg:tempSrc=/home/usr/desktop/temp
tempSrc=/home/usr/desktop/destTemp
I need the path to be in resources folder of project so that its not specific to system path. With resources folder it become generic to project resource folder and this helps my build in bamboo as well.
The following setup
String pathToMyOwnCert = "C:\Users\user\.ssh\idftpuser";
RSync rsync = new RSync()
.source("[email protected]:~/public_html/version/*")
.destination("./")
.archive(true)
.timeout(15)
.info("progress2");
rsync.rsh(Binaries.sshBinary() + " -i " + Binaries.convertPath(pathToMyOwnCert)
+ " -o StrictHostKeyChecking=no"
+ " -p " + 22);
returned
[ERR] Unexpected remote arg: [email protected]:~/public_html/version/*
rsync error: syntax or usage error (code 1)
rsync.rsh("ssh" + " -i " + Binaries.convertPath(pathToMyOwnCert)
;
returned the same
With Release v3.1.2-14 it works perfectly!
As far as I can see, the information on this page: https://fracpete.github.io/rsync4j/maven/ is not correct. https://mvnrepository.com/artifact/com.github.fracpete/rsync4j lists 3.1.2-8 as the latest release.
Can you please update the maven release? Can that be automated to ensure new releases in the maven repository?
When using --list-only
, a target is not required.
Hi.
The include + exclude option is not working. The exclude option must be the last one in arguments command line to work. See code below:
/usr/bin/rsync --verbose --archive --recursive --rsh=/usr/bin/ssh -i /Users/roberto/.ssh/id_rsa_sepel1_fumaca --timeout=300000 --compress --exclude=* --include=.jpeg --include=/ /Users/roberto/linux/ user@server:temp/roberto
if exclude is right after includes, it works.
Could you fixed it ?
--additional ...
option to supply any other option to the underlying rsync binary, multiple timesHi,
I have two Windows machines and I wanted to set-up rsync4j in that way that rsync will synchronize my files automatically.
Is there a way to make rsync work on two windows machines? From your documentation, it seems like it but on the other hand, rsync daemon is Linux based and I don't think it would work. Using ssh with keys will allow such connection?
Thanks in advance
Currently rsync 3.2.7 is out (Oct 20th, 2022). Once final version has been released and cygwin also made a release available, upgrade.
Hi,
unless I'm doing something wrong, java code like this:
RSync rsync = new RSync()
.source("c:\tmp\myfile.txt")
.destination("rsync://localhost:21234/someDir/")
.recursive(false);
does not run on windows machines anymore since version (I think) 3.2.3-10, because it seems that you changed the way paths are converted on windows. In my example above 'rsync://localhost:21234/...' is converted to 'rsync:/localhost..." (or '//localhost:2134...' is converted to 'localhost:2134...') and then the rsync command fails with exit code 12.
Is there some way around this? I need to transparently use the tool on linux and windows machines, so I do not want to adjust paths myself (so something like 'C:/tmp/' should still work on windows machines, but the tool should recognize if you pass something like 'rsync://...'). This includes correctly setting permissions on windows machines (which is another issue, I guess)
Kind regards
upgrade OpenSSH Windows binaries to 9.1
I have the following issue:
I want to rsync files from the source having a certain pattern, let's say "*.inf" to my target directory.
So, the source is /home/sweet/home/s/*.inf
and the target is
/home/sweet/home/t
.
When I do this using rsync4j, it fails with exit code 23 and the following error message:
rsync: link_stat "/home/sweet/home/s/*.inf" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]
When I use the command line options built by rsync4j and execute them manually in a shell, the command runs fine, though:
/usr/bin/rsync --recursive --links --times --compress --itemize-changes /home/sweet/home/s/*.inf /home/sweet/home/t
Any hints?
Copying resource 'com/github/fracpete/rsync4j/windows-x86_64/cygcom_err-2.dll' to 'C:\Users\XXXXXXXXX\rsync4j\bin\cygcom_err-2.dll'
com.github.fracpete.rsync4j.Binaries : Copying failed!
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:159) ~[na:1.8.0_121]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[na:1.8.0_121]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[na:1.8.0_121]
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[na:1.8.0_121]
at java.io.FilterInputStream.read(FilterInputStream.java:107) ~[na:1.8.0_121]
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1719) ~[commons-io-2.3.jar!/:2.3]
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1696) ~[commons-io-2.3.jar!/:2.3]
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1671) ~[commons-io-2.3.jar!/:2.3]
at com.github.fracpete.rsync4j.Binaries.copyResourceTo(Binaries.java:114) [rsync4j-3.1.2-5.jar!/:na]
at com.github.fracpete.rsync4j.Binaries.extractBinaries(Binaries.java:173) [rsync4j-3.1.2-5.jar!/:na]
at com.github.fracpete.rsync4j.Binaries.rsyncBinary(Binaries.java:210) [rsync4j-3.1.2-5.jar!/:na]
at com.github.fracpete.rsync4j.RSync.commandLineArgs(RSync.java:1752) [rsync4j-3.1.2-5.jar!/:na]
Using Windows Server 2008 ,64 bit
Just like the tite .
Please help to answer
The following setup does not work on plain-protocol rsync servers :
RSync rsync = new RSync()
.recursive(true)
.times(true)
.dirs(true)
.verbose(true)
.rsh(null)
.passwordFile("C:\tmp\somepwfile")
.source("rsync://user@server/dir/dir1/")
.destination("c:\tmp\dir1")
....
It should be possible to disable rsh command.
..
.rsh(null)
..
leads to a NPE.
Possible solution :
I was testing the following code as provided in the documentation on windows machine.
Ssh ssh = new Ssh()
.outputCommandline(true)
.verbose(1)
.hostname("localhost")
.command("ll");
ConsoleOutputProcessOutput output = new ConsoleOutputProcessOutput();
output.monitor(ssh.builder());
This was the output
Sep 02, 2021 2:17:06 PM com.github.fracpete.rsync4j.core.AbstractBinary builder
INFO: Command-line: C:\Users\vaibhavp\rsync4j\bin\ssh.exe -v localhost ll
OpenSSH_8.7p1, OpenSSL 1.1.1f 31 Mar 2020
debug1: Connecting to localhost [::1] port 22.
debug1: connect to address ::1 port 22: Connection refused
debug1: Connecting to localhost [127.0.0.1] port 22.
debug1: connect to address 127.0.0.1 port 22: Connection refused
ssh: connect to host localhost port 22: Connection refused
Also, the given library can only execute the rsync command right?, If we need information about the files and folder it could only print on console the result of relevant commands for that?
Hi.
I'm getting error bellow while executing the command:
Command-line: /usr/bin/rsync --verbose --archive --recursive --rsh="ssh -i /Users/roberto/.ssh/id_rsa" --timeout=300000 --compress --exclude=*i.xml /Users/roberto/linux/aplic/ user@server:temp/roberto/config
ProcessadorSaidaRSync - [ERR] rsync: Failed to exec ssh -i /Users/roberto/.ssh/id_rsa_sepel1_fumaca: No such file or directory (2)
ProcessadorSaidaRSync - [ERR] rsync error: error in IPC code (code 14)
ProcessadorSaidaRSync - [ERR] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
ProcessadorSaidaRSync - [ERR] rsync error: error in rsync protocol data stream (code 12) at /BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-52/rsync/io.c(453) [sender=2.6.9]
I have ssh on $PATH.
Command works on terminal, but not when using rsync4j api.
Could you help me ?
ssh-keygen
I think it would be desirable to make rsync path customizable so by default on Linux/Mac it would use /usr/bin/rsync but would give user an option to override.
rsync process may consume too much time. In this case, it should be killed. I suggest you add this feature (you may use org.apache.commons.exec.ExecuteWatchdog)
We have a user whose username contains a space which results in the rsync command erroring with
rsync: [sender] Failed to exec C:\Users\Joe: No such file or directory (2)
rsync error: error in IPC code (code 14) at pipe.c(85) [sender=3.2.4dev]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.4dev]
It seems like you've had another issue with the single quotes, can something similar be implemented for spaces?
Concurrent transmission is necessary. Is it possible to add concurrent features?
rsync itself allows multiple source folders to be specified by simply listing them separated by spaces. If I do the same using rsync4j, it works on Windows when surrounding each directory with double quotes, e.g. like this:
RSync rsync = new RSync().source("\"/home/user/test1\" \"/home/user/test2\"");
However, this same approach does not work on Linux. I have tried with different variations, with or without quotes, different kinds of quotes, but it does not seem to be possible. For the above example, the error given by rsync is something along the lines of:
rsync: link_stat "/path/to/current/dir/"/home/user/test1" "/home/user/test2"" failed: No such file or directory (2)
Any idea how to get this scenario working with rsync4j?
Hi @fracpete
I have noticed that rsync4j is still depending on the 0.0.7 version of processoutput4j that is prior to the Apache license addition of this project. Do you think there would be any issue to update the pom reference to 0.0.11 of processoutput4j?
All the best
Upgrade cygwin's openssh to 8.0p1-2
And also update any other libraries
Update Windows binaries to OpenSSH 9.0:
https://www.openssh.com/releasenotes.html#9.0
Once cygwin has updated its package:
http://ucmirror.canterbury.ac.nz/cygwin/x86_64/release/openssh/
Hello,
I would like to use your library in my OSGi project in Apache Karaf. For it to work in my project i'll need to make it compatible by wrapping the library in a Karaf-bundle.
If you are interested in adding your library to this environment, respond to me, and i'll help you.
Hi - We are using this library to run some rsync operations and have used a output processing timeout to be 1800 secs(30 mins). Once in a while, it not working and the thread which runs the rsync process gets stuck at at wait() method here in monitor() (AbstractProcessOutput.java).
This thread got stuck for several days and was recovered by manual intervention. We are using rsync version rsync4j-all:3.2.3-10.A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.