stenevang / sftp Goto Github PK
View Code? Open in Web Editor NEWR-package with convenience functions for SFTP operations wrapping RCurl functions
License: GNU General Public License v3.0
R-package with convenience functions for SFTP operations wrapping RCurl functions
License: GNU General Public License v3.0
Thanks for the useful package! There's a small typo in the sftp_listdirs function causing an error. I corrected it in my environment and fixing the typo fixed the issue.
sft_connection should read sftp_connection below in your function.
sftp_listdirs <- function(sftp_connection = sftp_con,
verbose = TRUE,
curlPerformVerbose = FALSE,
recurse = FALSE) {
final <- sftp_list(sftp_connection = sft_connection, ...
Thanks!
Hello, wondering if there's any plans for this package to support R 4.2.1? Thanks!
Hi guys,
I have a list of about 600 files to transfer from multiple subfolders in an sftp server.
I can usually use sftp for this so far without issue. Today though I've been having failures to transfer with error code :
(I've obviously reducted connection info for security reasons sorry)
Error in function (type, msg, asError = TRUE) : Failed to connect to server_address_reducted port REDUCTED: Timed out
My code looks like this:
`sftp_con <- sftp_connect(server = "",
folder = "",
username = "",
password = "",
protocol = "sftp://" ,
port = "")
sftp_makedir("./____/Data/")
sftp_changedir("./ModelData/Data/")
done <- pblapply(split(upload_fileList, upload_fileList$SN, drop = T), function(filedata){
sftp_upload(file = filedata$filename, fromfolder = filedata$ParentFolder )
out <- 1
return(out)
} )
print(paste("Done"))`
Using the opportunity, I feel it'd be much faster to feed the function a list of strings the string being the complete filepath of the file like C:/myfile/myfile.csv
Any ideas how I can correct my timeout error?
Thanks in advance !
Hello!
Thank you for the excellent library.
Facing an issue when I am listing SFTP content. It seems to be happening when the last folder in the SFTP directory is empty.
Command:
sftp::sftp_list(
sftp_connection = sftp_con_i01_v01,
verbose = T,
curlPerformVerbose = T,
recurse = T
)
Error
> rlang::last_trace()
<error/vctrs_error_incompatible_type>
Can't combine `..1$type` <character> and `..2$type` <logical>.
Backtrace:
x
1. \-sftp::sftp_list(...)
2. \-sftp::sftp_list(sftp_connection = d_con, recurse = T)
3. +-dplyr::bind_rows(df2, d_list) %>% dplyr::arrange(name)
4. | \-base::eval(lhs, parent, parent)
5. | \-base::eval(lhs, parent, parent)
6. \-dplyr::bind_rows(df2, d_list)
7. \-vctrs::vec_rbind(!!!dots, .names_to = .id)
8. \-(function () ...
9. \-vctrs::vec_default_ptype2(...)
10. \-vctrs::stop_incompatible_type(...)
11. \-vctrs:::stop_incompatible(...)
12. \-vctrs:::stop_vctrs(...)
When using an absolute path starting from root on mac/linux (Ie one that starts with /
), the sftp_upload()
function immediately calls the internal function trim_slashes(fromfolder)
, which removes leading and trailing forward slashes.
This causes the upload to fail for paths starting from root, because the path without the leading slash assumes that it is starting from the current working directory.
When specifying a log file, a deprecation warning is issued:
Warning message:
The `path` argument of `write_lines()` is deprecated as of readr 1.4.0.
Please use the `file` argument instead.
Add new parameter "timeout" to the function sftp_connect().
The value of "timeout" will specify the time in seconds to set as timeout limit for RCurl when making connections in other functions of the sftp package. "timeout" will be another object in the list that is returned by sftp_connect(), so it can be referenced from the other functions and specified as an argument to the RCurl functions used.
RCurl function calls that need the connecttimeout option added:
RCurl::getBinaryURL
RCurl::getURL
RCurl::curlPerform
RCurl::ftpUpload
Any idea when you might support SSH keyfile implementations?
Trying to upload with sftp_upload()
.
Get the error:
Error in function (type, msg, asError = TRUE) :
Protocol "sftp" not supported or disabled in libcurl
System: OSX 12.1 (Monterey)
R: 4.1.2
sftp: 2.0.10
curl: 4.3.2
Is there a way to retry to download a file if sftp_download
fails the first time?
if !("sftp" %in% RCurl::curlVersion()$protocols) stop("Your systems curl installation does not have sftp support. Ask Google for the rest.")
Hi There,
I updated the sftp package and I am now getting the error message Error in function (type, msg, asError = TRUE)
when trying to list files recursively in a folder on and sFTP site. This was working properly before the update.
below is the code I am running that results in the error:
sftp_con <- sftp_connect(server = server,
username = username ,
password = password ,
folder = 'folder')
sftp_list(sftp_con, recurse = T, verbose = F, type = 'f')
If I try to list the files in a sub directory it does work:
sftp_con <- sftp_connect(server = server,
username = username ,
password = password ,
folder = 'folder/subfolder')
sftp_list(sftp_con, recurse = T, verbose = F, type = 'f')
Hi, this package seems like it fills a hole in the existing R ecosystem. Have you considered publishing it to CRAN?
Extending the API to support passing of custom RCurl::curlOptions
objects would be useful
Trying to upload, and I get the following (this is on a different system than my other issue):
Upload from folder: xxx
SFTP destination: xxx
2 file(s) to upload.
Error in function (type, msg, asError = TRUE) :
There is no other error info. I enabled logging, and nothing is in the log.
Same thing happens when trying sftp_list()
I'm having issues using the SFTP component of this package. Allow me to demonstrate.
If I enter my credential information:
sftp_con <- sftp_connect(server = "sftp.name.com",
username = "username",
password = "password",
protocol = "sftp://",
port = 2222)
sftp_download("*")
I am met immediately with the following error message:
Error in function (type, msg, asError = TRUE) :
Protocol "sftp" not supported or disabled in libcurl
Is there a way around this? If it helps, I'm using a MacBook to run this.
Hi there,
I'm not that experienced using this package so I want to apologize if this is a somewhat stupid question.
I want to download some filed from an sftp folder that has a whitespace in its folder name. This code does work:
sftp_con <- sftp_connect(
server = "some.ftp.com",
folder = "persons/alice",
username = user,
password = password
)
However, this does not:
sftp_con <- sftp_connect(
server = "some.ftp.com",
folder = "persons/bob and mallory ", <- Target folder contains whitespaces
username = user,
password = password
)
I already tried to put the folder in additional quotation. Does someone know how to handle this?
Thank you in advance.
Hi, Stenevang:
First, thanks for creating this powerful and easy-to-use package. I like sftp a lot. By the way, I am Sam from University of Illinois.
My question is that when I set up sftp_download to download files from a SFTP server, it abruptly stopped after successfully downloaded first 2-3 files, as shown below:
sftp_con <- sftp_connect(server = "sftp://quagmire.igb.illinois.edu/",
folder = "home/shared/licor_data/maize/raw/",
username = Sys.getenv("MY_SFTP_USER"),
password = Sys.getenv("MY_SFTP_PASS"),
protocol = "sftp://",
port = 22)
files_list <-
sftp_list(sftp_connection = sftp_con,
verbose = TRUE,
curlPerformVerbose = FALSE,
type = "file")
sftp_download(files_list$name, tofolder = "content/post/eddy_covariance/data/energy_farm/maize")
and here is the error:
SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: content/post/eddy_covariance/data/energy_farm/maize
100 file(s) to download.
2019-12-07T013000_AIU-0496_maize.ghg downloaded
2019-12-07T020000_AIU-0496_maize.ghg downloaded
Error in function (type, msg, asError = TRUE) :
Failed to connect to quagmire.igb.illinois.edu port 22: Timed out
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.