abrupt in downloading using sftp_download about sftp HOT 5 CLOSED

Hi, the error message says that the connection failed with a Time out, meaning that the server quagmire.igb.illinois.edu did not respond soon enough following the request to connect. It is hard to tell what the reason for this might be. If the download of each file takes considerable time, such as several minutes, it could be that your login session expires. You may have to modify your code a bit, so that you loop over your list of files to download one file at a time, and re-create the sftp_con object in each iteration, i.e. immediately before downloading each file. Please try it.

from sftp.

Hi, Stenevang:

Really appreciated for your quick response. I tried both ways for iterate/loop: for loop and purrr::map, but regrettably not working:

For loop
` ## for loop

for (i in 1: nrow(files_list)){
sftp_con <- sftp_connect(server = "sftp://quagmire.igb.illinois.edu/",
folder = "home/shared/licor_data/maize/raw/",
username = Sys.getenv("MY_SFTP_USER"),
password = Sys.getenv("MY_SFTP_PASS"),
protocol = "sftp://",
port = 22)

files_list <-
sftp_list(sftp_connection = sftp_con,
verbose = TRUE,
curlPerformVerbose = FALSE,
type = "file") %>%
tail(100)

sftp_download(files_list$name[i], tofolder = "./content/post/eddy_covariance/data/energy_farm/maize/",
verbose = TRUE)
}`

SFTP url: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
8521 file(s) in SFTP folder.
SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: ./content/post/eddy_covariance/data/energy_farm/maize
1 file(s) to download.
2019-12-07T013000_AIU-0496_maize.ghg downloaded
1 file(s) downloaded.
SFTP url: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
8521 file(s) in SFTP folder.
SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: ./content/post/eddy_covariance/data/energy_farm/maize
1 file(s) to download.
Error in function (type, msg, asError = TRUE) :
Failed to connect to quagmire.igb.illinois.edu port 22: Timed out

purrr::map
purrr::map(files_list$name, sftp_download, tofolder = "./content/post/eddy_covariance/data/energy_farm/maize/", verbose = TRUE)

SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: ./content/post/eddy_covariance/data/energy_farm/maize
1 file(s) to download.
2019-12-07T013000_AIU-0496_maize.ghg downloaded
1 file(s) downloaded.
SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: ./content/post/eddy_covariance/data/energy_farm/maize
1 file(s) to download.
2019-12-07T020000_AIU-0496_maize.ghg downloaded
1 file(s) downloaded.
SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: ./content/post/eddy_covariance/data/energy_farm/maize
1 file(s) to download.
Error in function (type, msg, asError = TRUE) :
Failed to connect to quagmire.igb.illinois.edu port 22: Timed out

from sftp.

Hi, Stenevang:
I guess my issue might be similar to this stackoverflow question.

Would you mind take a looking?

Appreciated

from sftp.

Yes, it would be a good idea to expose the RCurl connecttimeout setting so that users can raise it when needed. I'll try to add that as soon as I have time.

from sftp.

connecttimeout setting added.

from sftp.