ida-scripts's Introduction

This repository was made for an older version of IDA which is no longer in use

iput_wrapper

Wrapper for uploading files using iput for iRODS 4.X.X clients.

Handles logging, checksum/size checking, iput 'large file retries', wildcards, deep directory creation, skipping existing files (when checksums and sizes match), skipping existing directories, etc.

The purpose is to:

Imitate irsync, which is buggy and missing most of the features of iput
Create detailed log entries of transfers (creation of directories,
individual file transfer errors, checksum comparisons)
Force users to use best practices when tranferring files (avoid using
recursive transfers, compare checksums, etc.)
Help in transferring large directories
Provide iRODS operators better logging for debugging user file
transfers assuming users remember/are able to use the wrappper and
are able to provide the transfer logs

Requirements

iRODS 4.X.X iCommands CLI
bash 4.2

Usage

Set the execute bit for the script

$ chmod 755 iput_wrapper.bash

Execute the script without parameters to get the help text

$ ./iput_wrapper.bash

License

MIT

Author Information

Taneli Riitaoja 2015-2016

ida-scripts's People

Contributors

Stargazers

Watchers

ida-scripts's Issues

Trouble transfering a large file to IDA

Trying to transfer a ~500Gt file to IDA using iput_wrapper.bash. When the transfer, after a long time, reaches 100%, this happens:

PIP004b_from_ext_hd.tgz - 505780.424/505817.798 MB - 99.99% done   2016-09-03.02:56:51
PIP004b_from_ext_hd.tgz - 505812.423/505817.798 MB - 100.00% done   2016-09-03.02:56:55
PIP004b_from_ext_hd.tgz - 505817.798/505817.798 MB - 100.00% done   2016-09-03.02:56:55
ERROR: putUtil: put error for /ida/hy/uhradar/GPM/PIP/PIP004b_from_ext_hd.tgz, status = -806000 status = -806000 CAT_SQL_ERR
0/1 -  0.00% of files done   0.000/505817.798 MB -  0.00% of file sizes done
Processing PIP004b_from_ext_hd.tgz - 505817.798 MB   2016-09-03.02:56:59
PIP004b_from_ext_hd.tgz - 31.999/505817.798 MB -  0.01% done   2016-09-03.02:57:08
PIP004b_from_ext_hd.tgz - 63.999/505817.798 MB -  0.01% done   2016-09-03.02:57:22
PIP004b_from_ext_hd.tgz - 95.998/505817.798 MB -  0.02% done   2016-09-03.02:57:25

and after a while:

PIP004b_from_ext_hd.tgz - 1695.961/505817.798 MB -  0.34% done   2016-09-03.03:00:17
PIP004b_from_ext_hd.tgz - 1727.960/505817.798 MB -  0.34% done   2016-09-03.03:00:20
PIP004b_from_ext_hd.tgz - 1759.960/505817.798 MB -  0.35% done   2016-09-03.03:00:24
/home/jussitii/ida-scripts/iput_wrapper.bash: rivi 192:  3246 Muistialueen ylitys     iput $iput_o "${local_full_source_path}" "${irods_target_path}"
FAIL: /ida/hy/uhradar/GPM/PIP/PIP004b_from_ext_hd.tgz ()
END TRANSFER (1/1): la 3.9.2016 03.00.25 +0300 - PIP004b_from_ext_hd.tgz
-------------TOTAL TRANSFER STATS-------------
Successfully transferred 0/1 files
Skipped 0/1 files (checksums match, no need to transfer)
Failed to transfer 1/1 files

I tried it twice, and both times the result was like this.

Recommend Projects