Code Monkey home page Code Monkey logo

ltopers's Introduction

LTOpers Documentation

This documentation is up to date as of LTOpers version 2019-01-18.

LTOpers elements

General LTO-handling tools

  • formatlto
  • mountlto
  • readlto
  • renameschemas
  • verifylto
  • writelto

Database-specific tools

  • ingestcollectionchecksum
  • ingestschemas
  • searchlto

CUNY-specific tool

  • collectionchecksum
  • indexschemas

Installing LTOpers

You can easily install these tools using Homebrew. To install Homebrew, follow the instructions.

Install the LTFS utilities. Please use version 2.4 on the Quantum site.

To install the general LTO-handling tools, run the following two commands:

brew tap amiaopensource/amiaos
brew install ltopers

To install the general LTO-handling and the database-specific tools, run the following two commands:

brew tap amiaopensource/amiaos
brew install ltopers --with-db

To install the general LTO-handling, the database-specific and the CUNY-specific tools, run the following two commands:

brew tap amiaopensource/amiaos
brew install ltopers --with-cuny

To update LTOpers tools, run the following commands:

brew update
brew upgrade

Note

This will update Homebrew and upgrade every package you've installed using it.

By default, LTOpers scripts use the standard tape serials, consisting of 6 character strings of numbers and capital letters. (However, the flag -x allows to use the barcode label as tape serial for non-standard implementations.)

Formatting LTO

LTO tapes need to be formatted before they can be used. Run the following command:

formatlto

The script will ask you select a deck (if you have more than one attached) and to provide a tape serial your tape. The tape serial format is exactly 6 capital letters and/or numbers.

Example:

  • Annnnn

where nnnnn is a 5-digit sequential number, e.g. A00001, A00002, etc.

The script will format your tape in LTFS format, with no compression, and sets rules that allow smaller types of files to live in the index partition of the tape, which gives you faster access. If you need to change this, edit the mkltfs line in the script to suit your preferences.

Remember to physically label the tape with the tape serial!

Mounting LTO

Put the tape you want to mount in the deck (you do not have to push it in all the way).

To mount the tape run the following command:

mountlto

The script will check for attached tape drive, and will prompt for deck name if there are multiple drives.

The tape will load in deck and mount.

In Finder, go to Go > Go to Folder, and enter /Volumes/$tape_serial to navigate to tape in Finder.

Un-mounting LTO and creating XML

To unmount the tape, Ctrl-C in Terminal.

Whenever a tape is ejected, the script will export the tape’s index (as an XML .schema file) to the path set as LTO_INDEX_DIR in mmconfig. To set or change the path, type mmconfig to access the configuration GUI. If there is no path set, the index location will default to $HOME/Documents/lto_indexes. If a .schema file for that tape already exists in that location, it will just update the existing schema file with any new data.

Writing to LTO

To write data onto a tape, run the following command:

writelto

When prompted, enter the tape serial and the source directory.

The script will run gcp to transfer the data to the LTO tape (rsync is used as a backup check for gcp).

To read back and create checksums for the contents of a tape, and write checksums to a file named with the tape serial and date, run the following command:

writelto -v

The checksum file will be written to the LTO logs directory: LTO_INDEX_DIR (also set in mmconfig) or $HOME/Documents/lto_indexes (default if not set in mmconfig).

This command uses the tool md5deep, which has several flags to set options. To customize which flags you want to append to md5deep, set the variable LTO_MD5_FLAGS in mmconfig (type mmconfig to access the configuration GUI).

Database integration

LTOpers can integrate with the database reporting functions in mediamicroservices. For information on how to set up the database see the mediamicroservices' readme.

If the database is activated and configured LTOpers will store information from LTO schema files, fixity information and PREMIS event information relating to LTOpers and microservices.

Database-specific tools in LTOpers are searchlto, ingestschemas and ingestcollectionchecksum. Both ingestchemas and ingestcollectionchecksum are designed for importing already existing information to the database (LTO schema information and read back checksums respectively). They can be run either on individual files or on directories containing multiple files.

searchlto is for performing searches against the LTO schema information stored in the database. It accepts either a media ID or an LTO tape serial as an input and will return a list of matching service files. Options are -h display help, -c compare characteristics of an input file to records in the database, and -f show full results (this will show all matches to search term as opposed filtering for service files only).

License

LTOpers is licensed under an MIT License.

ltopers's People

Contributors

aeschweik avatar bleakleymcd avatar cschloss385 avatar dericed avatar dinahhandel avatar privatezero avatar retokromer avatar yvonneng avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ltopers's Issues

install via linuxbrew fails at gtk+3 dependency

Installing with linuxbrew fails on attempting to satisfy gtk+3 dependency.

It gives the error No package 'atk' found at that point. brew install atk informs me that atk is already installed. I'm hesitant to follow the recommendation to mess with environment variables (BASE_DEPENDENCIES_LIBS,BASE_DEPENDENCIES_CFLAGS) unless that's the only way to get this conflict out of the way.

I tried out the recommendations on this stackoverflow thread to install the dev package for libgtk3 (sudo apt-get install libgtk-3-dev) but got nowhere.

I opened a ticket on linuxbrew/homebrew-core that includes a gist (per their issue requirements) listing my config, build versions, etc.

I am motivated to investigate further, but I don't know where to start. Suggestions?

Thanks!

unable to mount tape with mountlto

Hello All

I'm quite new to LTO / LTFS and I'm reaching out as I feel you may be able to point me in the right direction.

$ mountlto
Checking for ready tapes...
Checking for tape barcode.....

Checked 1 decks and none appear to have tapes ready to mount.

Any tips on getting this tape to mount successfully?

Background:

I have recently acquired an standalone mLogic LTO-6 Thunderbolt tape drive. I've had issues with the mLogic GUI- i have transferred 3 folders of data in 3 separate sessions, however I am unable to access this data. When I mount the tape through the GUI I can clearly see that 2.4TB are 'in use' as it is noted below the options menu, however when I go to retrieve this data it is nowhere to be seen. It seems that the tape is being partitioned and I'm unable to see / access these partitions. The tape is also not visible in my 'volumes' in finder, which is strange as I'd have thought it would appear there like any other drive. It mounts through the following:

/Users/editor/Library/Application\ Support/mTape\ LTFS\ Utility/.MountIndexData/147291539

I initially named the tape AAL210 and it would appear as such when I see the drive in Mac 'Computer', however now it appears as a 9 digit number instead of the tape ID I assigned it.

What I'm trying to do:

I'm currently trying to access the tape through terminal using LTOPERS in an attempt to access all of the data on the tape. I keep running into an error and the tape is unable to mount, perhaps you could point me in the right direction. Here is some additional information.

Thanks in advance,
Dean

$ ltfs -f
307 LTFS14000I LTFS starting, LTFS version 2.4.0.2 (10071), log level 2.
307 LTFS14058I LTFS Format Specification version 2.4.0.
307 LTFS14104I Launched by "ltfs -f".
307 LTFS14105I This binary is built for Mac OS X .
307 LTFS14106I GCC version is 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66)).
307 LTFS17087I Kernel version: Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64.
307 LTFS14063I Sync type is "time", Sync time is 300 sec.
307 LTFS17085I Plugin: Loading "iokit" tape backend.
307 LTFS17085I Plugin: Loading "unified" iosched backend.
307 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
307 LTFS30810I Opening a device through iokit driver (0).
307 LTFS30814I Vendor ID is IBM .
307 LTFS30815I Product ID is 'ULTRIUM-HH6 '.
307 LTFS30816I Firmware revision is JAX1.
307 LTFS30817I Drive serial is 10WT135434.
307 LTFS17160I Maximum device block size is 1048576.
307 LTFS11330I Loading cartridge.
307 LTFS30854I Logical block protection is disabled.
307 LTFS11332I Load successful.
307 LTFS17157I Changing the drive setting to write-anywhere mode.
307 LTFS11005I Mounting the volume.
307 LTFS30854I Logical block protection is disabled.
307 LTFS17227I Tape attribute: Vendor = IBM .
307 LTFS17227I Tape attribute: Application Name = LTFS .
307 LTFS17227I Tape attribute: Application Version = 2.4.0.2 .
307 LTFS17227I Tape attribute: Medium Label = AAL216.
307 LTFS17228I Tape attribute: Text Localization ID = 0x81.
307 LTFS17227I Tape attribute: Barcode = AAL216 .
307 LTFS17227I Tape attribute: Application Format Version = 2.4.0 .
307 LTFS17228I Tape attribute: Volume Lock Status = 0x00.
307 LTFS17227I Tape attribute: Media Pool name = .
307 LTFS14094E Cannot get mount point (0).
307 LTFS30854I Logical block protection is disabled.

$ system_profiler SPSASDataType
SAS:

SAS Domain 0:

  Vendor: ATTO Technology, Inc.
  Product: ATTO ExpressSAS H608
  Revision: 0
  Initiator Identifier: 65536
  SAS Address: 50:01:08:60:00:96:00:A0

    SCSI Target Device @ 0:

      SAS Address: 50:05:07:63:12:56:BB:FD
      SCSI Target Identifier: 0
      SCSI Peripheral Device Type: 1
      Manufacturer: IBM
      Model: ULTRIUM-HH6
      Revision: JAX1

        SCSI Logical Unit @ 0:


          LUN Address: 00:00:00:00:00:00:00:00

[Wish] `readlto` script

The readlto would be the «reverse operation» of the existing writelto, while migratelto is a specific tool for migration (see #2 and the comments #2 (comment)). Part of the code would probably by the same, or at least very similar.

[wish] add volume name management

At the moment, the tape serial is used also as a volume name. I suggest to modify it this way:

  • If no volume name is specified, then the current behaviour is maintained.
  • If a volume name is passed, then (after checking) this value is used as volume name.

date-based versioning

When #94 is merged, I suggest to adopt the date-based versioning for the next release, instead of the semantic versioning which is not used in a consistent way.

[wish] simplify tape serial usage

  • add an optional flag which allows to use the bar code label as the tape serial
  • add the regex that checks the consistency of the bar code label for this not-standard use (this secondary regex would be used instead of the primary one when the flag is set)
  • delete the use of an additional global variable

I guess this would be a better fix for #167.

order in getopts

There is an error, because: should be checked before *. I am preparing a pull request.

`verifylto` needs database

  • move verifylto from general function to database-related functions
  • a database-independent verification should also be implemented

writelto doesn't seem to work well from going from tape to tape

I used writelto -t some_tape_id /Volumes/another_tape_id intending to copy a tape to another tape, but the tape being written to would throw errors and move to read only mode and act broken. I think copying via the gcp command in here doesn't work well from tape to tape, although rsync does. :-/ Some research needed, but maybe the copying method should be changed when the source is also an ltfs tape.

running out of space on tape

Using writelto to write 2.14TB of data to LTO6 tape, capacity 2.41.

Continually, I am running out of space on tape. My guess it has to do with the temp files being written by rsync...? Largest file in transfer is 288G. 2.14 + .28 = 2.42. Would it make sense that a temp file is causing the space issue?

Any way around this?

Specific log error - rsync exited with 11 on the first pass and on the second pass.

2016/06/14 02:42:31 [4891] building file list
2016/06/14 02:42:31 [4891] 20160614-024230 'Volumes/3TB_LTO/LT004A/.' 136 '/Volumes/LT004A/'
2016/06/14 05:25:53 [4891] 20160614-024230 'Volumes/3TB_LTO/LT004A/2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__Derivatives_R2/2015_167_1_1__DER_02_R2.mov' 288343503197 '/Volumes/LT004A/'
2016/06/14 05:25:53 [4891] rsync: read errors mapping "/Volumes/3TB_LTO/LT004A/2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__Derivatives_R2/2015_167_1_1__DER_02_R2.mov": Input/output error (5)
2016/06/14 05:25:53 [4891] WARNING: 2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__Derivatives_R2/2015_167_1_1__DER_02_R2.mov failed verification -- update discarded (will try again).
2016/06/14 07:02:05 [4891] rsync: write failed on "/Volumes/LT004A/2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__Derivatives_R2/2015_167_1_1__DER_02_R2.mov": No space left on device (28)
2016/06/14 07:30:40 [4891] rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.2]

mountlto can't find tape ready to mount

We are starting to write LTO again for the first time in about 12 months.

Having some issues :)

/Users/medialab
≈:≈ formatlto -f
 Will force formatting.
1) 0
2) 32
Which LTO deck? 1
Enter the tape identifier: CHICAG
LTFS15000I Starting mkltfs, LTFS version 2.4.0.2, log level 2.
LTFS15041I Launched by "mkltfs -f -c --device=0 --tape-serial=CHICAG --volume-name=CHICAG".
LTFS15042I This binary is built for Mac OS X .
LTFS15043I GCC version is 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2).
LTFS17087I Kernel version: Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64.
LTFS15003I Formatting device '0'.
LTFS15004I LTFS volume blocksize: 524288.
LTFS15005I Index partition placement policy: None.

LTFS11337I Update index-dirty flag (1) - CHICAG (0x0x7fab044050e0).
LTFS17085I Plugin: Loading "iokit" tape backend.
LTFS30810I Opening a device through iokit driver (0).
LTFS30814I Vendor ID is QUANTUM .
LTFS30815I Product ID is 'ULTRIUM-HH8     '.
LTFS30816I Firmware revision is J4D1.
LTFS30817I Drive serial is 1097001104.
LTFS17160I Maximum device block size is 1048576.
LTFS11330I Loading cartridge.
LTFS30854I Logical block protection is disabled.
LTFS11332I Load successful.
LTFS17157I Changing the drive setting to write-anywhere mode.
LTFS15049I Checking the medium (load).
LTFS30854I Logical block protection is disabled.
LTFS15010I Creating data partition b on SCSI partition 1.
LTFS15011I Creating index partition a on SCSI partition 0.
LTFS17165I Resetting the medium's capacity proportion.
LTFS11097I Partitioning the medium.
LTFS30865I MODESELECT returns Mode Parameters Rounded (-20101) 0.
LTFS11100I Writing label to partition a.
LTFS11100I Writing label to partition b.
LTFS11278I Writing index to partition b.
LTFS17235I Writing index of CHICAG to b (Reason: Format, 0 files) 1097001104.
LTFS17236I Wrote index of CHICAG (b, 1097001104).
LTFS11337I Update index-dirty flag (0) - CHICAG (0x0x7fab044050e0).
LTFS11278I Writing index to partition a.
LTFS17235I Writing index of CHICAG to a (Reason: Format, 0 files) 1097001104.
LTFS17236I Wrote index of CHICAG (a, 1097001104).
LTFS15013I Volume UUID is: fa445c8d-1838-4086-be14-3323afe8ed2e.

LTFS15019I Volume capacity is 11712 GB.
LTFS30854I Logical block protection is disabled.
LTFS15024I Medium formatted successfully.

/Users/medialab
≈:≈ mountlto
 Checking for ready tapes...
Checking for tape barcode.....
Checking for tape barcode.....
 Checked 2 decks and none appear to have tapes ready to mount.

/Users/medialab
≈:≈ ltfs -V
LTFS version 2.4.0.2.
LTFS Format Specification version 2.4.0.

/Users/medialab
≈:≈ ltfs -f -o devname=0
307 LTFS14000I LTFS starting, LTFS version 2.4.0.2, log level 2.
307 LTFS14058I LTFS Format Specification version 2.4.0.
307 LTFS14104I Launched by "ltfs -f -o devname=0".
307 LTFS14105I This binary is built for Mac OS X .
307 LTFS14106I GCC version is 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2).
307 LTFS17087I Kernel version: Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64.
307 LTFS14063I Sync type is "time", Sync time is 300 sec.
307 LTFS17085I Plugin: Loading "iokit" tape backend.
307 LTFS17085I Plugin: Loading "unified" iosched backend.
307 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
307 LTFS30810I Opening a device through iokit driver (0).
307 LTFS30814I Vendor ID is QUANTUM .
307 LTFS30815I Product ID is 'ULTRIUM-HH8     '.
307 LTFS30816I Firmware revision is J4D1.
307 LTFS30817I Drive serial is 1097001104.
307 LTFS17160I Maximum device block size is 1048576.
307 LTFS11330I Loading cartridge.
307 LTFS30854I Logical block protection is disabled.
307 LTFS11332I Load successful.
307 LTFS17157I Changing the drive setting to write-anywhere mode.
307 LTFS11005I Mounting the volume.
307 LTFS30854I Logical block protection is disabled.
307 LTFS30819W A length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 523288).
307 LTFS30819W A length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 523288).
307 LTFS12049E Cannot read: backend call failed (-21716).
307 LTFS17039E XML parser: failed to read a block from the medium (-21716).
Entity: line 1: parser error : Extra content at the end of the document

^
307 LTFS17037E XML parser: failed to read from XML stream.
307 LTFS17016E Cannot parse index direct from medium.
307 LTFS11194W Cannot read index: failed to read and parse XML data (-1014).
307 LTFS11024E Cannot mount volume: read index failed on the index partition.
307 LTFS14013E Cannot mount the volume.
307 LTFS30854I Logical block protection is disabled.

Our OSXFUSE version is 3.10.3

output of `writelto -h` starts with other information

When running writelto -h, now I get at the beginning some informations that has nothing to do with the help itself:

[07:35:59]macOS@test:~/Desktop$ writelto -h
LTO_INDEX_DIR is not set in mmconfig. Default of /Users/macOS/Documents/lto_indexes will be used.
If a different location is desired please set this in mmconfig
writelto 2017-08-04
This script writes the contents of a specified directory onto a mounted
LTFS-formated LTO tape.
Dependencies: gcp mmfunctions

Usage: writelto -t [-e] [-v] | -h
  -t  tape serial
  -e  state yes (Y) or no (N) to ejecting the tape after write, default
      is yes
  -v  reads back and creates checksums for the contents of a tape, and
      writes checksums to a file named with the tape name and date,
      located in the LTO logs directory
  -h  display this help
[07:36:37]macOS@test:~/Desktop$ 

cc @BleakleyMcD

warning on writelto with deckserial-named mount

On some ltfs instances, mountlto is unable to determine the tape name and thus uses the deck serial number as a backup. When using writelto, then, the deck serial is used for the -t option. However recently writelto is checking the value of -t against the barcode regex so the command gives warnings such as this:

time writelto -t HU1212MNU1 /Volumes/GDRIVE8/WRITETHIS
HU1212MNU1 is not valid. The tape id must be exactly 6 capital letters and/or numbers, possibly followed by 'L5', 'L6' or 'L7' specifying the LTO generation.
writelto 0.9
This script writes the contents of a specified directory onto a mounted
LTFS-formated LTO tape.
Dependencies: mmfunctions
Usage: writelto -t [-e] [-v] | -h  -t  tape serial
  -e  state yes (Y) or no (N) to ejecting the tape after write, default
      is yes
  -v  reads back and creates checksums for the contents of a tape, and
      writes checksums to a file named with the tape name and date,
      located in the LTO logs directory
  -h  display this help
cat: /Volumes/GDRIVE8/WRITETHIS/tapechecksum.md5: No such file or directory
'/Volumes/GDRIVE8/WRITETHIS/WRITETHIS.mov' -> '/Volumes/HU1212MNU1/WRITETHIS.mov'

I think since mountlto can not in all cases get the tape barcode, then -t in writelto should be flexible.

Also I see there's a cat error since it's looking for a file that is not there.

report to a database

Sometimes there is an error with the way that lto2filemaker delivers information to filemaker, and if a show has been written to lto it doesn't always show up in the filemaker database.

warn or suggest when a permission issue blocks the creation of a directory to mount to

I can mountlto with sudo but not without. Unsure why.

/Users/medialab
≈:≈ mountlto
Enter the 6 character tape number: LT001B
MOUNTING TAPE : THE TAPE WILL MOUNT TO THE /Volumes/LT001B FOLDER
MOUNTING TAPE : AFTER MOUNT PRESS CONTROL-C TO EJECT
mkdir: /Volumes/LT001B: Permission denied
507 LTFS14000I LTFS starting, LTFS version 2.2.0.1 (5100), log level 2
507 LTFS14058I LTFS Format Specification version 2.2.0
507 LTFS14104I Launched by "ltfs -f -o work_directory=/Users/medialab/Documents/lto_indexes -o eject -o capture_index -o devname=0 -o volname=LT001B -o uid=503 -o sync_type=unmount /Volumes/LT001B"
507 LTFS14105I This binary is built for Mac OS X
507 LTFS14106I GCC version is 4.2.1 (Apple Inc. build 5666) (dot 3)
507 LTFS17087I Kernel version: Darwin Kernel Version 13.4.0: Mon Jan 11 18:17:34 PST 2016; root:xnu-2422.115.15~1/RELEASE_X86_64
507 LTFS14064I Sync type is "unmount"
507 LTFS17085I Plugin: Loading "iokit" driver
507 LTFS17085I Plugin: Loading "unified" iosched
507 LTFS12158I Opening a device through iokit driver (0)
507 LTFS12118I Drive identification is 'ULTRIUM-HH6 '
507 LTFS12162I Vendor ID is IBM
507 LTFS12159I Firmware revision is E4J1
507 LTFS12160I Drive serial is 10WT045909
507 LTFS17160I Maximum device block size is 1048576
507 LTFS17157I Changing the drive setting to append-only mode
507 LTFS11005I Mounting the volume
507 LTFS12207I Logical block protection is disabled
507 LTFS17227I Tape attribute: Vendor = IBM
507 LTFS17227I Tape attribute: Application Name = LTFS
507 LTFS17227I Tape attribute: Application Version = 2.2.0.1
507 LTFS17227I Tape attribute: Medium Label = LT001B
507 LTFS17228I Tape attribute: Text Localization ID = 0x81
507 LTFS17227I Tape attribute: Barcode = LT001B
507 LTFS17227I Tape attribute: Application Format Version = 2.2.0
fuse: bad mount point `/Volumes/LT001B': No such file or directory
507 LTFS14094E Cannot get mount point (-1)
507 LTFS12207I Logical block protection is disabled

/Users/medialab
≈:≈

.schema not writing to LTO_LOGS

schema files are not writing to the specified LTO_LOGS directory as specified in mmconfig. I can't figure out why.
other log files such as writelto.txt and readback.md5 are writing to the correct directory.
.schema files are defaulting to $HOME/Documents

update writeLTO

update writeLTO to include enhanced rsync logging functionality.

Does `writelto` use `rsync`?

I am updating the instructions (WIP). And I have a question to ask: is writelto still using twice rsync? In my opinion, it’s using once gcp – or I am missing something?

No such file or directory - verifylto

I'm pretty sure it's an bad installation, user error likely. But this is the error I get when I try to upgrade ltopers:

todrobbins$ brew upgrade ltopers
==> Upgrading 1 outdated package, with result:
amiaopensource/amiaos/ltopers 0.1.3_1
==> Upgrading amiaopensource/amiaos/ltopers
==> Downloading https://github.com/amiaopensource/ltopers/archive/v0.1.3.zip
Already downloaded: /Users/todrobbins/Library/Caches/Homebrew/ltopers-0.1.3.zip
Error: No such file or directory - verifylto

create recoverLTO script

recoverLTO is a script that will report on the tape’s history (report from hp_ltt /usr/local/HP/LTT/hp_ltt), will look at contents of tape and determine if it is only packages, if there’s checksums, or if B tape has to be compared to A.

Mtape MLogic ERROR

Hi everybody I'm working with iMac Pro thunderbolt 3 Mtape Mlogic unit with LTO7/8. We bought 2 new unit and it seems there is something it doesn't work. We tried lots of test, Mtape Logic said SAS ATTO 1208GT is working fine with the unit and we installed every drive. We use Yoyotta and Canister both the software read the tape but cannot mount and format. I attache last attempt we did trying to mount with Terminale.

We thanks every single advice you can give us.

Schermata 2021-08-24 alle 20 40 13

Schermata 2021-08-25 alle 18 53 39

Output di Terminale salvato.zip

clarification for LTO naming

Users at MIPoPS were following the directions for running formatlto for the first time and ran into some problems with the naming conventions. When they read the following:

The script will ask you select a deck (if you have more than one attached) and to name your tape. There are two different tape ID formats:
the 6-char ID must be exactly 6 capital letters and/or numbers
the 8-char ID must be exactly 6 capital letters and/or numbers, followed by 'L5', 'L6' or 'L7' specifying the LTO generation.

they interpreted this as meaning they could use either a 6 or 8 character tape ID and selected an 8 character id MIPOPSL7 which lead to an error due to the ID not being 6 characters. They have since switched to the format of A00001 ... with success.

I guess that the documentation here could be cleaned up a little bit to make this difference a little more clear for institutions transitioning to LTOpers for the first time? I would edit myself, but I have only worked with 6 character conventions and am not sure of when an 8 character tape id would work and when it will fail. Is this dependent on machine?

Issues with error reporting vs error exiting

We are attempting to write a tape with ltopers, and are running into a problem with it erroring and exiting during the rsync process. There appears to be a certain conflict with the way this is structured in that calling migratefies here using the _run_critical_event function from mmfunctions will cause writelto to fail should an rsync error occur (as is happening).

However, the subsequent parts of writelto such as here imply that it should continue processing and report the errors instead of killing the entire transfer.

This seems like something that should be resolved to one or the other?

${RSYNC_ERR_2} is not reporting

rsync error reports are not reporting err_2. It is just blank.

ex.

rsync exited with 23 on the first pass and on the second pass.
rsync exited with 0 on the first pass and on the second pass.

writelto cant' find tapes ready to mount

After ltopers updates, formatlto works as expected but mountlto reports "Checked 1 decks and none appear to have tapes ready to mount."

That seems odd...

We are using the same workflow as before the recent update. Running remotely so no physical access to LTO deck.

Misc. Documentation Updates

Here are a couple of items that need to be updated in the documentation. Thanks!

1.) Make the reference to the tape name/number/id/serial consistent across all documentation.

2.) In the "Writing to LTO" section, the command should be updated (you are not prompted to do anything when just running "writelto"):
writelto -t ${tapeserial} /Volumes/${sourcedirectory}

Homebrew upgrade warning about rysnc

I've been getting this upgrade error via brew upgrade the last day or two:

==> Installing amiaopensource/amiaos/ltopers
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
==> Downloading https://github.com/amiaopensource/ltopers/archive/v0.1.13.tar.gz
==> Downloading from https://codeload.github.com/amiaopensource/ltopers/tar.gz/v0.1.13
######################################################################## 100.0%
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
🍺  /usr/local/Cellar/ltopers/0.1.13: 14 files, 32.5KB, built in 1 second
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync
Warning: Use rsync instead of deprecated homebrew/dupes/rsync

Any idea what I may have messed up along the way?

Homebrew 1.1.12
Homebrew/homebrew-core (git revision 6ac6; last commit 2017-04-04)

support >6 character tape serials

Opening this ticket to continue the conversation which occurred within: #164, #165, and https://twitter.com/AVpres/status/964199679771717632.

I did find once citation in a tolisgroup.com page which referred an 8 character tape serial, see this article.

However the CLI for mkltfs from IBM, Quantum, Tandberg, list a 6 character restriction on tape-serial such as

  -s, --tape-serial=<id>    Tape serial number (6 alphanumeric ASCII characters)

The SNIA LTFS Format Specification also lists the "volume serial number" as having a length of 6. Elsewhere in the specification is defines ltfs.volumeSerial as "Serial number for the LTFS Volume specified at format time", which I'm presuming means that "volume serial number" of the specification means the same as "tape serial" within the ltfs/mkltfs cli.

I may write to tolisgroup to follow up on their article to find a reference to the "Library version of the LTFS tools" that they mention. Ideally, if there are ltfs/mkltfs utilities using an alternate tape serial of a different length than 6, than I'd like to find a way to query the ltfs/mkltfs help docs to determine the acceptable range, length, and regex of tape serial and apply the correct one, defaulting to the LTFS specification of 6 if unknown. To do so requires seeing the ltfs/mkltfs of a version that does not use the standard 6 character tape serial to figure out how to try to query it to determine that length.

If you have any information or citations about non-6-char tape serials working with ltfs/mkltfs, please share.

optimize naming of mountpoint

Currently mountlto uses /Volumes/${BARCODE} as a default mount point but if it can not determine that then it uses /Volumes/${DECK_SERIAL}. The process of getting these values involves running an lfts command without a mountpoint, which the intent that it fails but provides deck and tape metadata along the way to use in a second correct lfts command to do the mount.

This process requires a lot of time since it has to wait for an ltfs command to fail until running it correctly. I've been considering have the mountpoint always be /Volumes/${DECK_SERIAL} and to skip the first ltfs command (the one that gathers data) and do the second one directly. However I'm finding that it's not easy to make a quick request of the attached LTO deck's serial.

So far I have this command:

ioreg -c IOSCSITargetDevice | grep "INQUIRY Unit Serial Number" | cut -d"=" -f2

This is close but grabs a bunch of other serial numbers that aren't needed. Any better way to gather some form of deck specific identifier?

Fail to compile tnftp during install

FYI:

qblab@qblab-tape:~$ brew install ltopers
==> Installing ltopers from amiaopensource/amiaos

==> Installing dependencies for amiaopensource/amiaos/ltopers: tnftp, tree, xmlstarlet and mediamicroservices/mm/mm
==> Installing amiaopensource/amiaos/ltopers dependency: tnftp
==> Downloading https://opensource.apple.com/tarballs/lukemftp/lukemftp-16.tar.gz
Already downloaded: /home/qblab/.cache/Homebrew/downloads/1faa4668197613d2025e91fa19f8ff1a73539f7316a0888a26d8a9149e5c942e--lukemftp-16.tar.gz
==> ./configure
==> make all
Last 15 lines from /home/qblab/.cache/Homebrew/Logs/tnftp/02.make:
gcc-5 -g -O2 -I. -I./.. -I. -I..  -I./../libnetbsd  -c sl_init.c
gcc-5 -g -O2 -I. -I./.. -I. -I..  -I./../libnetbsd  -c strlcat.c
gcc-5 -g -O2 -I. -I./.. -I. -I..  -I./../libnetbsd  -c strlcpy.c
gcc-5 -g -O2 -I. -I./.. -I. -I..  -I./../libnetbsd  -c glob.c
glob.c: In function 'globextend':
glob.c:723:62: error: 'ARG_MAX' undeclared (first use in this function)
  if ((pglob->gl_flags & GLOB_LIMIT) && (newsize + *limit) >= ARG_MAX) {
                                                              ^
glob.c:723:62: note: each undeclared identifier is reported only once for each function it appears in
Makefile:29: recipe for target 'glob.o' failed
make[1]: *** [glob.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/tmp/tnftp-20200105-20353-7c2n5z/lukemftp-16/tnftp/libnetbsd'
Makefile:22: recipe for target 'libnetbsd.a' failed
make: *** [libnetbsd.a] Error 2

This issue has yet been reported in linuxbrew. Any idea?

Add option to not have tape eject when writing is finished

We often need to add some final documentation to the tape that is not available until it is finished writing. It would be nice to have a flag in writelto (or maybe it is mountlto where it needs to be...) that would tell the script not to eject the tape until the user tells it to.

Wishlist: `migratelto` script

Not entirely sure what this would entail yet. Would possibly be new script for reading ltos, recording checksums and creating log, process log that records when lto fails to read/write. Could have component that checks packet contents, records what is missing.

[Wish] more LTFS implementations

Currently LTOpers considers the LTFS implementation by Quantum. Should we made it «multilingual» and consider also e.g. the LTFS implementation by HPE? If so, then I would suggest to add (here or in mm) a pseudo-script that codes the differences in a grouped manner. This way the existing scripts should need minimal modifications.

The goal is to simplify the adoption of LTOpers in existing environments.

cc @dericed @privatezero @CSchloss385

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.