usf-imars / data-docs Goto Github PK

write up SOP for organization and editing of our shared cruise files in box.com

Naming conventions from AOML changed recently.

Sebastian reports that last summer's file naming pattern was different from 2023 spring.

Sebastian should email Marion (or someone else?).

two depth timestamp columns: one timestamp and one by second(?)

upload imars-objects/homes/cruises to box.com

Frank & Chris K have more info

search is ongoing. places searched:

• makenzie spot 1 - found eDNA stuff instead
• makenzie spot 2 - found eDNA stuff instead
• ?

• create .md file here
• add row to spreadsheet for Matt (TODO: link)
• register into bio eco portal
• upload raw data to (where?)
• upload to (datatype|domain) specific locations:
  • CTD : GCOOS, ERDDAP, NCEI
  • Bio optical : SEABASS
    • BB3
    • GER
  • taxa occurrences : OBIS
  • animal telemetry: ATN
• update BIO ECO info with links to all the above
• present data to MBON group

I've gone through and delete about as much as I can for now. However, I also found a number of WV images that were directly ftp'd to me from PGC and do not appear to have made it into the extra_data directories. Are you able to transfer them into their respective date-based folders? They are located here:

X:\mmccarthy\Matt\USF\Other\CHIMMP\Circe\Raw\Images

Ingesting these is important for organizational purposes, but also to ensure we have as many images as possible.

things to check

• sometimes columns show up as characters
• sometimes lat/lon is wrong

This is something to add to the cruise checklist set up by #4

Spreadsheet is being created manually

• Processes listed in mermaid dg here
• inventory spreadsheet

Column should include:

• who completed the process
• dt of process completion

Almost ready.

Sebastian is checking details of samples.
Verifying the sample IDs.

@sebastiandig has organized MBON cruise data in box.com. We can use rclone to download a copy of the data to IMaRS servers.

advice from Doug:

There is no recommended way since Box doesn’t officially support linux and most of USF IT works in Windows, not linux. I would say whatever way works best for you. There are people using webdav and I also had made some notes on using ftps to transfer files. I’ll send those to you directly in case you want to play with that method.

ftp.box.com notes are in my email.

We are up to December 2022 as of today.

Starting with CDOM then will go back and do the pads.

CDOM processing takes 3 days.

Every week 4 days of filter pads.

12-24 samples/day.

Not doing duplicates but they are being separated. Doing the odds, saving the evens.
Jenn expressed that the duplicates looked good and processing both was not necessary.
SeaBASS data rarely has duplicates on filter pad data.