penn-cnt / rns_processing_toolbox Goto Github PK

Daylight savings time may change the conversion from local to UTC. This should be handled anytime local time is calculated from UTC time, and is currently NOT handled.

2023 Update: not sure what I was going on about here.... potentially should ignore this issue

Currently, .dat files are converted to .mef (mef 2.0 format) files via the MEF_writer.jar, but upload to Blackfynn fails.

Potential issues:

• mef writer isn't working (could be a bad type conversion)
• plain .dat files are not uploading (could be because they have no inherent time axis)

Potential workarounds:

• try a different conversion (.dat to .edf for example)

The .yml file does not contain all necessary packages for inflating a conda environment that just runs.

Originally, the device number column was completely removed. However, we were losing information about which device was being used. Currently, I simply take all unique device numbers and reassign them to "1,2,etc."

However, I think the first digit (1 vs. 2) of the device ID may correspond to which model of device is implanted. Ideally, this column would preserve that aspect of the original device ID number, then add an increment counter. (e.g if original device IDs are is 1xxxx, 1yyyy, 2zzzz, then updated IDs are [1.1, 1.2, 2.1], or something like that)

I began this process, but had issues determining where to store the log file.

The log file could contain PHI, so ideally it would be stored in the RAW_DATA_FOLDER. However this presents an issue when the RAW_DATA_FOLDER path points directly to the mounted Box drive (because it is read only).

Not sure what the best path forward is for this.

EventStartIdx and EventEndIdx appear incorrect (startIDX occurs before end Idx in deidentifeied ecog_catalog.csv file).

No order issues appear in original data, .dat file appears to be normal (not empty).

Current data structure makes the data unwieldily and hard to use. A tabulated data format would be much more intuitive than a concatenated matrix of discontinuous data.