Code Monkey home page Code Monkey logo

bahanonu / ciatah Goto Github PK

View Code? Open in Web Editor NEW
81.0 7.0 20.0 53.28 MB

CIAtah (pronounced cheetah): a software package for calcium imaging analysis of one- and two-photon imaging datasets. Documentation: https://git.io/ciatah_docs. Formerly known as calciumImagingAnalysis (ciapkg).

Home Page: https://git.io/ciatah_docs

License: MIT License

MATLAB 83.78% C 11.76% Python 0.07% HTML 0.59% Smarty 1.12% CSS 0.16% PHP 0.68% M 1.04% ImageJ Macro 0.79%
calcium-imaging motion-correction signal-extraction miniscope-imaging matlab animal-tracking

ciatah's Introduction

CIAtah

Maintenance GitHub release (latest by date) GitHub top language GitHub license GitHub code size in bytes visitors


CIAtah (pronounced cheetah; formerly calciumImagingAnalysis [ciapkg]) is a software package for analyzing one- and two-photon calcium imaging datasets. It can also be used to process other imaging datasets (e.g. from non-calcium indicators and dyes).

ciatah_logo

CIAtah currently requires MATLAB and runs on all major operating systems (Windows, Linux [e.g. Ubuntu], and macOS).

  • Note: CIAtah version v4 moves the remaining (i.e. all except external packages and software) CIAtah functions into the ciapkg package to improve namespace handling and requires MATLAB R2019b or above (due to package import changes).

Full documentation at https://bahanonu.github.io/ciatah/.

Below are recordings and additional documents for users who want to learn more about calcium imaging analysis/experiments and the CIAtah pipeline.

Book chapter — We have a book chapter that goes over all steps of miniscope imaging: viral injections, GRIN lens probe implant, pain experimental design, data processing and neural/behavioral analysis, and more.

Webinar — This webinar gives an overview of calcium imaging analysis (with a focus on CIAtah) along with tips for improving experiments and analysis: https://info.inscopix.com/inscopix-inspire-view-webinarbiafra-ahanonu-signal-in-the-noise-distinguishing-relevant-neural-activity-in-calcium-imaging.

Workshop tutorial — This recording gives an overview of setting up and using CIAtah: https://www.youtube.com/watch?v=I6abW3uuJJw.

Imaging analysis tools My table with many current imaging analysis tools: https://github.com/bahanonu/imaging_tools.

GRINjector — A surgical device to help with implanting gradient-refractive index (GRIN) lens probes into the brain or other regions: https://github.com/bahanonu/GRINjector.

Upcoming motion correction methods — Methods for motion correction of spinal imaging data using feature identification (e.g. with DeepLabCut), control point registration, and other methods. Additional updates on integration into CIAtah in the future.

Contents

Contact: Biafra Ahanonu, PhD (github [at] bahanonu [dot] com).

Made in USA.
USA


CIAtah features

  • CIAtah package-enclosed functions (in +ciapkg sub-folders) can be used to create GUI-less, command line-ready analysis pipelines. As all functions are within the ciapkg package for improve namespace handling to allow incorporating into other programs.
  • A GUI, via ciatah class, with different modules for large-scale batch analysis.
  • Includes all major calcium imaging analysis steps:
    • movie visualization (including reading from disk, for fast viewing of large movies);
    • pre-processing (motion correction [e.g. TurboReg, NoRMCorre] , spatiotemporal downsampling, spatial filtering, relative fluorescence calculation, etc.);
    • support for multiple cell-extraction methods:
    • manual classification of cells via GUIs;
    • automated cell classification (i.e. CLEAN algorithm, coming soon!);
    • cross-session cell alignment;
    • and more.
  • Includes example one- and two-photon calcium imaging datasets for testing CIAtah.
  • Supports a plethora of major imaging movie file formats: HDF5, NWB, AVI, MP4, ISXD [Inscopix], TIFF, BigTIFF, SLD [SlideBook], and Bio-Formats compatible formats (Olympus [OIR] and Zeiss [CZI and LSM] currently, additional support to be added or upon request).
  • Supports Neurodata Without Borders data standard (see calcium imaging tutorial) for reading/writing cell-extraction and imaging movie files.
  • Animal position tracking (e.g. in open-field assay) via ImageJ plugin.
  • Requires MATLAB and runs on all major operating systems (Windows, Linux [e.g. Ubuntu], and macOS).

CIAtah example features

Support for entire calcium imaging pipeline.

ciapkgMovie

Movie processing, cell extraction, and analysis validation.

• Press play if auto-play disabled.

ciapkgMovie

Cell sorting GUI.

• Press play if auto-play disabled.

ciapkgMovie

Stable cell alignment across imaging sessions.

• Press play if auto-play disabled.

m121_matchedCells


Quick start guide

Below are steps needed to quickly get started using the CIAtah software package in MATLAB.

Download and install CIAtah

  • Clone the CIAtah repository (using GitHub desktop or command line) or download the repository zip and unzip (e.g. run below MATLAB command).
% Optional: this will set MATLAB working folder to the default user path. Make sure you have read/write permissions.
try; cd(userpath); catch; end;

% Download and unzip current repository
unzip('https://github.com/bahanonu/ciatah/archive/master.zip');

% Make CIAtah the working folder
cd('ciatah-master')

Check required toolboxes are installed

CIAtah depends on several MATLAB toolboxes to run properly. Run the below command to have CIAtah check whether dependencies have been installed properly. If not, use the Add-Ons (https://www.mathworks.com/help/matlab/matlab_env/get-add-ons.html) explorer to install each toolbox.

ciapkg.io.matlabToolboxCheck();

Setup CIAtah

  • Download and install CIAtah dependencies. These will all be located in the _external_programs sub-folder within the CIAtah main code directory.
ciapkg.io.loadDependencies;
  • Run CIAtah using the below MATLAB commands. Call obj; in the MATLAB command window each time you want to go back to the main GUI.
    • Note: calciumImagingAnalysis class is now called ciatah, all functionality is the same.
% Loads the class into an object for use in this session
obj = ciatah();

% Runs routines to check dependencies and help user get setup.
obj.setup();

% Open the class menu (always type `obj` then enter load the class/modules menu)
obj % then hit enter, no semicolon!
  • Afterwards, run the modelAddNewFolders module to add data folders the current CIAtah class object.
  • Run obj; in the command window to see the main GUI.
  • Full documentation at https://bahanonu.github.io/ciatah/.
  • [Optional] Users on Windows systems can download Everything (https://www.voidtools.com/). It is a very useful and extremely fast search engine for files and folders that allows users to quickly obtain lists of full folder paths for analysis in CIAtah.
  • [Optional] If run into issues opening certain AVI files (e.g. due to codec issues) with CIAtah/MATLAB, install K-Lite Codec Pack (https://codecguide.com/download_kl.htm) or similar for added support.

Visualize movies quickly using read from disk

Users can quickly visualize movies in any of the supported formats (HDF5, NWB, AVI, TIFF, ISXD, etc.) using the playMovie function. This will read directly from disk, allowing users to scroll through frames to visually check movies before or after processing.

Users can run via the command-line:

% Use the absolute path to the movie file or a valid relative path.
ciapkg.api.playMovie('ABSOLUTE\PATH\TO\MOVIE');

When using HDF5 files, check the dataset name containing movie with h5disp then input the full dataset name (e.g. below is for a standard NWB-formatted HDF5 file):

ciapkg.api.playMovie('ABSOLUTE\PATH\TO\MOVIE','inputDatasetName','/acquisition/TwoPhotonSeries/data');

Alternatively, using the ciatah GUI class, users can select loaded folders and change the regular expression to match the name of the files in the movie, both for the raw data and for any processed movies in the folder. See below:

ciapkgMovie

Quick start (command line or GUI-less batch analysis)

After downloading CIAtah and running the setup as above, users interested in command-line processing can open up the example M-file by running the below command. By running individual code-block cells, users are guided from pre-processing through cell-extraction to cross-session analysis.

edit ciapkg.demo.cmdLinePipeline

Users can import the CIAtah ciapkg that contains the command-line functions using the below command at the beginning of their functions. This will import all ciapkg functions into the functions workspace such that ciapkg.api.loadMovieList() (an alias for ciapkg.io.loadMovieList()) can be called as loadMovieList().

import ciapkg.api.* % import CIAtah functions in ciapkg package API.

Acknowledgments

Thanks to Jones G. Parker, PhD (https://parker-laboratory.com/) for providing extensive user feedback during the development of the CIAtah software package.

Additional thanks to Drs. Jesse Marshall, Jérôme Lecoq, Tony H. Kim, Hakan Inan, Lacey Kitch, Maggie Larkin, Elizabeth Otto Hamel, Laurie Burns, and Claudia Schmuckermair for providing feedback, specific functions, or helping develop aspects of the code used in CIAtah.

References

Please cite Corder*, Ahanonu*, et al. 2019 Science publication or the Ahanonu, 2018 Zenodo release if you used the software package or code from this repository to advance/help your research:

@article{corderahanonu2019amygdalar,
  title={An amygdalar neural ensemble that encodes the unpleasantness of pain},
  author={Corder, Gregory and Ahanonu, Biafra and Grewe, Benjamin F and Wang, Dong and Schnitzer, Mark J and Scherrer, Gr{\'e}gory},
  journal={Science},
  volume={363},
  number={6424},
  pages={276--281},
  year={2019},
  publisher={American Association for the Advancement of Science}
}

Please cite the Ahanonu and Corder, 2022 book chapter if you used procedures detailed therein.

@incollection{ahanonu2022recording,
  title={Recording Pain-Related Brain Activity in Behaving Animals Using Calcium Imaging and Miniature Microscopes},
  author={Ahanonu, Biafra and Corder, Gregory},
  booktitle={Contemporary Approaches to the Study of Pain},
  pages={217--276},
  year={2022},
  publisher={Springer}
}

Please cite the Ahanonu*, Crowther*, 2023 preprint on spinal cord imaging if you use related methods or procedures.

@article{ahanonu2023long,
  title={Long-term optical imaging of the spinal cord in awake, behaving animals},
  author={Ahanonu, Biafra and Crowther, Andrew and Kania, Artur and Casillas, Mariela Rosa and Basbaum, Allan},
  journal={bioRxiv},
  pages={2023--05},
  year={2023},
  publisher={Cold Spring Harbor Laboratory}
}

Please see https://bahanonu.github.io/ciatah/references/ for additional references depending on processing steps undertaken.

Questions?

Please email any additional questions not covered in the repository to github [at] bahanonu [dot] com or open an issue.

Users with versions of MATLAB earlier than R2019b can download CIAtah version v3 (see Releases) until pre-R2019b MATLAB support is fully integrated into v4.

License

Copyright (C) 2013-2023 Biafra Ahanonu

This project is licensed under the terms of the MIT license. See LICENSE file for details.

Repository stats

Statistics on total hits on the main repository and documentation pages. Several badges are present as certain services have gone offline or count page hits differently.

  • visitors (starting 2020.09.22)
  • Hits (starting 2020.09.16)
  • visitors (starting 2022.03.02, backup)
  • visitors (starting 2022.03.02, backup)
  • visitors
  • Visits Badge (starting 2023.12.17)
  • visitors (starting 2023.12.17)
  • visitors

ciatah's People

Contributors

bahanonu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

ciatah's Issues

Downsample raw movie with avi format?

Hi,

Is it possible that modelDownSampleRawMovies method can support the avi file? We got the avi video from UCLA miniscope directly but it is too big to be processed in ciatah (30 min video with 4 gb size). I'm considering to downsample these videos but found only TIFF/H5/ISXD are supported.
Do you have any ideas that avi file can be downsampled in ciatah or I can use other software to downsample them?

PS: I tried ffmpeg to downscale or crop the raw video but found the output files lost much quality than I expected. So I'm worried about the following cell extraction effect.

Thank you!
Hemin

Incomplete data upload during CNMF-E cell extraction

Hi Biafra,

I am a new user of your analysis pipeline for calcium imaging. First, a big thank you developing this great tool.

We are currently using the pipeline to analyse calcium imaging data acquired with UCLA miniscopes. The data acquisition system saves the miniscope videos in separate files, each with 1000 frames. The videos we want to analyse correspond to a 20min session during wich calcium signals were acquired at 30fps. The initial videos were saved with GREY compression and we had to convert them to .TIF in order to run them through the pipeline.

We successfully managed to run modelPreprocessMovie with a downsampling factor of 4 and got the expected output. Our issue came when running modelExtractSignalsFromMovie with CNMF-E. Apparently out of 37 videos, each with 1000 frames the pipeline only uploaded 1000 frames, leading to the error copied below.

information at the beginning of modelExtractSignalsFromMovie
The data has 608 X 608 pixels X 1000 frames.
Loading all data (double precision) requires 2.754 GB RAM

error message during modelExtractSignalsFromMovie
@@@@@@@
Error using calciumImagingAnalysis/modelExtractSignalsFromMovie/saveRunTimes (line 587)
Unable to perform assignment because the size of the left side is 1-by-1 and the size of the right side is 1-by-37.
Error in calciumImagingAnalysis/modelExtractSignalsFromMovie (line 365)
saveRunTimes('cnmfe');
Error in calciumImagingAnalysis/runPipeline (line 202)
obj.(thisFxn{1});
Error in calciumImagingAnalysis/display (line 817)
obj.runPipeline;
@@@@@@@

Any idea why the pipeline might be skipping the upload of the rest of the data?

All the best,
Miguel

Error with ciatah update modelpreprocessMovie

Hi,
I just updated to the most up to date version of ciatah but upon running modelpreprocessMovie to preprocess a .tif movie we run in the error below. We are running in MATLAB R2021b
Let us know what could be the issue: the regex and video filename are corresponding (precisely 077_01, if this could in any way be the issue).

Running standard TIFF
@@@@@@@
Undefined function 'tiffread' for input arguments of type 'char'.

Error in ciapkg.io.load_tif_movie/standardTIFF_new (line 181)
            out.Movie = tiffread(filename,[],'ReadUnknownTags',1);

Error in ciapkg.io.load_tif_movie (line 85)
		standardTIFF_new();

Error in ciapkg.api.load_tif_movie (line 13)
	[out] = ciapkg.io.load_tif_movie(filename,downsample_xy,'passArgs', varargin);

Error in ciapkg.io.loadMovieList (line 643)
						outputMovie = load_tif_movie(thisMoviePath,1,'displayInfo',options.displayInfo,'fileInfo',fileInfoH);

Error in ciapkg.api.loadMovieList (line 9)
	[outputMovie, movieDims, nPixels, nFrames] = ciapkg.io.loadMovieList(movieList, 'passArgs', varargin);

Error in ciatah/modelPreprocessMovieFunction (line 814)
						thisMovie = loadMovieList(movieList,'convertToDouble',0,'frameList',thisFrameList,'inputDatasetName',options.datasetName,'treatMoviesAsContinuous',options.turboreg.treatMoviesAsContinuousSwitch,'loadSpecificImgClass','single','largeMovieLoad',options.turboreg.largeMovieLoad);

Error in ciatah/modelPreprocessMovie (line 42)
	obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);

Error in ciatah/runPipeline (line 209)
				obj.(thisFxn{1});

Error in ciatah/display (line 6)
	obj.runPipeline;
@@@@@@@
+++++++
no movie!

Infinite loop

I tried to use the ciatah signalSorter function as specified here. However, I got an loop error.

It gives this warning at the start.

Warning: File: computeSignalSnr.m Line: 148 Column: 5
The temporary variable 'loopMean' will be cleared at the beginning of each
iteration of the parfor-loop. If 'loopMean' is used before it is set, a runtime
error will occur. For more information, see Parallel for Loops in MATLAB,
"Uninitialized Temporaries".
In signalSorter (line 413)

Then keeps repeating this error until it is stopped.

@@@@@@@
Index exceeds the number of array elements (2).

Error in compareSignalToMovie (line 101)
nPoints = options.inputMovieDims(3);

Error in signalSorter>viewMontage (line 2505)
croppedPeakImages = compareSignalToMovie(inputMovie, inputImage, thisTrace,'getOnlyPeakImages',1,'waitbarOn',0,'extendedCrosshairs',2,'crossHairVal',maxValMovie*options.crossHairPercent,'outlines',1,'signalPeakArray',signalPeakArray,'cropSize',cropSizeLength,'addPadding',1,'outlineVal',NaN,'xCoords',xCoords,'yCoords',yCoords,'inputDatasetName',options.inputDatasetName,'inputMovieDims',options.inputMovieDims,'hdf5Fid',options.hdf5Fid,'keepFileOpen',options.keepFileOpen);

Error in signalSorter>chooseSignals (line 986)
[objCutImagesCollection{i}, ~] = viewMontage(options.inputMovie,thisImage,options,thisTrace,[signalPeakIdx{i}],minValMovie,maxValMovie,options.cropSizeLength,i,1);

Error in signalSorter (line 603)
[choices, safeExit] = chooseSignals(options,1:nSignals, inputImages,inputSignals,objMap, valid, inputStr,tmpDir,sessionID,signalPeakIdx,signalSnr,inputImagesThres,inputImageSizes,peakOutputStat,ROItraces,imgStats,inputSignalSignal,inputSignalNoise,inputImagesBoundaryIndices,signalPeakIdxOriginal,signalPeaksOriginal,instructionStr);
@@@@@@@
@@@@@@@
Index exceeds the number of array elements (0).

Error in signalSorter>chooseSignals (line 1046)
title(['imageCorr = ' num2str(round(peakOutputStat.outputMeanImageCorrs(i)*sigDig)/sigDig) 10 'Press Y to go to last sorted signal.'],'FontSize',options.fontSize,'Interpreter','tex');

Error in signalSorter (line 603)
[choices, safeExit] = chooseSignals(options,1:nSignals, inputImages,inputSignals,objMap, valid, inputStr,tmpDir,sessionID,signalPeakIdx,signalSnr,inputImagesThres,inputImageSizes,peakOutputStat,ROItraces,imgStats,inputSignalSignal,inputSignalNoise,inputImagesBoundaryIndices,signalPeakIdxOriginal,signalPeaksOriginal,instructionStr);
@@@@@@@
0 0
cellmap type: max
@@@@@@@
Index exceeds the number of array elements (0).

Error in signalSorter>chooseSignals (line 1279)
' | imageCorr = ' num2str(round(peakOutputStat.outputMeanImageCorrs(i)*sigDig)/sigDig) ',' num2str(peakOutputStat.outputMeanImageCorrs2(i))...

Error in signalSorter (line 603)
[choices, safeExit] = chooseSignals(options,1:nSignals, inputImages,inputSignals,objMap, valid, inputStr,tmpDir,sessionID,signalPeakIdx,signalSnr,inputImagesThres,inputImageSizes,peakOutputStat,ROItraces,imgStats,inputSignalSignal,inputSignalNoise,inputImagesBoundaryIndices,signalPeakIdxOriginal,signalPeaksOriginal,instructionStr);
@@@@@@@
All warnings have the state 'off'.
calculating movie min/max...
finding centroids...
Finding cell neighbors: 10|100
Done assigning neighbor IDs...

ans =

'Frame 1/4000'

@@@@@@@

New error in modelGetSignalsImages

Hey Biafra! Since pulling the newest updates in the master, I am running into a new error code after running cnmf-e. Everything seems to be fine until the very end, when I get this error:
image

... Which leads to this error:
image

Do you know what could be causing this? I have attached the full command window output.
Omid
Untitled 2.pdf

Issue with computeMatchObjBtwnTrials

Hi Biafra,
After "modelVarsFromFiles", I'm trying to run computeMatchObjBtwnTrials, and get the following error:
image

I was successfully able to do this with the same data set just last week.. so I'm probably just making a stupid mistake. Any ideas what would cause this?

Omid

Info on modelPreprocessMovie

Hi,
My name is Anto and I am relatively new to the use of Ciatah. First of all thank you, it's very very useful!
I had a doubt concerning modelPreprocessMovie and in particular Registration_normalization and Movie_normalization as from the preprocessing options. Filtering and any parameters set in Registration_Noralization are only used to motion correct and not actually applied on the output movie, right? On the other hand, Movie_normalization parameters are used to filter the video and these filtering is actually kept in the output, correct?
Thanks a lot for the clarification!
Best,
Anto

HPC implementation for large videos

Dear Biafra,

we are wondering if it would be possible to deploy ciatah for batch processing in a high performance computing cluster?
Is this something that you have done before or have any experience with?

We have large videos and even with temporal (to 5 Hz) and spatial (4x) downsampling, the process is using up our RAM (of around 300 GB).

Any other suggestion is also welcome!

Thank you!
Natalia

How to use viewCellExtractionOnMovie?

I have a bigTIFF video and have ran downsampling, motion correction, df/f, followed by CNMF-E (saved in .NWB format). I'm not sure how to correctly run viewCellExtractionOnMovie to obtain something like what's in the step-by-step guide where the outlines of extracted cells are superimposed on the video. When I run it via the GUI, I can get the video to display but there aren't any outlines. The .NWB output of CNMF-E is in the same folder as the video. Any help is appreciated!

Can`t install

Hi Biafra,

I'm trying to run CIAtah 4.0 on MATLAB R2020b, on a windows.
but i even cant install it well.
I cant find the file cvx_setup.m (likely calcuumagingAnalysis/_external_programs/cvx_rd).

modelPreprocessMovie NormCorre bug

Hi Biafra,
When trying to use NormCorre from the modelPreprocessMovie function in the gui, I run in the error below.
I guess it must be related to me not having run getNoRMCorreParams, but I am not sure..

Thanks again for your help,
Anto

`Getting NoRMCorre params...
@@@@@@@
Unable to resolve the name 'normcorre.NoRMCorreSetParms'.

Error in ciapkg.motion_correction.getNoRMCorreParams (line 75)
optsNC = normcorre.NoRMCorreSetParms(...

Error in ciapkg.motion_correction.turboregMovie/turboregMovieParallel (line 602)
optsNoRMCorre = ciapkg.motion_correction.getNoRMCorreParams(size(inputMovieCropped),'guiDisplay',0);

Error in ciapkg.motion_correction.turboregMovie (line 377)
turboregMovieParallel();

Error in ciapkg.api.turboregMovie (line 10)
[inputMovie, ResultsOutOriginal] = ciapkg.motion_correction.turboregMovie(inputMovie,'passArgs', varargin);

Error in ciatah/modelPreprocessMovieFunction/subfxn_turboregInputMovie (line 2019)
[thisMovie(:,:,movieSubset), ResultsOutOriginal{iterationNo}{thisSet}] = turboregMovie(thisMovie(:,:,movieSubset),'options',ioptions);

Error in ciatah/modelPreprocessMovieFunction (line 906)
subfxn_turboregInputMovie();

Error in ciatah/modelPreprocessMovie (line 42)
obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@
@@@@@@@
Unable to resolve the name 'normcorre.normcorre_batch'.

Error in ciapkg.motion_correction.turboregMovie/turboregMovieParallel (line 610)
[~,ResultsOut,template2] = normcorre.normcorre_batch(...

Error in ciapkg.motion_correction.turboregMovie (line 377)
turboregMovieParallel();

Error in ciapkg.api.turboregMovie (line 10)
[inputMovie, ResultsOutOriginal] = ciapkg.motion_correction.turboregMovie(inputMovie,'passArgs', varargin);

Error in ciatah/modelPreprocessMovieFunction/subfxn_turboregInputMovie (line 2019)
[thisMovie(:,:,movieSubset), ResultsOutOriginal{iterationNo}{thisSet}] = turboregMovie(thisMovie(:,:,movieSubset),'options',ioptions);

Error in ciatah/modelPreprocessMovieFunction (line 906)
subfxn_turboregInputMovie();

Error in ciatah/modelPreprocessMovie (line 42)
obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)`

signalSorter gone

Hi Guys,
First of all, thanks a lot for your hard work.
I am working with peculiar videos and running ciatah for motioncorr and EXTRACTing calcium data separately to fine tune parameters for cell finding and refinement.
I then would like to use signalSorter but it looks like it's not there in the repo. I hope I am not doing something stupid :') and thank you in advance for your help!!

Correction: it is in the ciapkg folder of the cloned repo so i guess it's just a problem on my side... Thanks anyways and sorry for bothering

Antonio

SNR data export

Hi, Biafra~

Lately, I've been working on a two-photon .tiff file through computeManualSortSignals on MatLab 2020b, win10. Normally, all things pretty good so far. Thank you for the update version2.5.1, it's amazing.

Could you please concretely enlighten a handy approach or code to help me to export the SNR value of every identified cell while or finishing the computeManualSortSignals part. I could only read it from the interface like below on-by-one.

Biafra_参数含义02

If there is no misunderstanding on SNR here in CIAtah v2.5.1, that is Signals Noise Ratio, an important parameter I would use someday. So, hope you would help me with that.

And, one more thing, have you ever considered uploading a teaching video on the Internet about the rest of the operation after computeManualSortSignals?

Best Wishes to you!

Benson
from SheepMountain, NJ

Error: Undefined function 'getFileList'

Hi,

after some hours of trying to get Ciatah to run, I installed it anew by cloning it from github with matlab and decided to ask for help. I wanted to test varius functions of Ciatah and use it for cellextraction from 1photon miniscope data.

I started by using the function modelDownsampleRawMovies. After the input GUI it returns following error message.

Capture

I tried the function viewMovieRegistrationTest and get the same error message. I am running Matlab2020b, all dependencies are loaded correctly, as it is stated when starting up Ciatah.

Where is my mistake? any help is appreciated

Regards

Consider moving to ImageJ-MATLAB instead of Miji

As mentioned on https://imagej.net/plugins/miji, this legacy project is mostly replaced by the functionality in ImageJ-MATLAB:

https://imagej.net/scripting/matlab#running-imagej2-within-matlab

I don't know the usage of Miji in ciatah in detail, and I don't know how involved a transition would be, but it might simplify it for users to just enable the ImageJ-MATLAB update site in Fiji (especially since the Miji website seems to be offline at the moment).

I know that @kouichi-c-nakamura worked quite a bit with the ImageJ-MATLAB integration, he might be able to give helpful advice.

Batch converting isxd to h5 specify inscopix path

Hi, I am batch converting many isxd movies to h5 via the command line. Is there an option to specify a path to the Inscopix software installation in my script?

import ciapkg.inscopix.*
import ciapkg.image.*
import ciapkg.api.*

file_name = strcat(root_dir, '/ISXD_files/', arg1);
opts.maxChunkSize = 5000;
opts.downsampleFactor = 1;
opts.baseInscopixPath = strcat('path_to_inscopix/MATLAB'); % <----
opts.saveFolder = strcat(root_dir, '/HDF5');
convertInscopixIsxdToHdf5(file_name,'options',opts);

Ciatah skipping files

Dear Biafra,

we are running Ciatah (latest version) on matlab 2020 and 2021, and turboreging the videos (.h5).
However, we manage to run it once correctly and then we ge the following error:


1/1: D:\favilan
SKIPPING ANALYSIS FOR THIS FOLDER
pre-allocating movies to display...
Miji loaded!

It seems to be skipping folders for some reason?
Could you help us?

Thanks!
Natalia

Question about registration output

Hello @bahanonu ! First off, thank you for creating the calciumImagingAnalysis MATLAB package. We are using your software to process some miniscope calcium imaging data. We are trying to interpret some of the output of the registration step (turboreg) and have a few questions. Thank you for the help!

  • Can you please explain what the different indices of the ResultsOutOriginal variable (output from the turboreg registration step) correspond to?
  • We noticed registration output (e.g., ResultsOutOriginal{1}{1}) has N+1 elements for N video frames. From the log file, it looks like the reference frame may be included. If so, which index is it?
  • Are the translation values given for each iteration in ResultsOutOriginal the absolute translation applied to the frame or is it relative to the previous iteration (i.e., to get the total translation, we would have to sum all of the translations across iterations)?

GUI very ugly

屏幕截图 2021-10-21 112742
A bunch of warnings pops up as soon as I open it. The command line font is adjusted very small, and there are a lot of garbled characters. The font of the graphical interface is instead too large, causing the display to be truncated.
This extremely terrible GUI makes my first impression very bad, and now I have no desire to use it at all.
I recommend that if the GUI is not well done, don't show it, and honestly do a pure codeful API.

parameters in PCA-ICA analysis

Dear Biafra,

we are using Ciatah to analyse our Ca imaging data, but we are struggling finding good parameter values for the PCA-ICA cell extraction.
Can you tell me what the mu and term_tol paramters are? or point me towards any documentation or literature where they are explained?

Thank you!
Natalia

Turboreg issue in CIAtah 4.0.0

Hi Biafra,

I'm running CIAtah 4.0 on MATLAB R2020b, on a mac.
So far, the speed of the functions and UI are a huge improvement from the previous version! Things run a lot more smoothly.
In the preprocessing step, after the videos are cropped for motion correction, I get the following error:

image

Omid

Trouble loading demo data

Hi Biafra,
I'm running MATLAB_2017b on Mac OS. I know you haven't tested on either of these, but I'm hoping there might be an easy solution.

I have downloaded the demo data to the path:
/omidd/calciumImagingAnalysis/data/2014_04_01_p203_m19_check01_raw, and typed that exact pathname in the window that opens up after you click ModelAddNewFolders.

When I then try to view the movie through the viewMovie module, a series of prompts (animal number, session number, etc) leads me to an open Finder window where I'm asked to open a file. I'm not sure what I'm supposed to open in this window. Is it the movie file? Unfortunately that seems to be greyed out. Does this mean I have not loaded the files correctly?

The error code I ultimately get is:
image

Miji has been set up correctly, because I can use the command MIJ.start.

Omid

Unrecognized field name "GroupHierarchy" + other errors when running modelExtractSignalsFromMovie with EXTRACT algorithm

Any help is appreciated. Matlab R2022b, CIAtah v4.5.9. All toolboxes installed. Full traceback:
`1

!!!!!!!!!!!!!!!!!!!!!
Running: obj.modelExtractSignalsFromMovie

options =

struct with fields:

 signalExtractionRootPath: 'C:\Users\rellis01\Documents\MATLAB\ciatah-master\_external_programs'
            saveNwbOutput: 0
            nwbSaveFolder: 'nwbFiles'
           settingsFolder: 'C:\Users\rellis01\Documents\MATLAB\ciatah-master\+ciapkg\+settings'
settingsPrivateSaveFolder: 'C:\Users\rellis01\Documents\MATLAB\ciatah-master\private\settings\signal_extraction'
               nwbYmlPath: 'C:\Users\rellis01\Documents\MATLAB\ciatah-master\_external_programs\nwbpkg\+nwbpkg\ExampleMetadata.yml'

Did not find: extractor.m
Adding folders: C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\extract\EXTRACT\main_functions;C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\extract\EXTRACT\main_functions\debug_utils;C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\extract\EXTRACT\main_functions\debug_utils\brewermap;C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\extract\EXTRACT\main_functions\helper_functions;C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\extract\EXTRACT\main_functions\solvers;
Found: extractor.m
Warning: The DrawMode property will be removed in a future release. Use the SortMethod property instead.

In inputdlg (line 265)
In ciatah/modelExtractSignalsFromMovie (line 121)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)
Warning: The DrawMode property will be removed in a future release. Use the SortMethod property instead.
In inputdlgcol (line 316)
In ciatah/modelExtractSignalsFromMovie/getAlgorithmOptions (line 1000)
In ciatah/modelExtractSignalsFromMovie (line 167)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)
=====================
Subject: m000

Getting movie info for 1\1: C:\Users\rellis01\projects\juanita_calcium_analysis_feb2023\5minTL.2023-02-14-15-56-28\dfof_ciatah_video_2_5minTL.2023-02-14-15-56-28.h5
Warning: HDF5 dataset name not found in hinfo.Datasets.Name. Trying another method to load HDF5 dataset.

In ciapkg.io.loadMovieList/getHdf5Info (line 1081)
In ciapkg.io.loadMovieList (line 373)
In ciapkg.api.loadMovieList (line 9)
In ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 2044)
In ciatah/modelExtractSignalsFromMovie (line 202)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)
Warning: Could not find /1 in C:\Users\rellis01\projects\juanita_calcium_analysis_feb2023\5minTL.2023-02-14-15-56-28\dfof_ciatah_video_2_5minTL.2023-02-14-15-56-28.h5.
In ciapkg.io.loadMovieList/getHdf5Info (line 1096)
In ciapkg.io.loadMovieList (line 373)
In ciapkg.api.loadMovieList (line 9)
In ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 2044)
In ciatah/modelExtractSignalsFromMovie (line 202)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)
Warning: HDF5 hinfo.GroupHierarchy.Groups.Datasets contains no dataset .
In ciapkg.io.loadMovieList/getHdf5Info (line 1111)
In ciapkg.io.loadMovieList (line 373)
In ciapkg.api.loadMovieList (line 9)
In ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 2044)
In ciatah/modelExtractSignalsFromMovie (line 202)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)
@@@@@@@
Unrecognized field name "GroupHierarchy".

Error in ciapkg.io.loadMovieList/getHdf5Info (line 1115)
nGroups = length(hinfo.GroupHierarchy.Groups);

Error in ciapkg.io.loadMovieList (line 373)
[hReadInfo, thisDatasetName, datasetDims] = getHdf5Info();

Error in ciapkg.api.loadMovieList (line 9)
[outputMovie, movieDims, nPixels, nFrames] = ciapkg.io.loadMovieList(movieList, 'passArgs', varargin);

Error in ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 2044)
DFOFList.(thisSubjectStr) = loadMovieList(movieList,'convertToDouble',0,'frameList',[1:500],'inputDatasetName',obj.inputDatasetName,'treatMoviesAsContinuous',1);

Error in ciatah/modelExtractSignalsFromMovie (line 202)
[gridWidth gridSpacing] = subfxnSignalSizeSpacing();

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@

Subject: m000
@@@@@@@
Unrecognized function or variable 'DFOFList'.

Error in ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 2066)
DFOF = DFOFList.(thisSubjectStr);

Error in ciatah/modelExtractSignalsFromMovie (line 202)
[gridWidth gridSpacing] = subfxnSignalSizeSpacing();

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@
@@@@@@@
Output argument "gridWidth" (and possibly others) not assigned a value in the execution with "ciatah/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing" function.

Error in ciatah/modelExtractSignalsFromMovie (line 202)
[gridWidth gridSpacing] = subfxnSignalSizeSpacing();

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@
Removing unneeded directory from "to add" path list: extractor.m.
Removing unneeded directory from "to add" path list: patchwarp.m.
Adding to JAVA path: C:\Users\rellis01\Documents\MATLAB\ciatah-master_external_programs\bfmatlab_readers\SlideBook6Reader.jar
Folders still need to be removed.
Removing unneeded directory from path: extractor.m.
Removing unneeded directory from path: patchwarp.m.
ImageJ already loaded!
Skipping loading of Miji.

Run processing pipeline by typing below (or clicking link) into command window (no semi-colon!):
obj`

Dot indexing is not supported for variables of this type

Hello Biafra,

First of all, I would like to thank you for the fantastic job in the Algorithms-based Neuroscience Analysis, it helps me a lot while studying calcium imaging analysis. Although I know nothing about codes, algorithms, Matlab, etc. before Jan. 2021, still I believe I could accomplish more results meaningfully following your amazing Matlab protocols, as its name CIAtah.

However, lately, I got some confusing outputs as below every time when I run obj2 relative commands. Frankly, it has been checked whether it procedurally or manually inappropriate compared with Quick start (command-line) and CIAtah main GUI notes. Sadly, I didnt
find any reasons for them. would you please help me to handle all these errors?

Sincerely appreciate your time!
Benson Kin

@@@@@@@
Dot indexing is not supported for variables of this type.
Error in calciumImagingAnalysis/viewMovie/getProperFrameList (line 738)
nMovieFrames = sum(movieDims.z);
Error in calciumImagingAnalysis/viewMovie (line 287)
[frameListTmp] = getProperFrameList('primary');
Error in calciumImagingAnalysis/runPipeline (line 202)
obj.(thisFxn{1});
Error in calciumImagingAnalysis/display (line 817)
obj.runPipeline;
@@@@@@@

Error when trying to view miniscope avi

I am trying to view a .avi file created using miniscope software. I am on a mac and have tried in matlab 2016b and 2018b. I keep getting the error:

Error using VideoReader/init (line 619)
Could not read file due to an unexpected error. Reason: Cannot Decode

I am able to open the video in VLC though not in quicktime. I thought perhaps it was an issue with matlab 2018b based on this thread but going down to 2016b also did not work. Any advice? Thanks!

Error in running viewMovieRegistrationTest Pipeline. How can I solve this?

Hi, I'm trying to run the viewMovieRegistrationTest Pipeline with the example images provided, but I am not able to proceed with the pipeline; I don't see the images I want to see at the end of the pipeline. How can I go around this? I get the following errors:

_movieList = 0×0 empty cell array

Index exceeds the number of array elements (0).

Error in ciatah/viewMovieRegistrationTest>subfxnCropSelection (line 472)
inputFilePath = movieList{movieNo};

Error in ciatah/viewMovieRegistrationTest (line 114)
[registrationCoords] = subfxnCropSelection(toptions,folderList);

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;_

"Please input a file path."

Upgraded to the new version of CIATAH and now I get "Please input a file path." during preprocess movie. Everything is the same as how I have done it in other versions....

Running: obj.modelPreprocessMovie
Loading MIJI + ImageJ.
Adding to Java path: C:\Users\HomePC\OneDrive - Northwestern University\Documents\ciatah-master\_external_programs\imagej\mij.jar
Warning: Invalid file or directory 'C:\Users\HomePC\OneDrive - Northwestern
University\Documents\ciatah-master\_external_programs\imagej\mij.jar'. 
> In javaclasspath>local_validate_dynamic_path (line 271)
In javaclasspath>local_javapath (line 187)
In javaclasspath (line 124)
In javaaddpath (line 71)
In ciapkg.io.manageMiji (line 122)
In ciapkg.api.manageMiji (line 10)
In ciapkg.io.modelAddOutsideDependencies (line 43)
In ciapkg.api.modelAddOutsideDependencies (line 10)
In ciatah/modelPreprocessMovie (line 21)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6) 
Loading MIJI + ImageJ.
Adding to Java path: C:\Users\HomePC\OneDrive - Northwestern University\Documents\ciatah-master\_external_programs\imagej\ij.jar
Warning: Invalid file or directory 'C:\Users\HomePC\OneDrive - Northwestern
University\Documents\ciatah-master\_external_programs\imagej\ij.jar'. 
> In javaclasspath>local_validate_dynamic_path (line 271)
In javaclasspath>local_javapath (line 187)
In javaclasspath (line 124)
In javaaddpath (line 71)
In ciapkg.io.manageMiji (line 122)
In ciapkg.api.manageMiji (line 10)
In ciapkg.io.modelAddOutsideDependencies (line 43)
In ciapkg.api.modelAddOutsideDependencies (line 10)
In ciatah/modelPreprocessMovie (line 21)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6) 
                   matfilePath: ''
                matfileVarname: 'thisMovie'
                    runMatfile: 0
                   showFigures: 1
                folderListPath: {'C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope'}
       checkConcurrentAnalysis: 1
    concurrentAnalysisFilename: '_currentlyAnalyzingFolderCheck.mat'
       processMoviesSeparately: 0
                 processMovies: 1
           loadOptionsFromFile: 0
       turboregNumFramesSubset: 3000
                  turboregType: 'preselect'
                      turboreg: [1×1 struct]
                     dfofMovie: 1
                      dfofType: 'dfof'
              downsampleFactor: 4
                      pxToCrop: 14
              fileFilterRegexp: 'concat_.*.h5'
                   inputPCAICA: 0
                   askForPCICs: 0
                     frameList: []
                   datasetName: '/1'
             outputDatasetName: '/1'
                  refCropFrame: 1
                  deflateLevel: 1
                supportedTypes: {'.h5'  '.hdf5'  '.tif'  '.tiff'  '.avi'  '.isxd'}
    calcDroppedFramesFromMovie: []
                       minType: 'soft'
                    minSoftPct: 1.0000e-03
                    saveMovies: 0
                 saveDfofMovie: 0
                 turboregMovie: 1
                normalizeMovie: 0
               downsampleMovie: 1
                 logFileRegexp: '(recording.*.(txt|xml)|.*_metadata.mat)'
        nwbSettingsDatasetname: '/general/optophysiology/imaging_plane/description'
         h5SettingsDatasetname: '/movie/preprocessingSettingsAll'
                saveFilePrefix: ''
                saveFileSuffix: ''
                   videoPlayer: 'matlab'


ans =

     1

Warning: The DrawMode property will be removed in a future release. Use the SortMethod property instead. 
> In inputdlg (line 258)
In ciatah/modelPreprocessMovieFunction (line 487)
In ciatah/modelPreprocessMovie (line 42)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6) 
    {0×0 char}
    {'raw'   }
    {'/1'    }
    {'/1'    }
    {'1'     }
    {'0'     }
    {0×0 char}
    {0×0 char}
    {'matlab'}
    {0×0 char}


ans = 

  struct with fields:

                           MAIN______________: [1×1 struct]
                      checkConcurrentAnalysis: [1×1 struct]
                            resetParallelPool: [1×1 struct]
                              analyzeFromDisk: [1×1 struct]
                   calcDroppedFramesFromMovie: [1×1 struct]
                   REGISTRATION______________: [1×1 struct]
                                     parallel: [1×1 struct]
                              registrationFxn: [1×1 struct]
                             turboregRotation: [1×1 struct]
                                    RegisType: [1×1 struct]
                        numTurboregIterations: [1×1 struct]
                      turboregNumFramesSubset: [1×1 struct]
                                     pxToCrop: [1×1 struct]
                     motionCorrectionRefFrame: [1×1 struct]
     REGISTRATION_NORMALIZATION______________: [1×1 struct]
                        normalizeMeanSubtract: [1×1 struct]
               normalizeMeanSubtractNormalize: [1×1 struct]
                    normalizeComplementMatrix: [1×1 struct]
                                normalizeType: [1×1 struct]
                             normalizeFreqLow: [1×1 struct]
                            normalizeFreqHigh: [1×1 struct]
                        normalizeBandpassType: [1×1 struct]
                        normalizeBandpassMask: [1×1 struct]
                                      SmoothX: [1×1 struct]
                                      SmoothY: [1×1 struct]
                                      zapMean: [1×1 struct]
            MOVIE_NORMALIZATION______________: [1×1 struct]
                         filterBeforeRegister: [1×1 struct]
                     saveBeforeFilterRegister: [1×1 struct]
                     saveFilterBeforeRegister: [1×1 struct]
                       filterBeforeRegFreqLow: [1×1 struct]
                      filterBeforeRegFreqHigh: [1×1 struct]
                filterBeforeRegImagejFFTLarge: [1×1 struct]
                filterBeforeRegImagejFFTSmall: [1×1 struct]
                             medianFilterSize: [1×1 struct]
             MOVIE_DOWNSAMPLING______________: [1×1 struct]
                         downsampleFactorTime: [1×1 struct]
                        downsampleFactorSpace: [1×1 struct]
    IO_and_MOVIE_IDENTIFICATION______________: [1×1 struct]
                             inputDatasetName: [1×1 struct]
                            outputDatasetName: [1×1 struct]
                             fileFilterRegexp: [1×1 struct]
                      processMoviesSeparately: [1×1 struct]
                       loadMoviesFrameByFrame: [1×1 struct]
                treatMoviesAsContinuousSwitch: [1×1 struct]
                        loadMovieInEqualParts: [1×1 struct]
                                  useParallel: [1×1 struct]
                             nParallelWorkers: [1×1 struct]
                 STRIPE_REMOVAL______________: [1×1 struct]
                       stripOrientationRemove: [1×1 struct]
                                    stripSize: [1×1 struct]
                          stripfreqLowExclude: [1×1 struct]
                         stripfreqHighExclude: [1×1 struct]
                        stripfreqBandpassType: [1×1 struct]


ans =

     1

:
                           MAIN______________: {'====================='}
                      checkConcurrentAnalysis: 0
                            resetParallelPool: 0
                              analyzeFromDisk: 0
                   calcDroppedFramesFromMovie: []
                   REGISTRATION______________: {'====================='}
                                     mcMethod: 'turboreg'
                                     parallel: 1
                              registrationFxn: 'transfturboreg'
                             turboregRotation: 0
                                    RegisType: 1
                        numTurboregIterations: 1
                      turboregNumFramesSubset: 5000
                                     pxToCrop: 14
                     motionCorrectionRefFrame: 100
                     regRegionUseBtwnSessions: 1
     REGISTRATION_NORMALIZATION______________: {'====================='}
                        normalizeMeanSubtract: 1
               normalizeMeanSubtractNormalize: 1
                    normalizeComplementMatrix: 1
                                normalizeType: 'bandpass'
                             normalizeFreqLow: 70
                            normalizeFreqHigh: 100
                        normalizeBandpassType: 'bandpass'
                        normalizeBandpassMask: 'gaussian'
                                      SmoothX: 10
                                      SmoothY: 10
                                      zapMean: 0
            MOVIE_NORMALIZATION______________: {'====================='}
                         filterBeforeRegister: []
                     saveBeforeFilterRegister: 0
                     saveFilterBeforeRegister: []
                       filterBeforeRegFreqLow: 0
                      filterBeforeRegFreqHigh: 10
                filterBeforeRegImagejFFTLarge: 10000
                filterBeforeRegImagejFFTSmall: 80
                             medianFilterSize: 3
                  MOVIE_DETREND______________: {'====================='}
                                detrendDegree: 1
             MOVIE_DOWNSAMPLING______________: {'====================='}
                         downsampleFactorTime: 2
                        downsampleFactorSpace: 3
    IO_and_MOVIE_IDENTIFICATION______________: {'====================='}
                             inputDatasetName: '/1'
                            outputDatasetName: '/1'
                             fileFilterRegexp: 'raw'
                      processMoviesSeparately: 0
                       loadMoviesFrameByFrame: 0
                treatMoviesAsContinuousSwitch: 1
                        loadMovieInEqualParts: 0
                                  useParallel: 1
                             nParallelWorkers: 11
                               largeMovieLoad: 0
                 STRIPE_REMOVAL______________: {'====================='}
                       stripOrientationRemove: 'none'
                                    stripSize: 7
                          stripfreqLowExclude: 20
                         stripfreqHighExclude: 20
                        stripfreqBandpassType: 'highpass'
                                 refCropFrame: 100

Saving settings to: C:\Users\HomePC\OneDrive - Northwestern University\Documents\ciatah-master\private\settings\20230419_004031_modelPreprocessMovieFunction_settings.mat.
folderListMinusComments class: double
1/1: C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope
options.fileFilterRegexp: raw
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(1).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(2).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(3).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(4).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(5).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(6).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(7).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(8).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(9).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(10).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(11).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(12).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(13).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(14).avi
C:\Users\HomePC\OneDrive - Northwestern University\Desktop\videos\eyeblink\022223\2023_04_18\19_01_33\My_V4_Miniscope\raw(15).avi
Please input a file path.
>> `

Using extract (2021)

Hi Biafra,
Super excited that the new cell extraction method is released and supported by CIAtah!
I was able to load EXTRACT into the external_programs folder, but under Cell Extraction Methods in the GUI, the only "EXTRACT" is still the 2017 version. Is this the correct method?
Omid

image

issue loading AVI -> variable does not support dot indexing (solved)

Line 415 in ciapkg.io.loadMovieList.m seems to have an extra line that needs to be commented out in the AVI case, or else AVIs will fail to load correctly. I commented out that line and everything ran smoothly.

case 'avi'
    xyloObj = VideoReader(thisMoviePath);
    dims.x(iMovie) = xyloObj.Height;
    dims.y(iMovie) = xyloObj.Width;
    dims.z(iMovie) = xyloObj.NumberOfFrames;
    dims.one(iMovie) = xyloObj.Height;
    dims.two(iMovie) = xyloObj.Width;
    dims.three(iMovie) = xyloObj.NumberOfFrames;
    tmpFrame = read(xyloObj, 1, 'native');
>   %tmpFrame = tmpFrame.cdata;
    %tmpFrame = readFrame(xyloObj);

non-existent field 'globalIDs'

Hey Biafra!
I am running computeMatchObjectsbtwSessions on a new data set and encountering that error that you previously fixed, something about dividing by zeros after CNMF-e?

image

Omid

Using the cell extraction output when extracting cells with cnmf-e

Hi Biafra,

Some people in the lab are getting started with new imaging projects and have been using the calciumImagingAnalysis package with great success to preprocess movies and extract cells.
They've been using cnmf-e and I am a bit unsure about what should be used from the cnmfeAnalysisOutput file for further data analysis.
For example when I extract cells from movies for my project with Extract, I then use extractAnalysisOutput.traces. But cnmfeAnalysisOutput has extractedSignals, extractedSignalsEst, and extractedPeaks.

What is exactly the difference between these 3, and do you have any advice on what is best to use when one wants to plot rate maps for example?

Thanks!
Lucie

Memory problem

When I try to preprocess videos bigger than my JAVA heap memory (34Gb, max allowed) I get the error below and I was wondering how can I analyse videos with size bigger than that.
Thank you very much in advance for your help

pre-allocating single ones matrix: 800 1280 38801
@@@@@@@
Error using ones
Requested 800x1280x38801 (148.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
Error in loadMovieList (line 472)
outputMovie = ones(preallocSize,imgClass);
Error in ciatah/modelPreprocessMovieFunction (line 729)
thisMovie = loadMovieList(movieList,'convertToDouble',0,'frameList',thisFrameList,'inputDatasetName',options.datasetName,'treatMoviesAsContinuous',options.turboreg.treatMoviesAsContinuousSwitch,'loadSpecificImgClass','single');
Error in ciatah/modelPreprocessMovie (line 37)
obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);
Error in ciatah/runPipeline (line 202)
obj.(thisFxn{1});
Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@
pre-allocating movies to display...
+++++++
no movie!

MIJ 1.3.9: Matlab to ImageJ Interface

More Info: http://bigwww.epfl.ch/sage/soft/mij/
Help: MIJ.help
JVM> 1.3.9
JVM> Version: 1.8.0_181
JVM> Total amount of memory: 387584 Kb
JVM> Amount of free memory: 244514 Kb
ImageJ> Version:1.53k
ImageJ> Memory:141MB of 29964MB (<1%)
ImageJ> Directory plugins: Not specified
ImageJ> Directory macros: Not specified
ImageJ> Directory luts: C:\Users\realtime\Documents\MATLAB\luts
ImageJ> Directory image: Not specified
ImageJ> Directory imagej: C:\Users\realtime\Documents\MATLAB
ImageJ> Directory startup: C:\Users\realtime\Documents\MATLAB
ImageJ> Directory home: C:\Users\realtime\

Status> ImageJ is running.

Warning: The DrawMode property will be removed in a future release. Use the
SortMethod property instead.

In ciapkg.overloaded.msgbox (line 280)
In ciatah/modelPreprocessMovieFunction>playOutputMovies (line 2432)
In ciatah/modelPreprocessMovieFunction (line 1090)
In ciatah/modelPreprocessMovie (line 37)
In ciatah/runPipeline (line 202)
In ciatah/display (line 6)
+++++++
no movie!
ImageJ instance ended cleanly

Error during cross registration

Hi, I keep getting this error during cross registration

folderGlobalIdx: 2  3
@@@@@@@
Unable to perform assignment because the size of the left side is 2855-by-1 and the size of the right side
is 2855-by-2.

Error in ciatah/viewMatchObjBtwnSessions (line 213)
				globalIDs(:,addNo) = globalIDsTmp(:,folderGlobalIdx);

Error in ciatah/computeMatchObjBtwnTrials (line 237)
				obj.viewMatchObjBtwnSessions('runGui',0);

Error in ciatah/runPipeline (line 209)
				obj.(thisFxn{1});

Error in ciatah/display (line 6)
	obj.runPipeline;
@@@@@@@

I tried making it so it just accepted the first folderGlobalIdx but that gave me a bunch of other errors, so I figured I would consult the expert :)

PCA traces have negative values

Hi Biafra,

We are wondering what exactly are we getting in the traces from the PCA-ICA analysis?
Looking at the traces stored in the PCA-ICA output.mat file we noticed some have negative values. So we are wondering whether these are raw traces or dff?

Thank you!
Natalia

"Edge vector" error during signal extraction

Hey Biafra,

After running cnmf-e or PCA/ICA, I get the same error:
image

Here is an example of how I get there, perhaps there is a mistake along the way?
image
(Here I'm still using 'crop' because I chose not to manually modify the movie.)

image

Then the following Finder window opens up. I'm not sure which file to open here. I tried opening a few different ones, but all led to the same error.
image

Omid

Poor cell segmentation

I am getting cell segmentations that look pretty bad, and I think it might be because of how I am preparing the videos. Screenshots of the cell segmentations on the raw bigTIFF video and the preprocessed (downsampled 4x, turbo-reg'd, dfof) video are below:

Raw:
raw_movie

Pre-processed
dfof_movie

I think the problem might be how I am preparing the bigTIFF video. The video is prepared from TIF images using the following script:

import cv2
import os
import numpy as np
from tqdm import tqdm
from tifffile import *

def tif_to_video(input_path, output_filename):

    # Define the location of the TIF files
    dir_path = f'{input_path}'

    # List of TIF files
    tif_files = [f for f in os.listdir(dir_path) if f.endswith('.TIF')]

    # Sort image filenames by number following 'p' in filename
    framenums = []
    for tif_file in tif_files:
        p_idx = tif_file.find('p')
        underscore_idx = tif_file[p_idx:].find('_')
        framenum = tif_file[p_idx+1:p_idx+underscore_idx]
        framenums.append(int(framenum))
    
    # order filenames in a list
    tif_files = [tif_file for _, tif_file in sorted(zip(framenums, tif_files))]

    ### bigTIFF ###
    output_file = f'{output_filename}.tif'

    arr_list = []
    # Iterate over the TIF files, import and add each TIF frame to arr_list
    for i, tif_file in tqdm(enumerate(tif_files)):
        img = cv2.imread(os.path.join(dir_path, tif_file), 0)
        img = img[...,None] # add third dimension (i.e., frame)
        arr_list.append(img)
    
    # concatenate frames into numpy array
    tiff_arr = np.concatenate(arr_list, axis=2)
    
    # export numpy array in bigTIFF format at 20 FPS
    with TiffWriter(output_file, bigtiff=True) as tif:
        for frame in np.transpose(tiff_arr,(2,0,1)):
            tif.write(frame, metadata={'fps':20.0}, compression='JPEG')    

Any help with this is greatly appreciated. Thanks in advance!

Drift in delta F/F

image

Hi @bahanonu , I was analyzing video and do see drift in delta F/F. Do you have any idea why is that?
I am not sure, but it looks like this mouse show some transients. What do you think?

Thanks!

Merging components

Dear Biafra,

is there a way to merge two components into one? We are using PCA-ICA and some cells are segmented into "subcells" that we would like to merge together so that it is only one cell.

Thank you!
Natalia

.h5 file not found after preprocessing (motion correction) step

Hello,

I have been trying to preprocess a video, and in particular to apply motion correction to it (because I have done the other steps such as downsampling and spatially filtering it in the Inscopix Data Processing Software before exporting it as a .tiff file for motion correction and cell extraction) but I do not find the output .h5 file once the program runs turboreg (I have tried to save the motion corrected and the df/f movies) hence I can't further run cell extraction algorithms. Errors such as these appear :

Error using h5create (line 71)
The value of 'Dataset' is invalid. Expected DATASET to be nonempty.

Error in writeHDF5Data (line 74)
h5create(saveDir,options.datasetname,dataDims,'Datatype',inputClass,'ChunkSize',dataDimsChunkCopy,'Deflate',options.deflateLevel);

Error in calciumImagingAnalysis/modelPreprocessMovieFunction (line 645)
movieSaved = writeHDF5Data(thisMovie,savePathStr,'datasetname',options.outputDatasetName,'addInfo',{options.turboreg,analysisOptionStruct,optionsCopy},'addInfoName',{'/movie/processingSettings','/movie/analysisOperations','/movie/modelPreprocessMovieFunctionOptions'},'deflateLevel',options.deflateLevel)

Error in calciumImagingAnalysis/modelPreprocessMovie (line 37)
obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);

Error in calciumImagingAnalysis/runPipeline (line 1582)
obj.(thisFxn{1});

Error in calciumImagingAnalysis/display (line 882)
obj.runPipeline;
@@@@@@@

I am new to this and not sure of what is causing this issue, what could I try to fix this?

Thanks a lot for your support and guidance,
Best wishes

modelSaveMatchObjBtwnTrials

Hello,
I'm having an issue when I try to save my results from the cross-session cell alignment. When I try using modelSaveMatchObjBtwnTrials a gui pops up for me to select a folder to save my data. After having selected a folder then this pops up
Capture

The issue is that when I try to look through the folder that I selected to save my data then the file is not there. How can I save the data, and how can I access the file?

Error loading frame for modelPreprocessMovie

Hi there, I've been trying to get ciatah working on my ubuntu 20.04 machine. After some tinkering I've got videos loaded and can even take a folder with a bunch of UCLA miniscope avi files through the viewMovieRegistrationTest step.

However, when I try and run that same folder with the same regex strings through modelPreprocessMovie I get a blank image for selecting the ROI for motion correction. If I drag and double click anyway, i get an indexing error (see below).

I'm not 100% sure of the workflow here, if it would be better for me to try and downsample and convert to h5 first (running modelDownsampleRawMovies throws no errors but produces no output). Any help would be greatly appreciated!

@@@@@@@
Index in position 1 is invalid. Array indices must be positive integers or logical values.

Error in ciatah/modelPreprocessMovieFunction>localfxn_turboregCropSelection (line 2563)
thisFrameCropped = thisFrame(pts(2):pts(4), pts(1):pts(3));

Error in ciatah/modelPreprocessMovieFunction (line 637)
[turboRegCoords] = localfxn_turboregCropSelection(options,folderList);

Error in ciatah/modelPreprocessMovie (line 42)
obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);

Error in ciatah/runPipeline (line 209)
obj.(thisFxn{1});

Error in ciatah/display (line 6)
obj.runPipeline;
@@@@@@@
Warning: User likely did not give calciumImagingAnalysis a proper raw input file regular expression or incorrect HDF5 input dataset
name!

In ciatah/modelPreprocessMovieFunction (line 642)
In ciatah/modelPreprocessMovie (line 42)
In ciatah/runPipeline (line 209)
In ciatah/display (line 6)

image

extracted peaks

Hi,
I remember reading something in a readme a while ago that there is a more exact way to get peak times than using extractedPeaks, but for the life of me I cannot find it anymore. Is it deprecated?

Thank you!

non-existent field 'GroupHierarchy'

Hi,Biafra
I'm following the steps of your video in youtube to run ciatah,but i have a error.
When viewMovieRegistrationTest is running,the following error occurs.
image
I don't know how to handle it. Would you help me find the problem? Looking forward to your reply.

Yancy

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.