Comments (11)
Thanks for the example. Could you please post an image of those particular frames, so I can try to determine what the problem was? We don't usually get this, unless the frame is really dark and unusable. Smoothing in time should not be hard to implement inside the algorithm.
from suite2p.
Thanks for the response. I uploaded complete example data (5GB) to Google Drive, it has the same issue as shown above. This .mat file contains uncorrected raw data (3500 frames), motion corrected data (using your phase correlation) and also the final motion correction parameters (x and y shifts) given by the algorithm. The only change I made to your code is that I picked 40 frames in the initialization (pick_reg_init) instead of 20, which should not cause any problems. In this sample file, erroneous corrections were applied to frames 854, 855 and 856 (spurious correction spikes similar to above picture).
https://drive.google.com/open?id=0B67BFxux4io7NENjUFNObW5KMG8
PS. this sample data comes from V1 of transgenic EmxCre Ai93 cTeT mouse, so the signal is weaker than you typically get via virus injection method. However, this particular mouse has even lower signal than usual.
from suite2p.
Any updates on this issue? I also uploaded another dataset with even more severe case of spurious corrections (see image and those huge spikes). This data also have stronger signal, so I don't quite understand why alignment still fails.
https://drive.google.com/file/d/0B67BFxux4io7b2lyTkd2QXJFamM/view?usp=sharing
from suite2p.
Sorry about the delay, I had a deadline. @xigumalu has had a similar problem and tried some smoothing options on her own data. It did not seem to work very well for her, but perhaps it will help on your data. I know she was planning on updating the repository with the new options. We will let you know when it's done.
from suite2p.
Ok, thanks for the update.
I also tried smoothing, but it did not help this issue (almost no effect at all). The only fix I can come up with is to set an upper limit for allowed shifts. This could be done for example by using a smooth-edged window function that is 1 for zero shift and decays rapidly to zero after some large total shift (e.g., couple of typical neuron diameters, >30 pixels or so). This would force the algorithm to look for largest correlation (i.e., FFT peak) inside allowed diameter. Unfortunately user must then define some empirical maximum diameter, which depends on data (e.g. pixel dimension) and typical amount of motion.
Other way that I have also tried is to use iterative alignments based on mean squared error (e.g., imregister in Matlab), but this is much slower approach and resulting correction tends to miss largest motion shifts (i.e., solution is not a global optimal point).
There seems to be no one best method for all data, which is a bit frustrating.
PS. I accidentally used my lab account in my previous post, I'm the same person.
from suite2p.
Hi,
I updated the repository now. There is now an option smooth_time_space which convolves the raw movie with a Gaussian of specified standard deviations in the specified dimensions. A single number [t] smooths only in time, two numbers [t,s] smooths in time with STD t and in space with STD s, three numbers [t,x,y] smooths in time with STD t, in x direction (rows) with STD x, and in y direction (columns) with STD y.
E.g. you can add an option for your data set (in make_db):
db(i).smooth_time_space = [0 2];
to only smooth in space. (Or add ops0.smooth_time_space in the master_file)
Smoothing in time didn't help in my case, smoothing in space was much better but the final result was still not perfect (I recommend watching the registered movies instead of just looking at the determined shifts). Maybe it can be of help in some cases...
from suite2p.
Sorry, the latest comment should have come from me. For some reason, Nick was still logged in on my PC...
from suite2p.
Great, thanks! I'll give it a try. Very interesting, I would not have though doing temporal smoothing for alignment purposes. But it makes sense as it enforces similarity between nearest-neighbor corrections.
For the record, I implemented a version with a cut-off radius for allowed shifts. If no high-enough correlation peak is found inside given radius, shifts are interpolated for those bad frames. Might help in situations when only few well-separated spurious alignment spikes are present. I also made several other modification to improve speed or reduce memory usage. This is a stand-alone version as I don't currently use the whole Suite2P pipeline.
https://drive.google.com/file/d/0B67BFxux4io7MEtrQmxGdXMtVjA/view?usp=sharing
from suite2p.
If there are any outstanding issues, please re-open this issue, or create a new one.
from suite2p.
Hi Marius, Carsen and co.
I am reopening this issue because I am having similar problems with mis-aligned frames (sparse activity in L3 barrel cortex with 6f - so SNR is on the low side).
I tried the db(i).smooth_time_space option but it crashes my matlab (on both of my computers on of which has 64GB RAM and an 8GB Nvidia graphics card so I would think it is not too easy to crash...). .
I smoothed the .tiff files externally with a gaussian filter (spatial smoothing, std = 1) but it didn't help but I also didn't explore temporal smoothing or stronger spatial smoothing.
I played with the registration options (ops0.dobidi; rigid and non-rigid; non-whitened phase correlation; played with ops0.SubPixel, NimgFirstRegistration and mingbegend) but none of them seem to affect the outcome which is a few misaligned frames in the registered tif.
So I was wondering:
Is the db(i).smooth_time_space option still supported? Or do you know why it might make matlab crash?
Do you have any recommendation regarding things for me to try?
Any feedback would be great! (-:
from suite2p.
Hi Christina,
The smoothing should work. Sometimes, old versions of Matlab crash with the newest Nvidia GPUs (10xx), so it would be worth installing 2017a, as it's the only version that "officially" supports GTX 10xx. Otherwise, maybe your tiffs are very large? Though I think we split each tiff in batches.
Btw, if there are just a few misaligned frames, we now have an automatic step that detects those and excludes them. You will see NaNs for those timepoints in the final traces. You could also look by eye at those frames to check there is nothing there to align to.
Cheers,
Marius
from suite2p.
Related Issues (20)
- "NameError: name 'shutil' is not defined, line 231 "shutil.copy(classorig, classfile)" in "roi_detect _and_extract(ops)
- Error in reg2p: array exceeds maximum array size preference
- running suite2p with GUI on GPUs HOT 1
- using "look_one_level_down" doesn't use "bad_frames.npy"
- Input to Suite2P
- Merging ROIs in suite2P matlab version HOT 1
- Interpreting Deconvolution Results HOT 4
- Enquiry regarding diameter of ROI HOT 1
- Cannot set up batch processing
- saving registered uint16 data
- GUI error
- deconvolution_standalone2 not standalone
- tau value for GCaMP8 HOT 2
- combine sbx files HOT 1
- contiguous scanning ROIs HOT 1
- multiplane_parallel in an HPC (Slum) HOT 1
- Transposing F matrix in spike deconvolution
- Optimal tau value for GCaMP8m HOT 1
- Save motion corrected tiffs
- Suite2p on mac
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from suite2p.