cabooster / deepcad Goto Github PK

View Code? Open in Web Editor NEW

96.0 96.0 21.0 52.41 MB

DeepCAD: Deep self-supervised learning for calcium imaging denoising

License: GNU General Public License v2.0

Python 100.00%

deepcad's People

Contributors

Stargazers

Watchers

deepcad's Issues

Clarification of input arguments and output files when training my own model

Hi cabooster,

I think I have successfully trained a model on my data, which is 23,075 frames of continuous 15 Hz imaging. I have a few questions about the parameters going in and the outputs:

What does '--select_img_num' mean? I see it says 'select the number of images', is this the number of images from the dataset that will actually go in to the training (i.e. leaving some data out to test on?). I left this at 10,000 for my 23,075 frames.
What does '--train_datasets_size' mean? Is this meant to be the total number of frames in my training dataset? I left this at 1,200, but I have 23,075 frames, was this incorrect?
The files going in the '--output_dir' are .tif files - what are these? It looks like an image of 'noise' and 'output' at the end of each training epoch?
The files going in to the '--pth_path' are .pth or .yaml. For your example training there was only a single .pth file, 'G_12_1199.pth'. Is each .pth file a model from each epoch of training? Is it best to use the model from the final epoch of training to test on new data?
Linking these questions, should I be looking for the best 'output.tif' file from all the epochs and then using that epoch's .pth file in future testing?
Is '--normalize_factor' related to the bit-depth of the image? I see it is 65535 by default, is this because you were working with 16-bit images?

Many thanks in advance for your answers,

Rob

error of dividing zeros

I got an error when I tried to train a model myself, and I think this "s_num" variable is not defined by myself. Could I possibly try to change it to a different value somewhere in the code for fixing this issue? Thanks!

"stack_num -----> 172
whole_w -----> 512
whole_h -----> 512
whole_s -----> 500
w_num -----> 5
h_num -----> 5
s_num -----> 1
Traceback (most recent call last):
File "train.py", line 66, in
name_list, noise_img, coordinate_list = train_preprocess_lessMemoryMulStacks(opt)
File "/home/Documents/DeepCAD/DeepCAD_pytorch/data_process.py", line 266, in train_preprocess_lessMemoryMulStacks
gap_s2 = get_gap_s(args, noise_im, stack_num)
File "/home/Documents/DeepCAD/DeepCAD_pytorch/data_process.py", line 186, in get_gap_s
gap_s = math.floor((whole_s-args.img_s*2)/(s_num-1))
ZeroDivisionError: division by zero
"

Tiling effect and "black hole" artifacts

Hi cabooster,

I tested with the pretrained model G_12_1199.pth on my microscopy image data, and the output images show a patch edge effect (please see the image below, left: input; right: output).

Also, I could observe "black hole"-like artifacts, i.e., some strange black circle-shaped artifacts (please see the image below, left: input; right: output).

Thank you!

Testing multiple models

Can the test.py argument '--denoise_model' be a directory containing multiple models? This is implied by the tip: 'A folder containing models to be tested', but I wanted to check with you. If not, I guess I could run the test.py programmatically in a loop for each model with different arguments?
Will it create an output for each model during testing in the --output_dir, if the above answer is yes?

CUDA out of memory when using test data

Hi,

I really liked your paper on bioRxiv and wanted to compare results between DeepCAD and DeepInterpolation (Allen Institute).

I managed to get model training to work on our analysis server with GPU, as per your instructions on Github. But, for some reason testing (e.g. python script.py test) runs out of GPU memory.

Is there any way to change the batch size of testing so that I don't run out of GPU memory? I see that batch_size=1 here, presumably the lowest already?

We have a RTX 2080 Ti GPU with 11 GB RAM, an additional 187 GB system RAM and an Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz.

Many thanks,

Rob

the parameter of your training ----->                                                                                                     
Namespace(GPU=0, b1=0.5, b2=0.999, batch_size=1, cuda=False, datasets_folder='DataForPytorch', datasets_path='datasets', denoise_model='ModelForPytorch', epoch=0, gap_h=56, gap_s=400, gap_w=56, img_h=64, img_s=464, img_w=64, lr=0.001, n_epochs=100, normalize_factor=65535, output_dir='./results', pth_path='pth', test_datasize=6000, train_datasets_size=1000)
para.yaml                                                                                                                                 
{'GPU': 3, 'b1': 0.5, 'b2': 0.999, 'batch_size': 1, 'datasets_folder': '20201020_10_ZOOM1.3_550Vx575H_FOV_30Hz_170umdepth_soma_10AMP_0.3_0001', 'datasets_path': 'datasets', 'epoch': 0, 'gap_h': 64, 'gap_s': 150, 'gap_w': 64, 'img_h': 64, 'img_s': 464, 'img_w': 64, 'lr': 5e-05, 'n_epochs': 20, 'normalize_factor': 1, 'output_dir': './results', 'train_datasets_size': 1200}                                          
Using GPU.                                                                                                                                 
input_name ----->  DataForPytorch_x0_y0_z0                                                                                                
single_coordinate ----->  {'init_h': 0, 'end_h': 64, 'init_w': 0, 'end_w': 64, 'init_s': 0, 'end_s': 464, 'stack_start_w': 0, 'stack_end_w': 60.0, 'patch_start_w': 0, 'patch_end_w': 60.0, 'stack_start_h': 0, 'stack_end_h': 60.0, 'patch_start_h': 0, 'patch_end_h': 60.0, 'stack_start_s': 0, 'stack_end_s': 432.0, 'patch_start_s': 0, 'patch_end_s': 432.0}                                                            
real_A ----->  torch.Size([1, 1, 464, 64, 64])                                                                                             
[Batch 0/1215] [Time Left: 0:08:59.692619] [Time Cost: 0:00:00.444208] input_name ----->  DataForPytorch_x0_y0_z1
single_coordinate ----->  {'init_h': 0, 'end_h': 64, 'init_w': 0, 'end_w': 64, 'init_s': 400, 'end_s': 864, 'stack_start_w': 0, 'stack_end_w': 60.0, 'patch_start_w': 0, 'patch_end_w': 60.0, 'stack_start_h': 0, 'stack_end_h': 60.0, 'patch_start_h': 0, 'patch_end_h': 60.0, 'stack_start_s': 432.0, 'stack_end_s': 832.0, 'patch_start_s': 32.0, 'patch_end_s': 432.0}                                                   
real_A ----->  torch.Size([1, 1, 464, 64, 64])      
                                                                                      
Traceback (most recent call last):                                                                                                          
File "test.py", line 119, in <module>                                                                                                       
fake_B = denoise_generator(real_A)                                                                                                      
File "/home/rlees/anaconda3/envs/deepCAD/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl                   
result = self.forward(*input, **kwargs)                                                                                                 
File "/home/rlees/Documents/Code/DeepCAD/DeepCAD_pytorch/network.py", line 20, in forward     
fake_x = self.Generator(x)                                                                                                              
File "/home/rlees/anaconda3/envs/deepCAD/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl                   
result = self.forward(*input, **kwargs)                                                                                                 
File "/home/rlees/Documents/Code/DeepCAD/DeepCAD_pytorch/model_3DUnet.py", line 96, in forward                                            
x = decoder(encoder_features, x)                                                                                                        
File "/home/rlees/anaconda3/envs/deepCAD/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl                   
result = self.forward(*input, **kwargs)                                                                                                 
File "/home/rlees/Documents/Code/DeepCAD/DeepCAD_pytorch/buildingblocks.py", line 274, in forward                                           
x = F.interpolate(x, size=output_size, mode='nearest')                                                                                  
File "/home/rlees/anaconda3/envs/deepCAD/lib/python3.6/site-packages/torch/nn/functional.py", line 3134, in interpolate                     
return torch._C._nn.upsample_nearest3d(input, output_size, scale_factors)           
                                                  
RuntimeError: CUDA out of memory. Tried to allocate 928.00 MiB (GPU 0; 10.76 GiB total capacity; 9.23 GiB already allocated; 490.56 MiB free; 9.31 GiB reserved in total by PyTorch)

fiji error resahpe tensor

Dear authors, this seems to be a great tool. One that for sure many people will use. I was curious to try it out, and got the fiji plugin, but when trying to run it I get the following error - see below.

Any chance of help?

best,
Alvaro

Info: 1024 frames stack of 256x256px, using 8, 8 of x,y overlap and 32, 64 or 128 or z overlap.
Running on linux ubuntu

[ERROR] Module threw exception
java.lang.IllegalArgumentException: Input to reshape is a tensor with 2147483648 values, but the requested shape has 41943040
[[{{node group_normgn_enc_conv0/Reshape}}]]
at org.tensorflow.Session.run(Native Method)
at org.tensorflow.Session.access$100(Session.java:48)
at org.tensorflow.Session$Runner.runHelper(Session.java:326)
at org.tensorflow.Session$Runner.run(Session.java:276)
at network.ModelExecutor.N2NDenoising(ModelExecutor.java:82)
at network.ModelExecutor.run(ModelExecutor.java:351)
at commands.DetectTiledImage.run(DetectTiledImage.java:183)
at org.scijava.command.CommandModule.run(CommandModule.java:196)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

The effect of different values of img_h, img_w, img_s, gap_h, gap_w, gap_s when testing

When testing the trained model, how do different values of img_h, img_w, img_s, gap_h, gap_w, gap_s affect the output images?

Probelms of GPU acceleration

Hi,
Thanks for providing us the ImageJ plugin. I had one question about GPU acceleration. I already followed the pipeline listed here. But when I am using DeepCAD to process my images, ImageJ only takes 1% GPU but 60% CPU (No warning about only using CPU). I selected TensorFlow 1.15.0 GPU in the ImageJ and installed TensorFlow 2. How could I use my GPU for computation in ImageJ?

Help installing FIJI plugin

Hi,

I am following the instructions to setup the FIJI plugin but I can't get it installed. After downloading and installing, FIJI asks to reboot the application for changes to take effect. I did so, but then DeepCAD does not appear among the installed plugins or in the search bar. I made a fresh FIJI installation but the problem persisted, and same problem observed with both DeepCAD-0.3.0.jar and DeepCAD-0.3.6.jar I am running on Linux:

LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	RedHatEnterpriseServer
Description:	Red Hat Enterprise Linux Server release 7.9 (Maipo)
Release:	7.9
Codename:	Maipo
Fiji:		2.1.0
Java:		1.8.0

Unstitched output

I've been running DeepCAD with the following command:

main.py --GPU 0 --img_h 64 --img_w 64 --img_s 320 --train_epochs 30 --datasets_folder Channel0_Zavg_allT --normalize_factor 1 --lr 0.00005 --train_datasets_size 100

My data folder contains a single tif file of dimension 1000x512x512 (time, y, x). DeepCAD seems to run as expected and outputs a noise1, noise2, and output file for each small patch in the results folder. At the end, should DeepCAD stitch these tiles into an output with the same dimensions as the input?

Unfortunately my input file is too large for upload. Let me know if there's an alternate route for sending if needed. Thanks!

Tiled output appearance

Hi cabooster,

The output .tif files appear tiled, and this makes sense as the data is split in to tiles to denoise and then reconstructed again. Is it intended to appear tiled? Do you have any tips to rectify this?

In the case below I used your model, and the recommended 64x64 tile size with 56x56 gaps. The first few frames are very obvious, but it soon corrects to a point where it is not obvious, but it is still concerning as cells can be on the border of a tile - this messes with ROI detection and relative intensity from tile to tile.

RuntimeError: stack expects each tensor to be equal size, but got [1, 150, 150, 150] at entry 0 and [1, 149, 150, 150] at entry 3

I am currently experimenting with training. I have had 2 successful training runs on small amounts of frames (3000+3000 from L4 mouse and 3000+2982 from L6 mouse). This worked and gave me models that I could use for correction. May next attempt, training on all 4 files from both layers, however, keeps failing with the following error:

$ python train.py --datasets_folder L4_L6 --datasets_path /gpfs/soma_fs/home/voit/bbo/projects/2gramfiberscope/experiments/denoising_training_data/ --n_epochs 40 --GPU 0,1,2,3 --batch_size 4 --train_datasets_size 300 --select_img_num 11400
srun: job 31197 queued and waiting for resources
srun: job 31197 has been allocated resources
Training parameters -----> 
Namespace(GPU='0,1,2,3', b1=0.5, b2=0.999, batch_size=4, datasets_folder='L4_L6', datasets_path='/gpfs/soma_fs/home/voit/bbo/projects/2gramfiberscope/experiments/denoising_training_data/', fmap=16, gap_h=75, gap_s=75, gap_w=75, img_h=150, img_s=150, img_w=150, lr=5e-05, n_epochs=40, ngpu=4, normalize_factor=1, output_dir='./results', pth_path='pth', select_img_num=11400, train_datasets_size=300)
Image list for training -----> 
Total number ----->  4
M210601JKL_20210702_D4_00002_xscancorr_rigidXCorr_export.tif
M210602JKL_20210702_D8_00006_xscancorr_rigidXCorr_export.tif
M210602JKL_20210702_D8_00003_xscancorr_rigidXCorr_export.tif
M210601JKL_20210702_D4_00005_xscancorr_rigidXCorr_export.tif
Using 4 GPU for training -----> 
Traceback (most recent call last):
  File "train.py", line 96, in <module>
    for iteration, (input, target) in enumerate(trainloader):
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1179, in _next_data
    return self._process_data(data)
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 3.
Original Traceback (most recent call last):
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 83, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 83, in <listcomp>
    return [default_collate(samples) for samples in transposed]
  File "/gpfs/soma_fs/home/voit/anaconda3/envs/deepcad/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [1, 150, 150, 150] at entry 0 and [1, 149, 150, 150] at entry 2

I have checked that all files have the same spatial dimensions, in case that is important:

$ for i in *.tif; do echo $i;tiffinfo $i|head -7; done
TIFF Directory at offset 0x241f6 (147958)
  Image Width: 269 Image Length: 275
  Bits/Sample: 16
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Samples/Pixel: 1
M210601JKL_20210702_D4_00005_xscancorr_rigidXCorr_export.tif
TIFF Directory at offset 0x241f6 (147958)
  Image Width: 269 Image Length: 275
  Bits/Sample: 16
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Samples/Pixel: 1
M210602JKL_20210702_D8_00003_xscancorr_rigidXCorr_export.tif
TIFF Directory at offset 0x241f6 (147958)
  Image Width: 269 Image Length: 275
  Bits/Sample: 16
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Samples/Pixel: 1
M210602JKL_20210702_D8_00006_xscancorr_rigidXCorr_export.tif
TIFF Directory at offset 0x241f6 (147958)
  Image Width: 269 Image Length: 275
  Bits/Sample: 16
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Samples/Pixel: 1

Despite having read issue #2 , I am not yet sure if I interpret --train_datasets_size correctly. Is this the number or the size of the stacks? Do I have to make sure that --select_img_num is divisible by that number? I also find that the choice --train_datasets_size massively influences the calculation time. What would be an expected sweet spot here?

Expected processing time, sample data

I'm having some issues getting DeepCAD to run on the GPU, I think. It would help to know how long you estimate it will take to run the tutorial data?

On my Ubuntu 20.04 system with an rtx2080Ti, it seems to take about 22 minutes to process one of the tutorial data files. Is this what you would expect?

With the supplied model and default settings.

cabooster / deepcad Goto Github PK

deepcad's People

Contributors

Stargazers

Watchers

Forkers

deepcad's Issues

Recommend Projects

Recommend Topics

Recommend Org