csyben / pyro-nn-layers Goto Github PK

Implementation of reconstruction operators in CUDA for Tensorflow for the PYRO-NN Framework

License: Apache License 2.0

Python 7.57% C++ 88.27% C 2.02% Starlark 2.14%

pyro-nn-layers's Issues

Install PYRO-NN-Layers through Anaconda?

I saw that to use PYRO-NN, it is required to build PYRO-NN-Layers into the Tensorflow package. I use Python and Tensorflow through Anaconda, and the building instructions use pip package.

Is there a way to build PYRO-NN-Layers through Anaconda on Windows? I am a python beginner, so sorry if the question is too dumb.

My system configurations are:

Windows 10.
Anaconda 3
Conda 4.7.11
Python 3.6

Thank you!

one biginner's problem

Excuse me, I just download the pyro-nn-layers code from this github , but I see a lot of ‘.cc ’ files in it. Do I need to run through these files first? I tried to run the related '.py' files in 'examples' in pyronn-master folder directly in PyCharm, but there are many errors.

Backprojector non-deterministically fails to allocate mem

I noticed, that when using the PyroNN Layers for training, sporadically the training aborts, because the layers dont seem to get the required memory. This error is displayed:

GPUassert: out of memory pyronn_layers/cc/kernels/cone_backprojector_3D_CudaKernel_hardware_interp.cu.cc 129

The responsible line of code in the pyronn-layers is here:

PYRO-NN-Layers/cc/kernels/cone_backprojector_3D_CudaKernel_hardware_interp.cu.cc

Lines 125 to 134 in 9bec5cc

    
           cudaArray *projArray; 
        
           static cudaChannelFormatDesc channelDesc = cudaCreateChannelDesc<float>(); 
        
           gpuErrchk( cudaMalloc3DArray( &projArray, &channelDesc, projExtent, cudaArrayLayered ) ); 
        
           auto pitch_ptr = make_cudaPitchedPtr( const_cast<float*>( sinogram_ptr ), 
        
                                                       detector_size.x*sizeof(float), 
        
                                                       detector_size.x, 
        
                                                       detector_size.y

Then I noticed the comments describing exactly the problem I am experiencing (I believe):

PYRO-NN-Layers/cc/kernels/cone_backprojector_3D_CudaKernel_hardware_interp.cu.cc

Lines 89 to 103 in 9bec5cc

    
           /*************** WARNING ******************./ 
        
               *  
        
               *   Tensorflow is allocating the whole GPU memory for itself and just leave a small slack memory 
        
               *   using cudaMalloc and cudaMalloc3D will allocate memory in this small slack memory ! 
        
               *   Therefore, currently only small volumes can be used (they have to fit into the slack memory which TF does not allocae !) 
        
               *  
        
               *   This is the kernel based on texture interpolation, thus, the allocations are not within the Tensorflow managed memory. 
        
               *   If memory errors occure: 
        
               *    1. start Tensorflow with less gpu memory and allow growth 
        
               *    2. switch to software-based interpolation.  
        
               *  
        
               *   TODO: use context->allocate_tmp and context->allocate_persistent instead of cudaMalloc for the projection_matrices array 
        
               *       : https://stackoverflow.com/questions/48580580/tensorflow-new-op-cuda-kernel-memory-managment 
        
               *  
        
               */

Will this TODO be resolved anytime soon? I would really appreciate it and would love to help if I can.

Cheers,
Max

csyben / pyro-nn-layers Goto Github PK

pyro-nn-layers's People

Contributors

Stargazers

Watchers

Forkers

pyro-nn-layers's Issues

Install PYRO-NN-Layers through Anaconda?

one biginner's problem

Backprojector non-deterministically fails to allocate mem

Gradient to source position and rotation matrix

Any plans for TF 2.x ?

Shape errors and reduction errors from other Tensorflow layers if PYRO-NN-Layers are within the graph

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent


	cudaArray *projArray;

	static cudaChannelFormatDesc channelDesc = cudaCreateChannelDesc<float>();
	gpuErrchk( cudaMalloc3DArray( &projArray, &channelDesc, projExtent, cudaArrayLayered ) );

	auto pitch_ptr = make_cudaPitchedPtr( const_cast<float*>( sinogram_ptr ),
	detector_size.x*sizeof(float),
	detector_size.x,
	detector_size.y

	/************* WARNING ****************./
	*
	* Tensorflow is allocating the whole GPU memory for itself and just leave a small slack memory
	* using cudaMalloc and cudaMalloc3D will allocate memory in this small slack memory !
	* Therefore, currently only small volumes can be used (they have to fit into the slack memory which TF does not allocae !)
	*
	* This is the kernel based on texture interpolation, thus, the allocations are not within the Tensorflow managed memory.
	* If memory errors occure:
	* 1. start Tensorflow with less gpu memory and allow growth
	* 2. switch to software-based interpolation.
	*
	* TODO: use context->allocate_tmp and context->allocate_persistent instead of cudaMalloc for the projection_matrices array
	* : https://stackoverflow.com/questions/48580580/tensorflow-new-op-cuda-kernel-memory-managment
	*
	*/