Comments (3)
The DenseNet models require tons of memory. However, I was able to train the model on a 980M with 4GB of GPU memory, on Theano, with garbage collection enabled to allow for such a large model to run.
I don't see a reason why TF should run out of memory with 8 GB gpu ram, but then I also experienced an OOM when I used the TF backend with 4GB of ram. Perhaps, 4GB is simply not enough to load such a model without GC. When I dropped the batch size to 16 instead of 64, it seemed to work at about the same speed as theano for me.
As to the ImageNet version, I think the model trained on imagenet was the Bottleneck Compressed version of the networks, which have a similar number of parameters (especially the DenseNet-BC-190-40). The authors do mention that they use a batch size of 128 for ImageNet due to GPU memory constraints (see page 7 of the paper, just a bit above the Discussion paragraph), but I do not understand how they can accomplish this on a single GPU (I doubt even a Titan X with 12GB gpu memory can handle this model with such a large batch size for ImageNet images (224x224)).
from densenet.
Hmm, okay good to at least know you've seen something similar. Perhaps if they had 8x titan X machines it might be achievable, assuming a single batch can be distributed across GPUs. The last author is at Facebook so it is conceivable, though I'd expect an imagenet network to be much larger.
from densenet.
Almost all ImageNet results are generally tested on multiple GPU systems, dut to the 1.2 million images taking several weeks on even the most powerful GPUs.
DenseNets definitely are an efficient architecture in comparison to ResNets, but the author's show that the performance of even the largest DenseNet -161 with k=48 has a top 1 accuracy of approximately 78%, whereas Inception ResNet V2 surpasses that at 80.4%, albeit at the cost of a very large number of parameters.
from densenet.
Related Issues (20)
- Cifar10 weights HOT 1
- no longer works with newest keras HOT 8
- About implementation of __dense_block HOT 2
- Running Densnet in CPU HOT 1
- Why reshape in fc-densenet in top layer before applying softmax? HOT 2
- AttributeError: 'NoneType' object has no attribute 'get_file' HOT 2
- Where is connection to 12 layers of each dense block? HOT 1
- About DenseNet HOT 1
- About Densenet architecture HOT 1
- Plans for memory efficient implementation in Keras? HOT 1
- How to upload my own dataset instead of the Cifar10 dataset HOT 1
- How to use 'DENSENET_121_WEIGHTS_PATH_NO_TOP'? HOT 1
- Mistake in L2 regularization HOT 1
- creat model HOT 2
- creat model HOT 1
- imagenet_inference
- inter_channel HOT 1
- Poor CIFAR100 accuracy HOT 1
- Error about normalize_data_format
- Unable to convert to frozen graph or checkpoint
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from densenet.