Comments (3)
By default, Tensorflow will use all available GPUs, so to use some specific gpu, you have to use "CUDA_VISIBLE_DEVICES" flag, and specify all device ids that you want to use.
For example: export CUDA_VISIBLE_DEVICES="0,1"
from tensorflow-examples.
Thanks for your response. This works!
I have one more question. My goal here is to be able to run Alexnet code on multiple GPUs.
With multi-GPU setting (CUDA_VISIBLE_DEVICE flag on), it allocates on all specified devices (GPU Memory usage is high on Nvidia-smi), but does not do computations (no memory utilization from Nvidia-smi while the program is running) on none except one. Any suggestion to make it truly parallel?
from tensorflow-examples.
Yes, because you have to allocate yourself operations to specific gpu, Tensorflow can't do it automatically.
with tf.device('/gpu:0'):
...
with tf.device('/gpu:1'):
...
But in order to efficiently use multiple GPU to train neural networks, you have to coordinate training. I suggest you check the official tutorial, they update parameters synchronously after each GPU finished to process a batch of data.
https://www.tensorflow.org/versions/r0.7/tutorials/deep_cnn/index.html#training-a-model-using-multiple-gpu-cards
from tensorflow-examples.
Related Issues (20)
- 404 not found
- Please provide a example for stacked bidirectional LSTM for Tensorflow 2.x
- [Potential NAN bug] Loss may become NAN during training HOT 1
- Tensor
- How to get prediction code ?
- fixes for Word2Vec for Python 3
- ml_introduction.ipynb Links
- InternalError: Dst tensor is not initialized. [[{{node IteratorGetNext/_2}}]] [Op:__inference_distributed_function_24557]
- In the tf1 example: I replace the weigtht and bias with tf.layers.dense, I found the accuracy decrease. why??? HOT 1
- The CNN example diagram shows 3 conv & pooling layers but the implementation only has 2
- AttributeError on placeholder HOT 1
- fig
- In K-Means Example, when i am running "from tensorflow.contrib.factorization import KMeans" line, i am getting an error "ModuleNotFoundError: No module named 'tensorflow.contrib'" HOT 2
- possible issue at: tensorflow_v2/notebooks/3_NeuralNetworks/autoencoder.ipynb HOT 6
- Add a development container HOT 3
- Ikvvh
- TPU Usage
- Activities HOT 3
- Offo HOT 3
- How to autotune simple MLP onnx model on hand dataset ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorflow-examples.