Comments (9)
It doesn't really run the 'whole' graph 3 times for 1 loop, it only runs the computation subgraph necessary for every operation you run (So calling cost
will not compute gradients like optimizer
op does). But considering calling run
need to initialize some parameters, result = sess.run([optimizer, cost, merged_summary_op])
should be more efficient (as it only calls run once vs 3 before). But it shouldn't be a so big difference.
And about only using sess.run([cost,merged_summary_op]), do you run optimizer
op before or not?
from tensorflow-examples.
Results are correct, because you haven't started to train (by running the optimizer op) your network has random weights, so it give you such results. You can see accuracy is ~0.1, which is same as a random guess out of 10 classes of MNIST.
You have to use optimizer to train your model (by reducing the cost, applying backpropagation algorithm and updating variable weights). The cost
tensor only calculate the loss value, it doesn't train model.
from tensorflow-examples.
I think it would not work, if we only do sess.run([optimizer]), and "cost" has to be there.
For example, sess.run([optimizer, cost]).
I am really confused because "cost" is already included in "optimizer", why can not we just use "optimizer"?
from tensorflow-examples.
@aymericdamien Thank you for your explanation. For your last question, I only use sess.run([cost,merged_summary_op]) and there is no sess.run(optimizer) before or after. And I wonder how to run the whole graph?
from tensorflow-examples.
@aymericdamien I tried just run sess.run([cost]), here is code and result, obviously, the result is incorrect.
I think it makes sense and it shouldn't work. As you said, sess.run([cost]) only runs the subgraph. So only cost op is calculated.
from tensorflow-examples.
I am late, but yes. When you run a session you just send the parameters to the function that fetches and
feed_dict just load the parameters in the placeholders that the function you are running will use. I find it easier to see at it as a C function receiving a pointer. So basically it would be like a function witch takes a function and a structure and later it makes a pointer to the data and passes it to the function provided. Not really intuitive and pythonic but that is tf.
from tensorflow-examples.
I think this should works... Can you try to just run sess.run([cost])?
from tensorflow-examples.
@jmgrn61 I think this may answer your question.
tensorflow/tensorflow#13133 (comment)
from tensorflow-examples.
@jmgrn61 In case if you are using an Iterator then calling sess.run(cost) will give u the next batch of images and labels from the iterator. Same occurs if you call sess.run(optimizer). Thus, i think that if you call these two separately then its actually showing the cost of batch 'n' and optimizing based on the batch 'n+1' therefore completely skipping the 'nth' batch. So i always use them in same sess.run().
I also tested this and found out that iterations reduced when using them in separate sess.run() as 2 batches were always getting skipped in each Loop (1st skip on sess.run(loss), 2nd skip on sess.run(accuracy). Only set of images which is being trained is at sess.run(optimizer). Thus i am skipping every 2 batches per loop, meaning i am only training on 1/3 of my total training dataset.
Iterations when using l,_,acc = sess.run([loss,optimizer,accuracy]) : 42000 (exactly 3 times of this case)
from tensorflow-examples.
Related Issues (20)
- 404 not found
- Please provide a example for stacked bidirectional LSTM for Tensorflow 2.x
- [Potential NAN bug] Loss may become NAN during training HOT 1
- Tensor
- How to get prediction code ?
- fixes for Word2Vec for Python 3
- ml_introduction.ipynb Links
- InternalError: Dst tensor is not initialized. [[{{node IteratorGetNext/_2}}]] [Op:__inference_distributed_function_24557]
- In the tf1 example: I replace the weigtht and bias with tf.layers.dense, I found the accuracy decrease. why??? HOT 1
- The CNN example diagram shows 3 conv & pooling layers but the implementation only has 2
- AttributeError on placeholder HOT 1
- fig
- In K-Means Example, when i am running "from tensorflow.contrib.factorization import KMeans" line, i am getting an error "ModuleNotFoundError: No module named 'tensorflow.contrib'" HOT 2
- possible issue at: tensorflow_v2/notebooks/3_NeuralNetworks/autoencoder.ipynb HOT 6
- Add a development container HOT 3
- Ikvvh
- TPU Usage
- Activities HOT 3
- Offo HOT 3
- How to autotune simple MLP onnx model on hand dataset ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorflow-examples.