Comments (3)
fabioasdias
commented on August 15, 2024
My understanding is that since td is build over tf, not everything will be accessible as would be the case with "pure" tensorflow, meaning specifically that scopes are probably used by td to deal with the dynamic part of building the graph. I'm not entirely sure that the tf.variable_scope commands are compatible with the whole td.compiler thing. If they were, you wouldn't need a scopedlayer, IMHO.
A good example of that is the results of the 'word_cell' scope, that is "internally" translated to several blocks.
Of course, I'm a random internet dude, not a contributor to the project, so everything I said might be horribly wrong.
from fold.
delesley
commented on August 15, 2024
Why do you say it's ignoring the scope? The graph is showing the names of
various blocks, not the fully qualified names of the variables used by
those blocks. Each Layer you create (e.g. FC or ScopedLayer) will create
its own variable scope, nested in whatever the current variable scope is,
to hold the variables for that layer.
To be honest, what confuses me about the picture is that you have multiple
while loops, and multiple instantiations of your LSTM cells; I don't know
what's causing that.
If you want to see whether your variables have been created appropriately,
call
[v.name for v in tf.global_variables()]; that should show you the fully
qualified names of all variables.
…On Fri, Feb 24, 2017 at 2:12 PM, William Guss ***@***.***> wrote:
Hi,
I'm trying to scope certain components of my tensorflow fold model, and I
after rending it in tensorboard, it seems that its ignoring the scope in
which I construct different blocks. See this
http://i.imgur.com/Vtf9OhP.png
The associated source code which generates this graph is:
with tf.variable_scope("context_free_discriminator"):
with tf.variable_scope("embedding"):
pre_process = (td.InputTransform(lambda tokens: [tokeToVec(s, word2vec, indim) for s in tokens])
>> td.Map(td.Vector(indim)))
word_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=300), 'word_cell')
stacked_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=100), 'stacked_cell')
# word lstm convers word sequence into an lstm.
word_lstm = (pre_process >> td.RNN(word_cell))
hierarchical_lstm = (word_lstm
>> td.GetItem(0) #hidden states
>> td.RNN(stacked_cell))
embedding = (hierarchical_lstm
>> td.GetItem(1) # Final state (out, hidden)
>> td.GetItem(0) # Final output state.
>> td.FC(outdim))
with tf.variable_scope("model"):
logit_output_block = (embedding_block
>> td.FC(428)
>> td.FC(328)
>> td.FC(num_mocs, activation=None))
compiler = td.Compiler.create(logit_output_block)
logit_tensor = compiler.output_tensors
with tf.variable_scope("training"):
_desired = tf.placeholder(tf.int32)
_desired_onehot = tf.one_hot(_desired, num_mocs)
loss = cross_entropy = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(labels=_desired, logits=logit_tensor))
train_op = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#19>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGGbTWb9XH1cEmFARsGoyPrKWN7afbx5ks5rf1XZgaJpZM4MLvGB>
.
--
DeLesley Hutchins | Software Engineer | [email protected] | 505-206-0315
from fold.
moshelooks
commented on August 15, 2024
Yeah, TensorBoard graph view is not very helpful with Fold, unfortunately. Regarding scoping, if you know what you're doing you can pass in a scope as an argument to td.ScopedLayer and it will get used exactly as provided. If you pass in a string we go through some gymnastics to ensure that every ScopedLayer really does correspond to a unique scope. We do this to avoid some breakage and gotchas in the TF variable scoping mechanism. What Delesley said is a very good suggestion for making sure the right variables are being created. If you are observing undersharing (e.g. a single scoped layer that encapsulates variables that are being created more than once) that is bug, please do report it. But the view of the graph in tensorboard is unfortunately not sufficient to figure out what's going on.
Regarding the while loop, I'm confused; Fold certainly only creates one while loop per compiler; maybe you ran Block.eval() somewhere? (eval() is only for messing around at the repl and unit tests; it shouldn't be used in "real" models)
from fold.
Related Issues (20)
- NEAT Algorithm implementation HOT 1
- How do I get the root embedding tensors after training the model? HOT 1
- Is it still live?
- Compatability with Tensorflow 2.4+ HOT 3
- Not able to find GPU version of Fold
- can current fold work with tensorflow 1.3? Is it tested? HOT 8
- How to use fold in serving? HOT 2
- Fold & Eager HOT 3
- "undefined symbol" caused by building fold from source HOT 3
- Not compiled to use: SSE4.1 SSE4.2
- Is Fold not compatible with tensorflow-1.4.0? HOT 1
- Typos in sample
- using tf.constant() in td.Composition HOT 1
- "import tensorflow_fold" problem HOT 7
- How to build child-sum tree using tensorflow fold? HOT 2
- How to print intermediate outputs in recursion in TF Fold? HOT 4
- Printing predictions HOT 7
- Making Windows compatible HOT 2
- tensorflow_fold-0.0.1-cp27-cp27mu-manylinux1_x86_64.whl is not a supported wheel on this platform. HOT 1
- fold incompatible with TF-gpu 1.10.1 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fold.