tensorflow / lucid Goto Github PK
View Code? Open in Web Editor NEWA collection of infrastructure and tools for research in neural network interpretability.
License: Apache License 2.0
A collection of infrastructure and tools for research in neural network interpretability.
License: Apache License 2.0
Typical TF import:
import tensroflow as tf
Typical lucid import:
import lucid.misc.io.showing as show
from lucid.misc.io.loading import load
from lucid.optvis import objectives
from lucid.optvis import render
from lucid.optvis import style
from lucid.optvis.param.random import image_sample
Hi, first of all thank you for the great work. :)
I have a question: What does it mean if I get multiple images per threshold from render_vis()
? The numpy array vis
in this line has a shape of (1, 128, 128, 3) in the tutorial. With a custom model, however, I got the shape (64, 100, 221, 6). It corresponds to a height of 100, width of 221, 6 channels, I get that. But why am I getting 64 images?
Thank you.
Maybe inline comments & readme files are enough for now.
I'd really like to have a convenient way to evaluate a model once without going through the whole rigamarole of setting up a graph.
model.isolated_eval(layer, input)
.We need to get Done!model.labels
back.
It might make sense to attach expensive, precomputed assets to models. For example feature visualization spritemaps.
model.precomputed.vis_spritemaps["mixed4d"]
gives me a url?I'm trying to open an autoencoder model I've trained myself, on Lucid, and I'm using as reference the notebook Importing a graph into modelzoo.
I'm mostly in doubt on how to use the provided class:
model_path = 'nasnet_mobile_graphdef_frozen.pb.modelzoo'
image_shape = [224, 224, 3]
image_value_range = (0, 1)
input_name = 'input'
What should I define as image_shape
, image_value_range
? For what images I'm considering this? The output of a certain convolutional layer?
Also, for what is defined the input_name
?
transform
vs objectives
I feel there are good arguments either way, but I think we should decide on using either plural or singular for these modules.
This could also be a good test case (because non-essential) for a deprecation strategy. I can think of aliasing the module in an __init__
file and issuing a DeprecationWarning
We seem to have disabled showing images during the render_vis process. I would favor reinstating this, using @enjalot's fantastic misc.show.images()
.
It only uses general iPython stuff, so it should work in both colab and other jupyter/ipython frameworks. I think on the command line it will just null, although we should test it.
Are any of the tensorflow standard tools for freezing graph definitions compatible with our usecase?
My current understanding is that we mainly require all Variable
s to be turned into constants, so we avoid accidentally optimizing the model rather than the input. Are there additional considerations, @colah @znah ?
The documentation says read
has a mode argument that defaults to 'rb'
:
https://github.com/tensorflow/lucid/blob/master/lucid/misc/io/reading.py#L56
Unfortunately, there is no such mode
argument. The lack of mode is unfortunate, since I need to set it to 'r'
to get svelte to work in Python 3.
Link in README.md references: https://github.com/tensorflow/lucid/blob/master
Should be: https://github.com/tensorflow/lucid
-- or --
Should be: https://distill.pub/2018/building-blocks/
It looks like modelzoo could directly support the new SavedModel standard. We would still need the metadata entries in modelzoo, but no longer require manual freezing of Variables into Constants in the graph definition.
~61%~~64%~70%
Remaining:
lucid/optvis/transform.py 74 57 23%
lucid/optvis/objectives.py 203 133 34%
lucid/optvis/param/lowres.py 23 12 48%
lucid/misc/environment.py 14 7 50%
lucid/misc/io/showing.py 47 17 64%
lucid/optvis/render.py 90 22 76%
lucid/optvis/param/spatial.py 37 8 78%
lucid/misc/io/saving.py 41 7 83%
lucid/misc/io/serialize_array.py 47 8 83%
lucid/misc/io/reading.py 58 9 84%
lucid/optvis/param/resize_bilinear_nd.py 37 6 84%
lucid/misc/channel_reducer.py 26 3 88%
lucid/optvis/param/images.py 17 2 88%
lucid/misc/io/writing.py 29 3 90%
lucid/optvis/param/color.py 21 2 90%
lucid/misc/gradient_override.py 36 3 92%
lucid/modelzoo/vision_base.py 30 1 97%
lucid/misc/io/loading.py 46 1 98%
In this issue, we'll discusses some weaknesses of Lucid's present abstractions. We also present a couple ideas for possible alternatives, but don't presently have a strong view on the right path forward.
(A lot of these thoughts were developed in conversation with @ludwigschubert.)
A lot of the weird stuff about Lucid comes from us having different needs that most TF users. A normal TF workflow looks something like "define a graph, then train it for a while". Our needs are often very different: create one graph for 30 seconds, then throw it away. Create another similar graph, then throw it away too. Moreover, these graphs often have a composable structure, where we want to be able to talk about parts of the graph independent of a particular instantiation and use it over and over again.
At a very high level, Lucid's answer is to use closures. For example, when I declare an objective:
obj = objectives.channel("mixed4a", 37)
I'm approximately creating the closure:
obj = lambda T: tf.reduce_sum(T("mixed4a")[..., 37])
Where T
is an accessor, kind of like $
in jQuery: it allows you to conveniently access lots of things you might need, without writing a lot of code.
(This isn't quite true: the closure is actually further wrapped with in a convenience Objective
object, which allows them to be added and multiplied without explicitly escaping the closure. More on this in a minute.)
At a very high level, I believe that closures like this are the right abstraction for us. However, I also have a few concerns:
T
is a bit confused and may make things unnecessarily centralized.DeferredTensor
class for these kinds of closures.T
Generally, our closures take an argument T
, a special function which allows them to access things like neural networks layers:
obj = lambda T: ... T("mixed4a") ...
The fact that T
is getting passed in as an argument might make you believe it has a lot of special state. That's true to some extent -- it's kind of a grabbag of things -- but in it's main usage it equivalent to something like:
def T(name):
return tf.get_default_graph().get_tensor_by_name("import/" + name)
Thank goodness we don't have to type that all the time, it's quite a mouthful! But it could just as easily be global.
A little bit more state comes from us wanting T
to have special names for some nodes, to make them more user friendly. For the most part, this comes from the imported model. From that perspective, it might be more intuitive to do something like model["mixed4a"]
instead of T("mixed4a")
.
There are some things, like the pre-transformation input or the global step that we probably want to get from somewhere else. That said, we could probably do something like render.global_step()
if we wanted to get rid of T
.
So, what are the pros/cons of of the present T
closure arg set up?
Pros: It seems to me that the biggest one is actually preventing users from shooting themselves in the foot by trying to access graphs that haven't been imported or constructed.
Cons: Centralized and annoying to extend; passing around unnecessary variables; alternative setups might make error messages / debugging better.
Before we talk about APIs for dealing with our closures, I'd like to clarify their role a bit:
T
issue.(Observation: These deferring closures form a monad. Most of the interesting API options are ways of reifying Functor/Monad operations in Python.)
Broadly, there are three ways of handling the closures we create:
param_f = lambda: param.image(...)
This is the most transparent option, but can be a bit tedious.
obj = lambda: channel(..., 3)() + channel(..., 4)()
Objective
right now. This allows us to have the above just be:obj = channel(..., 3) + channel(..., 4)
In it's most general sense, this would suggest a kind of DeferredTensor
object. As we'll discuss shortly, this might have a number of interesting benefits.
My sense is that we should either do 1 or 3, and that 2 is a kind of unhappy intermediate version. Ideally, it would be nice for all of lucid to be consistent in our choices here.
There are a number of interesting benefits the arise from option 3 (DeferredTensor
):
A lot of error checking could be done at creation of the DeferredTensor
object.
T
to model, model[layer]
could check if the appropriate layer exists in the model before generating a DeferredTensor
object.DeferredTensor
could carry additional meta data.
DeferredTensor
could provide operator overloading / automatic coercion / etc to make the closures more convenient to manipulate. This carries the risk of becoming less transparent / confusing / abstraction creep.
Tensor
s if we wanted by registering with tensorflow. But I worry this would open the door to lots of really annoying subtle bugs, where someone accidentally coerces to a TF Tensor
in the wrong context and then people get surprised with graph mixing errors later on.If we went with the closures are user responsibility route, but got rid of T
:
# Get layers by indexing model instead of accessing T:
obj = lambda: L2(model["mixed4a", ..., 37])
If we went the DeferredTensor
route (and also got rid of T
):
# Deferred tensor convenience functions
obj = model["mixed4a", ..., 37].L2
# No model arg to render_vis -- it can be inferred
render_vis(obj)
# Evaluate layers in a standalone way:
model["mixed4a"].isolated_eval(...)
Hey,
I wanted to visualize a custom InceptionV3 that I trained, but there were no interpretable Visualizations.
Therefore, I visualized the Keras InceptionV3, trained on Imagenet. But also with this model, I could not find any good visualizations. The closest one is mixed6/concat
. To reproduce my problem, i created a Colab-Notebook:
https://colab.research.google.com/drive/1oIEeHZWxyU3vsqFpaQG1rs4YjHyYzfa1
If anyone knows, what I am doing wrong, or what the problem is, it would be great !
Initially noted as render_viz
being hard to test, I'd like to rethink some of its output.
At the moment we have no way of accessing the numerical values of the objectives
or print_objectives
.
I could see the results of render_viz
containing all of this metadata.
This could fit into a configurable stopping criterion, with defaults being based on thresholds
, but also offer running to convergence (in some default metric) or running to an arbitrary stopping criteria lambda function. This function could have access to the objective values/metadata described above and, for example, allow optimizing until a certain class probability becomes bigger than another class's probability, etc.
We may need to ask the tensorflow maintainers?
Not sure if this really makes sense, but a blur transform could be an interesting addition for lucid.optvis.transform - maybe it would allow to influence the behaviour of the high frequency patterns and/or the general scale of features.
lucid.optivis.param.image_sample is missing.
It is used by:
lucid.optivis.transform.collapse_alpha_random
Hi, do we have an example of loading a tensorflow model some where in the docs already? If not yet, can you provide one? thanks
Let's assert the tensor shape coming into collapse alpha to be known at graph creation time.
Currently you can get an exception if you ask for a collapse_alpha
transform to be applied after a transform that stochastically changes t_image
's shape (such as random_scale
).
Should have a specific link when you mention this:
"Check out other notebooks on how to make your own neuron visualizations."
It should link to this:
https://colab.sandbox.google.com/github/tensorflow/lucid/blob/master/notebooks/tutorial.ipynb
In https://github.com/tensorflow/lucid/blob/master/notebooks/building-blocks/SemanticDictionary.ipynb there is a dead link in the section "Spritemaps" (https://storage.googleapis.com/lucid-static/building-blocks/sprite_mixed4d_channel.jpeg).
This is what I get instead of the actual image:
<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
Anonymous caller does not have storage.objects.get access to lucid-static/building-blocks/sprite_mixed4d_channel.jpeg.
</Details>
</Error>
I would appreciate a working link. No biggy though.
Currently param_f
is the only argument to render that users manually need to wrap in a lambda.
I understand we need the lambda so we can create the input on the same graph as the model, but I think we should be able to return lambdas from our convenience constructors such as param.images.image
.
Happy to handle the coding if @colah agrees this is a good idea, but not confident that it is.
(I haven't created many custom parameterizations so far.)
Does lucid support to make my own TensorFlow model visualizations now?And how to make my own channel spritemaps?
Thanks!
In version 1.8.0 tensorflow switched to using the low-level C-API calls for graph construction. This breaks how we currently import graphs.
We are looking into fixes; for now use tensorflow 1.7.0 or earlier only—or set TF_C_API_GRAPH_CONSTRUCTION=0
as an env variable.
Blocked by #3
We should return np.float32
When np.int defaults to int32, 1e10 is out of bounds. Error message below, PR on the way.
~\MYPATH\lucid\misc\gradient_override.py in register_to_random_name(grad_f)
68 String that gradient function was registered to.
69 """
---> 70 grad_f_name = grad_f.__name__ + "_" + hex(np.random.randint(0, 1e10))[2:]
71 tf.RegisterGradient(grad_f_name)(grad_f)
72 return grad_f_name
mtrand.pyx in mtrand.RandomState.randint()
ValueError: high is out of bounds for int32
After each iteration in model training, whether and how can I compare the iteration results in the form of images using Lucid ?
When I run this demo, I can't get the predicted picture
, and when I open this demo in jupyter, there is another wrong that shows .
So, to show the picture in Terminal, we can
import matplotlib.pyplot as plt
then:
Note the green line and add those three lines above the green one.
At this time, we obtain the demo:
I wish to pull a request to fix this bug which is not a bug..LOL
Things I'd like; maybe together with the asset loader:
I'd like to be able to tell the asset loader to go ahead and request my asset. For example on hovering over an example chooser.
We added the grey loading screen to invalidate the current sprite so we don't show outdated images while the new ones are loading. This works great, but would be even better if it only set the grey loading background after a short delay, such as 100ms to prevent flashing.
I wonder how could i visualize an own model? Is it there an example for this?
Once we're public, I'd like to take on measuring code coverage (probably abysmal atm) and integrate sth. like Coveralls (free for open source software)
Let's discuss whether you want this, too, here! :-)
When I use lucid on my on pc (in a notebook) it doesn't use a gpu. Is there some command I should use with render_vis so it knows to use a gpu?
Please close if there's a banal explanation for this, but I noticed that the InceptionV1
labels do not match the labels according to the tensorflow/models
ImageNet data preprocessing.
In lucid,
>>> model = models.InceptionV1()
>>> model.load_graphdef()
>>> model.labels[:10]
['dummy',
'kit fox',
'English setter',
'Siberian husky',
'Australian terrier',
'English springer',
'grey whale',
'lesser panda',
'Egyptian cat',
'ibex']
In the download of ImageNet, I see
% head -n 10 <...>/imagenet-tfrecords/labels.txt
0:background
1:tench, Tinca tinca
2:goldfish, Carassius auratus
3:great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias
4:tiger shark, Galeocerdo cuvieri
5:hammerhead, hammerhead shark
6:electric ray, crampfish, numbfish, torpedo
7:stingray
8:cock
9:hen
Is one of these the standard?
Here the 'import/BiasAdd' does not have gradient. But 'import/BiasAdd/(BiasAdd) might have the gradient. While backprop, lucid is taking 'import/BiasAdd' and it returns the above 'No gradient' error. I have built a keras model ( in .h5) then converted it to (.pb). Now I am trying to get relevance for each channel of my CNN. But the gradient is not back propagating because of the error. Please help me in finding the relevance.
Would it theoretically be possible to create an objective that uses either a control map or a callback function that changes the channel objective on a coordinate-based criterion? A simple example would be a vertical gradient between two channels.
I guess this would be computationally expensive, but maybe there are also reasons why this is not possible at all?
/srv/venv/lib/python3.6/site-packages/lucid/scratch/web/svelte.py in build_svelte(html_fname)
33 print(subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT))
34 except subprocess.CalledProcessError as exception:
---> 35 print("svelte build failed!\n" + exception.output)
36 return js_fname
37
TypeError: must be str, not bytes
When using a notebooks/building-blocks/SemanticDictionary.ipynb which manually installs 0.0.5 (not the latest commit where that has been fixed)
File "/srv/venv/lib/python3.6/site-packages/lucid/scratch/web/svelte.py", line 31
print cmd
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(t cmd)?
tests/optvis/param/test_unit_balls.py
occasionally fails in the shape=(2, 5, 5), precondition=True all cases for L1 balls with tf.reduce_max values observed as high as 1.08 1.3.
Hi, thanks for provding the powerful visualization. I just wonder can we also apply lucid on other models such as VAE or GAN? If not, is there any possible alternative method to ahieve this goal? Thanks.
User feedback on the lucid tutorial notebook:
It's a great tutorial. It would be great if you could add a line to view the pre - trained model architecture so that we have idea of names of different layer-filters. I tried multiple ways but with Lucid model zoo I was not able to view complete architecture.
I tried checking model.summary() etc.
Maybe the documentation is outdated, but it looks like several of the examples are using methods that do not seem to be implemented:
https://github.com/tensorflow/lucid/tree/master/lucid/optvis
e.g.
obj = objectives.channel("mixed4a_pre_relu", 2)
param_f = lambda: tf.concat([
param.rgb_sigmoid(param.naive([1, 128, 128, 3])),
param.fancy_colors(param.naive([1, 128, 128, 8])/1.3),
param.rgb_sigmoid(param.laplacian_pyramid([1, 128, 128, 3])/2.),
param.fancy_colors(param.laplacian_pyramid([1, 128, 128, 8])/2./1.3),
], 0)
render_vis(model, obj, param_f)
Hey @colah @znah ;
should we consider automatically calling load_graphdef
when instantiating a modelzoo class?
To me this boils down to:
What can you currently do with an instantiated modelzoo.Model
, that you couldn't do with just the class?
It feels like this could simplify the current API, but it may also hide the fact that a graph definition may need to be downloaded.
Looking forward to your opinions!
In the module lucid.misc.io.showing
:
No support for w
to specify output width. showing.image()
and showing.images()
take w
arguments but the don't do anything. showing.show()
does not take a w
argument.
To fix this, _image_url()
should probably take a w
argument that causes appropriate zooming if set.
showing.show()
takes a domain argument, but on the images()
path it has not effect.
This is simply because images()
doesn't pass domain through to _image_url()
:
url = _image_url(array)
I am considering adding a code quality reporter once we're a public repo.
The main objection would be that its extra work to configure + we may not always agree with default stylistic choices. For example, most linters will flag our usage of the T
function as (technically correct) unidiomatic.
When i try to use lucid on my own network the output of a lot of filters is just gray (there are some pixels that vary however the returned image is gray for a human. The deeper the layer the more filters are grey (in the highest layers everything returned is completely grey). Is there something I am doing wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.