Comments (4)
Hi, @IlvaHou . Thank you for letting us know that.
It seems the cause of problem is in tf-keras-vis
, but I could NOT reproduce the error.
Could you please submit the stack trace of the error and the code snippet that reproduce the problem?
Thanks!
from tf-keras-vis.
This is the error.
For the code it might be hard to reproduce because I load pretrained weights in the model, but here is something:
DEFINE MODEL
import tensorflow.keras.backend as K
from tensorflow.keras.layers import Input, concatenate, Dense, Flatten, Conv2D, MaxPooling2D, AveragePooling2D, BatchNormalization, Dropout
from tensorflow.keras.models import Model
#Clear session
K.clear_session()
#L2-regularization
L2_weight = 0.00004
#Image branche
#Block 1
img_input = Input(shape=(250,250,1))
x_img = Conv2D(32, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(img_input)
x_img = BatchNormalization()(x_img)
x_img = Conv2D(32, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = Conv2D(64, kernel_size=(3,3), padding='same', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid')(x_img)
#Block 2
x_img = Conv2D(80, kernel_size=(1,1), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = Conv2D(192, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid')(x_img)
#Block 3
x_img = Conv2D(512, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = MaxPooling2D(pool_size=(1,1), strides=(2,2), padding='valid')(x_img)
#Block 4
x_img = Conv2D(128, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = MaxPooling2D(pool_size=(1,1), strides=(2,2), padding='valid')(x_img)
#Block 5
x_img = Conv2D(256, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = Conv2D(256, kernel_size=(5,5), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_img)
x_img = BatchNormalization()(x_img)
x_img = AveragePooling2D(pool_size=(8,8), padding='valid')(x_img)
img_output = Flatten()(x_img)
#Gender branch
gen_input = Input(shape=(1,))
gen_output = Dense(16, activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(gen_input)
#Concatenate
x_conc = concatenate([img_output, gen_output])
out1 = Dense(128, activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(x_conc)
out1 = Dropout(0.5)(out1)
out1 = Dense(64, activation='relu', kernel_regularizer=tf.keras.regularizers.l2(L2_weight))(out1)
out1 = Dropout(0.5)(out1)
out = Dense(1, activation='linear')(out1)
model = Model([img_input, gen_input],out)
SHOW INPUT INSTANCE
An example input X looks like this:
[array([[[[1. ],
[1. ],
[1. ],
...,
[0.227451 ],
[0.227451 ],
[0.38431376]],
[[0.22352943],
[0.18039216],
[0.16862746],
...,
[0.227451 ],
[0.21960786],
[0.4156863 ]],
[[0.2392157 ],
[0.18431373],
[0.17254902],
...,
[0.21568629],
[0.23529413],
[0.4156863 ]],
...,
[[0.61960787],
[0.18823531],
[0.16470589],
...,
[0.1764706 ],
[0.18431373],
[0.22352943]],
[[0.5137255 ],
[0.20000002],
[0.16862746],
...,
[0.19215688],
[0.20000002],
[0.2392157 ]],
[[0.54509807],
[0.21960786],
[0.20784315],
...,
[0.22352943],
[0.22352943],
[0.23529413]]]], dtype=float32), array([1])]
TRY GRADCAMPLUSPLUS
Like mentioned above, I tried using GradCAMPlusPLus using the following code:
from tf_keras_vis.gradcam import GradcamPlusPlus
def loss(output):
return (output-168.93)
gradcam = GradcamPlusPlus(model)
cam = gradcam(loss, X, penultimate_layer=21)
Where for convenience I put down the ground truth value in the loss function manually and also the penultimate_layer.
One big side-note is that I tried this using TensorFlow version 1.14 (because that is with which I wrote the architecture of my model). And I did not receive any errors complaining about version, so I pursued. But this might become a bottleneck somewhere.
Thank you for checking!
from tf-keras-vis.
I found a way to achieve the result that I want without the use of this library. However, I think I made a custom version of regression activation mapping (as described in this paper: https://arxiv.org/pdf/1703.10757.pdf), instead of gradient activation mapping:
Get output of final convolutional layer
before_gap_layer = model.get_layer('activation_7')
before_gap_layer_output = Model(inputs=model.input, outputs=before_gap_layer.output)
before_gap_output = before_gap_layer_output.predict(X) # shape is (1,31,31,256)
Get weights of first dense layer
first_dense_layer = model.get_layer('dense_1')
first_dense_layer_weights = first_dense_layer.get_weights()[0]
first_dense_layer_weights_image = first_dense_layer_weights[0:256,:] # since this layer is based on concatenation of image information and gender information, we are only interested in the first 256 weights coming from the image branche # shape is (256,64)
Get weights of second dense layer
second_dense_layer = model.get_layer('dense_2')
second_dense_layer_weights = second_dense_layer.get_weights()[0] # shape is (64,1)
Combine weights to find attribution of convolutional layer to final prediction
total_weights = np.dot(first_dense_layer_weights_image, second_dense_layer_weights) # shape is (256,1)
Apply weights to get weighted sum of convolutional output as RAM
before_gap_layer_weighted = before_gap_output # initialize
for i, val in enumerate(total_weights):
before_gap_layer_weighted[: ,: ,: ,i] = np.multiply(before_gap_output[: ,: ,: ,i],val[0]) # shape is (1, 31, 31, 256)
ram = np.sum(before_gap_layer_weighted[0], axis=2)
Rescale and overlay to original image
from scipy.ndimage import zoom
ram = zoom(ram, (16.12, 16.12), order=1) # round(31x16.12)=500 is shape of original image
fix, ax = plt.subplots()
ax.imshow(X[0][0], alpha=0.5)
ax.imshow(ram, cmap='jet', alpha=0.5)
plt.show()
(Image is a bit dark, but that has to do with the original image, not with this method)
But still if you know how to incorporate the gradients to create a grad-RAM for this model, please let me know!
from tf-keras-vis.
Hi, @IlvaHou . I'm sorry for the late reply.
One big side-note is that I tried this using TensorFlow version 1.14 (because that is with which I wrote the architecture of my model).
I regret that I have to say that tf-keras-vis
is NOT supported Tensorflow 1.x, but only Tensorflow 2.x.
The cause of the error is that.
tf-keras-vis
has been derived from keras-vis.
I've developed tf-keras-vis
because it was difficult to improve and maintain keras-vis to support TF2.x.
So, when you use TF1.x, you have to select keras-vis.
I found a way to achieve the result that I want without the use of this library.
Thank you for letting us the interesting paper!
I'm going to read it.
I will close this issue, but if you have any question, please feel free to reopen this.
Thanks!
from tf-keras-vis.
Related Issues (20)
- 'NoneType' object has no attribute 'ndim' HOT 1
- AttributeError: 'NoneType' object has no attribute 'ndim' HOT 1
- 'Tensor' object has no attribute 'ndim' HOT 1
- scorecam seems to be weighting all feature maps equally
- How to output the visualization intermediate layer
- TypeError: '<=' not supported between instances of 'int' and 'str'
- Add a Citation widget to the repo HOT 1
- Feature request: Support Python 3.10 HOT 4
- if vision transformers are supported ?
- cannot use BinaryScore HOT 3
- How to use the package with one dimensional data? HOT 1
- Problems with TF 2.9 HOT 2
- Incorrect implementation of LayerCam HOT 1
- It does not work with models with custom layers. HOT 3
- Reconstruction Error Score? HOT 2
- 'Tensor' object has no attribute 'ndim' HOT 5
- Tutorials on using 3D data
- GradCAM does not detect correct convolutional layer in multi-input case HOT 1
- Is it possible to compute the full inference at the same time as the activations? HOT 2
- value_range param is ignored
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tf-keras-vis.