Comments (14)
I got even better result by finding out that there is a
prev_layer = Activation('relu')(prev_layer)
lost at the top of
def residual_short(prev_layer, level, pad=1, lvl=1, sub_lvl=1, modify_stride=False):
And also the epsilon setting of bn_layer has to be 1e-5, which is different from the default setting of Keras 1e-3.
Anyway, the result is much better than before, but still has many miss predictions now. I think is should be the difference between keras bn can caffe bn layer.
You can have a look at the example in my fork.
from pspnet-keras-tensorflow.
Hi @wcy940418 . I havent found any implementation details about this processing in the original paper. As I dont have any experience in matlab, from code it seems that image is not actually flipped, but changed order of RGB channels to possible BGR... and then in score addition its changed back. In my pycaffe code i didnt do that "flipping" and it actually works pretty fine.
Will return to working with repo soon.
from pspnet-keras-tensorflow.
Thank you @Vladkryvoruchko , I see. I will try to figure out the missing detail in the original network.
from pspnet-keras-tensorflow.
@wcy940418 Thank you. Keep me up to date if you proceed :)
from pspnet-keras-tensorflow.
@Vladkryvoruchko How did you modify the caffe-tensorflow code to port out bn layer parameters? And I tested using pycaffe to predict image, it just works fine as you said.
from pspnet-keras-tensorflow.
Also, I changed some lines in your code, now I get some better results, but not perfect. Could you tell me the order you convert scale, offset, mean, variance from caffemodel?
from pspnet-keras-tensorflow.
@wcy940418 i will now upload in this repo original caffe-tensorflow code, and then commit modifications.
EDIT:
I'll try to have a look at the converter code again later, and uploaded converter in this repo
from pspnet-keras-tensorflow.
Hi @Vladkryvoruchko , I found a snippet in your caffe-tensorflow code to port out bn params. In kaffe/transformers.py line 258:
# Prescale the stats | scaling_factor = 1.0 / scale if scale != 0 else 0 | mean *= scaling_factor | variance *= scaling_factor
I am not sure mean and variance need to be prescaled. I think the meaning of four params of bn layer are identical in both caffe and tensorflow: scale * (data - mean) / variance + offset
from pspnet-keras-tensorflow.
@wcy940418 Yea, but this snippet is from original converter and should work just fine. If we look on the tutorial of converting DilatedNet(link) there is also supposed to be BN converting the same way and it works fine there
from pspnet-keras-tensorflow.
Okay, I see. Look at here, in keras batchnormolization core function definition:
if self.scale: self.gamma = self.add_weight(shape=shape, name='gamma', initializer=self.gamma_initializer, regularizer=self.gamma_regularizer, constraint=self.gamma_constraint) else: self.gamma = None if self.center: self.beta = self.add_weight(shape=shape, name='beta', initializer=self.beta_initializer, regularizer=self.beta_regularizer, constraint=self.beta_constraint) else: self.beta = None self.moving_mean = self.add_weight( shape=shape, name='moving_mean', initializer=self.moving_mean_initializer, trainable=False) self.moving_variance = self.add_weight( shape=shape, name='moving_variance', initializer=self.moving_variance_initializer, trainable=False) self.built = True
The order of parameters in bn layer is scale, offset, mean, variance.
But in your code:
model.get_layer(layer.name).set_weights([mean, variance, scale, offset])
The assigning order is mean, variance, scale, offset.
But even I change the assigning order, the result is not meaningful. So I still consider the ported parameters either have the wrong name or have unnecessary pre-process as I said before. Do you have any idea of this?
from pspnet-keras-tensorflow.
And I found another wrong code. Here in layers_builder.py line 177:
for i in range(3): #for i in range(2): old wrong code
level 3 should have 3_1 - 3_4 according to the original network.
from pspnet-keras-tensorflow.
@wcy940418 wow. I will now test it. impressive
EDIT: Oh, i remembered one thing. Earlier I was thinking about some ideas in pycaffe version of code.
So the first one:
- Implement channel swap and mean color extraction.
SCALE_SIZE = 473 MEAN_R = 123.68 MEAN_G = 116.779 MEAN_B = 103.939 CHANNEL_SWAP = [2,1,0] PROCESSOR = caffe.Classifier(MODEL_PATH, PRETRAINED_FILE_PATH, image_dims=[SCALE_SIZE,SCALE_SIZE], raw_scale=255.0, channel_swap=CHANNEL_SWAP, mean=np.array([MEAN_R, MEAN_G, MEAN_B], dtype='f4'))
2)And the other one which i actually didnt understand, is:
def forward(self, img): input_img = [caffe.io.load_image(img)] #Net processing predictions = self.PROCESSOR.predict(input_img, not 'store_true') return predictions[0]
- the line :not 'store true' i didnt quite understand.
EDIT: i suddenly got it :). not 'store_true' - equals to False. so the 1st arg is input_image and the second is False for the predict function
caffe/blob/master/python/caffe/classifier.py : line 47
def predict(self, inputs, oversample=True): """ Predict classification probabilities of inputs. Parameters ---------- inputs : iterable of (H x W x K) input ndarrays. oversample : boolean average predictions across center, corners, and mirrors when True (default). Center-only prediction when False.
Also line
input_img = [caffe.io.load_image(img)]
I made some experiments earlier and my predictions were different, if i substitute io.load_image to just open with PIL and convert to nd.array with dtype='float32', and thus was worse, similarly to output you are having right now
from pspnet-keras-tensorflow.
@Vladkryvoruchko But I used Pillow(PIL) to read the image for my pycaffe code, the result is just fine as the original code.
from pspnet-keras-tensorflow.
@wcy940418 Oh, I've missed such a small details, fixed by your modifications. Can you please make a pull request to this repo :)
from pspnet-keras-tensorflow.
Related Issues (20)
- is train.py written by python2?
- train.py hasn't finished yet? especially for preprocessing.py in python_utils? HOT 1
- Training on cityscapes HOT 1
- some question about multi_scale HOT 2
- Can't load .h5 weights HOT 3
- Did you ever use auxiliary loss to supervise training, I did not find the associated code
- Adaptive_Pooling Error HOT 1
- My results: wrong colours assignment HOT 5
- How to evaluate results? HOT 1
- scipy version and 'imresize' HOT 4
- Trained models not working in Python 3.6+ HOT 6
- ADE20K segmaps not correctly labeled HOT 1
- The accuracy on voc is not as expected HOT 3
- Who can provide me h5 files? HOT 1
- Illusion of a feature? HOT 2
- "Image not found" error HOT 2
- --ms and -s flags not implemented? HOT 4
- Updates for the repository
- can not down the .h5
- How to show the label of the result
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pspnet-keras-tensorflow.