For getting more accuracy and for precise object detection , which parameters i have to tune , anybody please give me any suggestions .

thanking you !!

@tossy-yossy @mtlouie-unm @franyoadam
This issues have been fixed.
The cause was the positive-negative ratio of the detections.
I fixed the number of RPNs so that the postive-negative ratio will be about 1:3, as in other implementations. Please pull the newest version to activate this.

The training will be stable with using pretrained RPN models.

The pretrained RPN for VGG is uploaded to:
https://drive.google.com/file/d/1teuXIRN4mvmbnIfWlxAAM69hJEceTpUm/view?usp=sharing
The trained vgg frcnn model is uploaded (is underfitting..):
https://drive.google.com/file/d/1IgxPP0aI5pxyPHVSM2ZJjN1p9dtE4_64/view?usp=sharing

Here is the example command.

python train_frcnn.py --network vgg -p to/your/voc --load rpn-mode.pth

Originally posted by @kentaroy47 in #6 (comment)

Hello,

could you tell me about how the data format and annotation format?
I downloaded the VOC2007 dataset to try out your code and am not sure how I should structure the data in directories and how to deal with the xml annotations.
Thanks for your work and help in advance.

Nora

I have seen a strange behaviour when calculating y_rpn_regr with a ground truth (green) and apply rpn_to_roi() on it, it does not project on ground truth again. (bbox with prob=1 is red)

I made an example in my fork https://github.com/k1l1/frcnn-from-scratch-with-keras/blob/7da9fe4d512a41fc37efe42bcba429fea331f091/rpn_error_example.py#L4 where I simply reversed the regression myself.

Can you take a look? Is this an implementation error or a design limitation?

I have a cancer dataset with a center pixel circular dataset along with the radius of the circle. Does your model only take data as 'path x1,y1,x2,y2 class' or can I change the format for my needs?

Can you please help me, I want to test image one by one, but I don't want to load weight and configuration file again. How to load the weight and configuration file once, but I can use it in each test? Thank you

when I tried to run the model I got the below error. also, I recogn version of Keras are deprecated how this can be fixed

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4267: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4409: The name tf.random_normal is deprecated. Please use tf.random.normal instead.

Traceback (most recent call last):
File "train_frcnn.py", line 156, in
classifier = nn.classifier(shared_layers, roi_input, C.num_rois, nb_classes=len(classes_count), trainable=True)
File "/content/drive/My Drive/object detector/frcnn-from-scratch-with-keras-master/keras_frcnn/vgg.py", line 109, in classifier
out_roi_pool = RoiPoolingConv(pooling_regions, num_rois)([base_layers, input_rois])
File "/content/drive/My Drive/object detector/frcnn-from-scratch-with-keras-master/keras_frcnn/RoiPoolingConv.py", line 30, in init
assert self.dim_ordering in {'tf', 'th'}, 'dim_ordering must be in {tf, th}'
AssertionError: dim_ordering must be in {tf, th}

I mean if I have 100 iterations per one epoch training stops on the 99 iteration and that's all.

Hello, I am glad to see that a fork of this project is still alive! Cheers for that.

I have a question regarding the structure of the network, do you think it is possible to train a RPN only to predict regions (I do not need classification as I am going to perform it with SVM) where for each proposed region, I would also need its feature map.

PS. If you have any idea how to get the feature map of a particular region in the input image I would be grateful.

Thanks!

Hi guys,

I downloaded vgg or other pretrained model(s) from the related link(s) and placed it in the root directory. However, when i used the command below:

python train_rpn.py --network vgg -o simple -p /mypathforannotations

I am given the error:

Not a valid model
Traceback (most recent call last):
File "train_rpn.py", line 113, in
raise ValueError
ValueError

Why cannot the code "train_rpn.py" find the model? May be you can describe step by step after downloading the weights, where can we placed them and how can we name these files?

Would you kindly help me, I will be pleased.

Hello,

I found the measure_map.py inside this repo, but I can't find the introduction about it.
So, could you tell me how to run this program?

I already and successfully trained and tested the VOC sample dataset.

Thank you!

File "train_frcnn.py", line 107, in <module> all_imgs, classes_count, class_mapping = get_data(options.train_path, options.cat) TypeError: get_data() takes 1 positional argument but 2 were given

Hey @kentaroy47 ,
Thank you very much for the project.
I am getting the above error (I am really new in this). Could you please help? My training set is having 2 categories (2 classes is the same?). I successfully train the rpn model, but when i proceed to train the whole frcnn this happened.

#6
Above you said this my friend:

I guess you want to see the calculated mAP? I will work with that feature so plz wait.

I can do it, but I dont think it will be coco format. Just reporting Precision and Recall. Will you merge?

Following a successful training of the VGG network on my new images for RPN (Step 3), when I run the Step 4 training step, while a models/vgg/voc.hdf5 file is created, there is no ./model_frcnn.hdf5 file created (which is the default output file given by Model_Checkpoint in the train script). This latter file is presumably the file you would later load into the model for testing (step 5). Specifically, the following command:
python test_frcnn.py --network vgg16 -p ./demo_input.txt --load ./model_frcnn.hdf5 --write
Results in the following output:
Using TensorFlow backend. Not a valid model Traceback (most recent call last): File "test_frcnn.py", line 73, in <module> raise ValueError ValueError

Replacing the ./model... path with the path to the voc file results in the same output.

What am I doing wrong?

Really a great work. Helped me to understand better. Can you please tell me the steps to follow to get region proposals for a test image?

config.pickle file to be used for downloaded pre-trained weight form vgg16 https://drive.google.com/file/d/1IgxPP0aI5pxyPHVSM2ZJjN1p9dtE4_64/view?usp=sharing

Dear All,

So I have some basic experience with frcnn and 3D CNN classifier. Wanted to try to create 3D frcnn code. Was able to modify the code for the purpose quite a bit. Having a hard time to modify the data_generator.py part and more specifically the calc_rpn function. I am trying to detect nodes in medical xray slices. Any help in where to look would also be much appreciated.

Hi, do you have any model trained with VOC or COCO dataset? Could you please share to me? Thank you so much in advance. Have a great day.

I successfully trained the RPN network using mobilenetv2 backbone. But when using the trained RPN network to train full FRCNN this error shows up, ''Tuple has no attribute dims''. I am using tf.keras.

When training with pascalvoc2007, I see person class detection very high while others lower.
Might be because of the class imbalance of the dataset.

{'aeroplane': 331,
'bg': 0,
'bicycle': 418,
'bird': 599,
'boat': 398,
'bottle': 634,
'bus': 272,
'car': 1644,
'cat': 389,
'chair': 1432,
'cow': 356,
'diningtable': 310,
'dog': 538,
'horse': 406,
'motorbike': 390,
'person': 5447,
'pottedplant': 625,
'sheep': 353,
'sofa': 425,
'train': 328,
'tvmonitor': 367}

Some sampling techniques should be introduced to prevent this.

I use google colaboratory to train faster rcnn, but at 138th iteration my training can't be continue because output limit. So I search on google, people say try to change verbose to 0 or 2, but when I try to change, nothing happen. Please help me.

Epoch 138/150 234/360 [==================>...........] - ETA: 1:27 - rpn_cls: 1.3414 - rpn_regr: 0.2049 - detector_cls: 0.0875 - detector_regr: 0.1825Average number of overlapping bounding boxes from RPN = 7.927777777777778 for 360 previous iterations 304/360 [========================>.....] - ETA: 38s - rpn_cls: 1.3611 - rpn_regr: 0.2040 - detector_cls: 0.0876 - detector_regr: 0.1835Buffered data was truncated after reaching the output size limit

I launched train_rpn.py script, gave some weights files.
Next I launch the train_frcnn.py and type
python3 train_frcnn.py --network vgg -p 'packaging1_voc' --rpn 'models/rpn/rpn.vgg.weights.02-0.06.hdf5'
but my weights don't load.
What's the problem?
I give here
model_rpn = model_rpn.load_weights(options.rpn_weight_path)
model_rpn as None
and loop with next errors:
Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch' Exception: 'NoneType' object has no attribute 'train_on_batch'
etc.

Cause it uses cpu

Hi.
I have a problem. I generate a labels.csv
E:/George/images/02.jpg,501,315,705,837,person
but everytime i run python train_rpn.py --network mobilenetv2 -o simple -p E:/George/images/labels.csv
then I get
: 0000:01:00.0, compute capability: 7.5) Parsing annotation files filename Traceback (most recent call last): File "train_rpn.py", line 130, in <module> all_imgs, classes_count, class_mapping = get_data(options.train_path) File "E:\George\keras_frcnn\simple_parser.py", line 39, in get_data (rows, cols) = img.shape[:2] AttributeError: 'NoneType' object has no attribute 'shape'

I try to change my labels.csv like /images/02.jpg or just 02.jpg but no sense can you please help me why that simple_parser.py cannot work?

Is it possible to add Tensorboard to allow for monitoring

This is what I've done in "train_rpn.py" but i only seem to see Graph

model_rpn.compile(optimizer=optimizer, loss=[losses.rpn_loss_cls(num_anchors), losses.rpn_loss_regr(num_anchors)],metrics=['accuracy'])
...
Tensorboard = TensorBoard(log_dir='logs/{}'.format(time())) callback=[Callbacks, Tensorboard] history = model_rpn.fit_generator(data_gen_train,epochs=options.num_epochs, validation_data=data_gen_val,steps_per_epoch=1000,callbacks=callback, validation_steps=10)

I can see rpn loss history is saved, but how can we graph it

import numpy numpy_loss_history = numpy.array(loss_history) numpy.savetxt(options.network+"_rpn_loss_history.txt", numpy_loss_history, delimiter=",")

As an example, I want to set up VOC2007 training.

I got the following error when using Keras 2.2.5.

$ python train_frcnn.py --network mobilenetv1 -p ./VOCdevkit
Using TensorFlow backend.
Traceback (most recent call last):
  File "train_frcnn.py", line 16, in <module>
    from keras_frcnn import losses as losses
  File "/Volumes/HD-EDS-A/Works/frcnn2/keras_frcnn/losses.py", line 4, in <module>
    if K.image_dim_ordering() == 'tf':
AttributeError: module 'keras.backend' has no attribute 'image_dim_ordering'

In Keras 2.2.x, image_dim_ordering seems to have been renamed to image_data_format, so I downgraded it to 2.1.6. But got a different error.

$ python train_frcnn.py --network mobilenetv1 -p ./VOCdevkit
Using TensorFlow backend.
WARNING: Logging before flag parsing goes to stderr.
W0828 18:12:06.414109 4485621184 deprecation_wrapper.py:119] From train_frcnn.py:23: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0828 18:12:06.414335 4485621184 deprecation_wrapper.py:119] From train_frcnn.py:25: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

Traceback (most recent call last):
  File "train_frcnn.py", line 88, in <module>
    from keras_frcnn import mobilenetv1 as nn
  File "/Volumes/HD-EDS-A/Works/frcnn2/keras_frcnn/mobilenetv1.py", line 14, in <module>
    from keras.layers import Flatten, Dense, Input, Conv2D, MaxPooling2D, Dropout, BatchNormalization, Add, ReLU
ImportError: cannot import name 'ReLU' from 'keras.layers' (/Users/toshi/.pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/keras/layers/__init__.py)

ReLU is defined in later Keras versions 2.2.x.
How can I try VOC2007 training?

Hi

The model is successfully running on cpu , but how to run the same model on GPU .I have Nvidia K40.

Secondly, input weight file can be used to place pretrained model? For testing the model for voc , I just need to load the trained model weight ?

Hi, I'm facing error below while testing my own trained model using vgg.I have only 4 classes (include bg) and using VOC-like dataset while training. May I know is there any changes needed to be done in vgg.py or test_frcnn.py in order to successfully test my dataset using own trained data ?

ValueError: Layer #21 (named "time_distributed_2"), weight <tf.Variable 'time_distributed_2/kernel:0' shape=(50176, 4096) dtype=float32_ref> has shape (50176, 4096), but the saved weight has shape (25088, 4096).

I have another question. Can you please provide a running example of how to convert the output to another format like pb? I try a few conversion scripts but, without success.

I am getting out of memory error. How much GPU mem is required to train with batch size 32?

Hello I want to ask, I tried to use this model to implement on my own dataset, but the anchor always bigger than the image. How to solve it?

Hello,
I trained model for 10h and training is completed model is saved into model_frcnn.hdf5 and I have config.pickle. I would like to start training next time using this model not pretrained resnet50_weights_tf_dim_ordering_tf_kernels.h5 but it seems that if I choose model_frcnn.hdf5 as input weights in next training weights are not loaded because loss is much higher than at the end of last training. Can I do that somehow? I don't want to train every time from beggining using pretrained resnet50 but my trained model_frcnn.hdf5

Did anyone try to use inception_v3 as backbone?It worked?

The training is slow because NMS operation is done in python.
should enable NMS be done in cython for faster training and inference.

Hi, thanks for the work.
I'm using a similar implementation and think training the RPN separately is fine. However I don't understand that we train the complete pre-trained network. Wouldn't we like to keep that stable and train the classifier on top only?
Or only train/replace the uppermost layers of the pre-trained network holding the complex features?
In my trainings I've realized that the RPN degrades rather quickly like after about 100 epochs. Is this normal/explainable?
Cheers, BKN

hey i try to read your code, but i am confused about how you get these value? why you using these to divide rpn_reg layer output?
# scaling the stdev self.std_scaling = 4.0 self.classifier_regr_std = [8.0, 8.0, 4.0, 4.0]

Thanks!!!!

I want to evaluate the model for VOC, as it is described that the model can be evaluated for VOC, pcan someone please describe how to do it? Thanks

Can you help me, how to add this code to measure mAP? I want to show confusion_matrix, but I can't found actual and predicted. Thanks

# Python script for confusion matrix creation. 
from sklearn.metrics import confusion_matrix 
from sklearn.metrics import accuracy_score 
from sklearn.metrics import classification_report 
actual = [1, 1, 0, 1, 0, 0, 1, 0, 0, 0] 
predicted = [1, 0, 0, 1, 0, 0, 1, 1, 1, 0] 
results = confusion_matrix(actual, predicted) 
print 'Confusion Matrix :'
print(results) 
print 'Accuracy Score :',accuracy_score(actual, predicted) 
print 'Report : '
print classification_report(actual, predicted) 

What changes i need to make in order to add dense net support ? I get the idea behinf nn_base method returning the feature maps. But why the classification layers are different for different base networks ?

After I successfully trained RPN and whole network with mobilenetv2, I wanted to get results through test_frcnn.py

However, though my parameters were carefully typed and set, I got this error at very beginning.

Loading weights from ./models/mobilenetv2/voc.hdf5
Traceback (most recent call last):
File "test_frcnn.py", line 194, in
model_classifier.load_weights(options.load, by_name=True)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/network.py", line 1163, in load_weights
reshape=reshape)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py", line 1149, in load_weights_from_hdf5_group_by_name
str(weight_values[i].shape) + '.')
ValueError: Layer #153 (named "conv1_td"), weight <tf.Variable 'conv1_td/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref> has shape (3, 3, 512, 512), but the saved weight has shape (512, 320, 3, 3).

As you see, my trained model has been saved in "/models/mobilenetv2/voc.hdf5". Did not you try this scheme before publishing it? Could you please help me? What is wrong? By the way, my 2 phase training was ok and I have seen the decrease in loss. My dataset is custom and contains 16 classes including 'bg' and I have simulated the dataset structure of vockit.
Here is my test argument line:

python3 test_frcnn.py --network mobilenetv2 -p ./VOCdevkit2007/VOC2007/TestImages/ --load ./models/mobilenetv2/voc.hdf5 --write

in the requirement, keras is mentioned but did not mention which version of keras to install. because in the latest version most of the methods used here is depreciated can someone suggest version for both keras and tensorflow

Hello, I've tried to run train_rpn.py , but I got some error like this. Maybe you can explain it to me

Using TensorFlow backend.
Traceback (most recent call last):
  File "train_rpn.py", line 18, in <module>
    from keras_frcnn import config_anc9 as config
ImportError: cannot import name 'config_anc9'

How to calculate recall, accuracy, and precision? Can you please add it? Thank you

When executing test_frcnn.py it seems that I pass in the path to where my test images are located, but I get zero detections when testing the model that was successfully trained. Wouldn't the testing phase need labeled data to see how well the model detects?

Is Faster RCNN using batch_size when training no or epoch_length is batch_size? Can you please explain it, I can't find the batch_size?