Hi
I would like to run a quick inference with the model, is it available, perhaps in ONXX format?
Many thanks
Robin

Hi,
I'm having trouble with this project. I'm using Anaconda and i'm trying to run this project but i can't because i get an error saying: "ModuleNotFoundError: No module named 'TeamClassifier'". Does someone know how to solve it?

i wanna do the transfer learning on this repo.
would you mind to upload the trained model weight?
i train the model follow the instruction but the pedestrian keep below 0.4 and mAP below 0.2.

Hi
the google drive images referenced under https://github.com/robmarkcole/aerial_pedestrian_detection#training-images returns 404

why the map and ap=0 during training,and the output shape is(None,None,None,3,0
Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, None, None, 3 0

conv1 (Conv2D) (None, None, None, 6 9408 input_1[0][0]

bn_conv1 (BatchNormalization) (None, None, None, 6 256 conv1[0][0]
1215 instances of class Biker with average precision: 0.0004
12 instances of class Car with average precision: 0.0000
20 instances of class Bus with average precision: 0.0000
62 instances of class Cart with average precision: 0.0000
23 instances of class Skater with average precision: 0.0000
5495 instances of class Pedestrian with average precision: 0.0000
mAP: 0.0001

resnet.py imports keras_resnet which is not available

import keras_resnet
ModuleNotFoundError: No module named 'keras_resnet'

Hello,
I am getting this error while running train.py in my custom data. i followed all the instruction very carefully.
I install progressbar like pip3 install progressbar. But it still throws same error.
Any idea to solve this problem??

my error----------------

keras_retinanet/bin/train.py --weights ./resnet50_coco_best_v2.1.0.h5 csv train_annotations data/train.csv data/classes.csv --val-annotations test.csv
Using TensorFlow backend.
Traceback (most recent call last):
  File "keras_retinanet/bin/train.py", line 39, in <module>
    from ..callbacks.eval import Evaluate
  File "keras_retinanet/bin/../../keras_retinanet/callbacks/eval.py", line 18, in <module>
    from ..utils.eval import evaluate
  File "keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 25, in <module>
    import progressbar
ImportError: No module named progressbar

I have trained RetinaNet on Stanford dataset over 50 epochs as you suggested.
First of all, when i am running convert_model script, no model generated from trained model that is suitable for inferencing.
Furthermore, When i tested trained model on validation data, merely a value of mAP 14% comes but your article claimed 63% on same dataset.

Is it possible to perform transfer learning and only update the weights in the Class+Box subnets? In other words, is it possible to prevent gradients from flowing back to the FPN/Feature extractor?
Thanks in advance

@priya-dwivedi When I train the model by using default config file, and convert it to tensorflow serving using this script:

from keras import backend as K
import tensorflow as tf
from tensorflow.python import saved_model
from tensorflow.python.saved_model.signature_def_utils_impl import (
    build_signature_def, predict_signature_def
)
from keras_retinanet import models
import shutil
import os

export_path = 'retinanet_savedmodel'
model = models.convert_model(
    model=models.backbone(backbone_name='resnet101').retinanet(num_classes=8),
    nms=True,
    class_specific_filter=True,
    anchor_params=None
)
model.load_weights('model.h5')

print('Output layers', [o.name[:-2] for o in model.outputs])
print('Input layer', model.inputs[0].name[:-2])
if os.path.isdir(export_path):
    shutil.rmtree(export_path)
builder = saved_model.builder.SavedModelBuilder(export_path)

signature = predict_signature_def(
    inputs={'images': model.input},
    outputs={
        'output1': model.outputs[0],
        'output2': model.outputs[1],
        'output3': model.outputs[2]
    }
)

sess = K.get_session()
builder.add_meta_graph_and_variables(sess=sess,
                                     tags=[saved_model.tag_constants.SERVING],
                                     signature_def_map={'predict': signature})
builder.save()

I am able to get successful prediction. But the successful predictions are observed only for default anchor parameters. In your code you have changed the anchor parameters.

But, I do not know a correct way to pass anchor_params to above script.

How do I pass anchor_params correctly to above script ?

Dear @priya-dwivedi

Backbone - Resnet50
I am trying to use retinanet with Stanford Drone Data set. When I try to evaluate model resnet50_csv_12_inference.h5 you have shared here https://drive.google.com/drive/u/0/folders/1QpE_iRDq1hUzYNBXSBSnmfe6SgTYE3J4

I'm getting following mAP values.

Running network: 100% (2587 of 2587)
Parsing annotations: 100% (2587 of 2587)
11749 instances of class Biker with average precision: 0.7769
889 instances of class Car with average precision: 0.9921
101 instances of class Bus with average precision: 0.9109
266 instances of class Cart with average precision: 0.5335
485 instances of class Skater with average precision: 0.3929
22188 instances of class Pedestrian with average precision: 0.8732
Inference time for 2587 images: 1.9893
mAP using the weighted average of precisions among classes: 0.8355
mAP: 0.7466

My command -

python evaluate.py --save-path \path to save\ train_annotations.csv labels.csv \path to model..\resnet50_csv_12_inference.h5

But in your document http://cs230.stanford.edu/projects_winter_2019/reports/15767730.pdf I can see you have got lower values than me. Why is that ?

And also I have another question. The file resnet50_csv_12_inference.h5 you have given did you generate this model using small anchors values (16 32 128 256) ?

I must be thankful to you if you will kindly reply me soon.

Regards,
Kulunu.

If the code can run on GPU, can you assist me what changes should I make?
Thanks

Hi,
I am currently trying to train keras retinanet form scratch using stanford dataset but the mAP i get fluctuates in the range of 20-30, whereas you got a great one of 63.

I was just wondering if you could share the hyperparameters you used to train your model? Did you change the steps per epoch?

I noticed that the repository is using 0.25 alpha value of focal loss but wouldn't it be better to use 0.75?

Hi,
how to run detector.py ?

Hi.
which tensorflow version should I use to train model?

Hi Priya...
Could you please upload the weight file?
I followed the steps but could not obtain the higher accuracy.
Thanks

# adjust this to point to your downloaded/trained model
# models can be downloaded here: https://github.com/fizyr/keras-retinanet/releases
model_path = os.path.join('/home/ritika/Downloads/new_downloads/1/resnet152_oid_v1.0.0.h5')

# load retinanet model
model = models.load_model(model_path, backbone_name='resnet50')

# if the model is not converted to an inference model, use the line below
# see: https://github.com/fizyr/keras-retinanet#converting-a-training-model-to-inference-model
#model = models.convert_model(model)

# print(model.summary())

# load label to names mapping for visualization purposes
labels_to_names = {0: 'Biker', 1: 'Car', 2: 'Bus', 3: 'Cart', 4: 'Skater', 5: 'Pedestrian'}
# labels_to_names = {0: 'person', 1: 'bicycle', 2: 'car', 3: 'motorcycle', 4: 'airplane', 5: 'bus', 6: 'train', 7: 'truck', 8: 'boat', 9: 'traffic light', 10: 'fire hydrant', 11: 'stop sign', 12: 'parking meter', 13: 'bench', 14: 'bird', 15: 'cat', 16: 'dog', 17: 'horse', 18: 'sheep', 19: 'cow', 20: 'elephant', 21: 'bear', 22: 'zebra', 23: 'giraffe', 24: 'backpack', 25: 'umbrella', 26: 'handbag', 27: 'tie', 28: 'suitcase', 29: 'frisbee', 30: 'skis', 31: 'snowboard', 32: 'sports ball', 33: 'kite', 34: 'baseball bat', 35: 'baseball glove', 36: 'skateboard', 37: 'surfboard', 38: 'tennis racket', 39: 'bottle', 40: 'wine glass', 41: 'cup', 42: 'fork', 43: 'knife', 44: 'spoon', 45: 'bowl', 46: 'banana', 47: 'apple', 48: 'sandwich', 49: 'orange', 50: 'broccoli', 51: 'carrot', 52: 'hot dog', 53: 'pizza', 54: 'donut', 55: 'cake', 56: 'chair', 57: 'couch', 58: 'potted plant', 59: 'bed', 60: 'dining table', 61: 'toilet', 62: 'tv', 63: 'laptop', 64: 'mouse', 65: 'remote', 66: 'keyboard', 67: 'cell phone', 68: 'microwave', 69: 'oven', 70: 'toaster', 71: 'sink', 72: 'refrigerator', 73: 'book', 74: 'clock', 75: 'vase', 76: 'scissors', 77: 'teddy bear', 78: 'hair drier', 79: 'toothbrush'}

Running this part of code in ResNet50RetinaNet-Video is giving me an error.

TypeError Traceback (most recent call last)
in
4
5 # load retinanet model
----> 6 model = models.load_model(model_path, backbone_name='resnet50')
7
8 # if the model is not converted to an inference model, use the line below

~/Downloads/new_downloads/1/keras_retinanet/models/init.py in load_model(filepath, backbone_name)
81 """
82 import keras.models
---> 83 return keras.models.load_model(filepath, custom_objects=backbone(backbone_name).custom_objects)
84
85

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile)
182 if (h5py is not None and (
183 isinstance(filepath, h5py.File) or h5py.is_hdf5(filepath))):
--> 184 return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
185
186 if sys.version_info >= (3, 4) and isinstance(filepath, pathlib.Path):

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/saving/hdf5_format.py in load_model_from_hdf5(filepath, custom_objects, compile)
176 model_config = json.loads(model_config.decode('utf-8'))
177 model = model_config_lib.model_from_config(model_config,
--> 178 custom_objects=custom_objects)
179
180 # set weights

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/saving/model_config.py in model_from_config(config, custom_objects)
53 'Sequential.from_config(config)?')
54 from tensorflow.python.keras.layers import deserialize # pylint: disable=g-import-not-at-top
---> 55 return deserialize(config, custom_objects=custom_objects)
56
57

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/layers/serialization.py in deserialize(config, custom_objects)
107 module_objects=globs,
108 custom_objects=custom_objects,
--> 109 printable_module_name='layer')

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
371 custom_objects=dict(
372 list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 373 list(custom_objects.items())))
374 with CustomObjectScope(custom_objects):
375 return cls.from_config(cls_config)

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in from_config(cls, config, custom_objects)
985 """
986 input_tensors, output_tensors, created_layers = reconstruct_from_config(
--> 987 config, custom_objects)
988 model = cls(inputs=input_tensors, outputs=output_tensors,
989 name=config.get('name'))

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in reconstruct_from_config(config, custom_objects, created_layers)
2017 # First, we create all layers and enqueue nodes to be processed
2018 for layer_data in config['layers']:
-> 2019 process_layer(layer_data)
2020 # Then we process nodes in order of layer depth.
2021 # Nodes that cannot yet be processed (if the inbound node

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in process_layer(layer_data)
1999 from tensorflow.python.keras.layers import deserialize as deserialize_layer # pylint: disable=g-import-not-at-top
2000
-> 2001 layer = deserialize_layer(layer_data, custom_objects=custom_objects)
2002 created_layers[layer_name] = layer
2003

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/layers/serialization.py in deserialize(config, custom_objects)
107 module_objects=globs,
108 custom_objects=custom_objects,
--> 109 printable_module_name='layer')

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
371 custom_objects=dict(
372 list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 373 list(custom_objects.items())))
374 with CustomObjectScope(custom_objects):
375 return cls.from_config(cls_config)

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in from_config(cls, config, custom_objects)
985 """
986 input_tensors, output_tensors, created_layers = reconstruct_from_config(
--> 987 config, custom_objects)
988 model = cls(inputs=input_tensors, outputs=output_tensors,
989 name=config.get('name'))

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in reconstruct_from_config(config, custom_objects, created_layers)
2027 if layer in unprocessed_nodes:
2028 for node_data in unprocessed_nodes.pop(layer):
-> 2029 process_node(layer, node_data)
2030
2031 input_tensors = []

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py in process_node(layer, node_data)
1975 if input_tensors is not None:
1976 input_tensors = base_layer_utils.unnest_if_single_tensor(input_tensors)
-> 1977 output_tensors = layer(input_tensors, **kwargs)
1978
1979 # Update node index map.

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
895 # Build layer if applicable (if the build method has been
896 # overridden).
--> 897 self._maybe_build(inputs)
898 cast_inputs = self._maybe_cast_inputs(inputs)
899

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in _maybe_build(self, inputs)
2414 # operations.
2415 with tf_utils.maybe_init_scope(self):
-> 2416 self.build(input_shapes) # pylint:disable=not-callable
2417 # We must set also ensure that the layer is marked as built, and the build
2418 # shape is stored since user defined build functions may not be calling

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py in build(self, input_shape)
170 constraint=self.bias_constraint,
171 trainable=True,
--> 172 dtype=self.dtype)
173 else:
174 self.bias = None

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint, partitioner, use_resource, synchronization, aggregation, **kwargs)
575 synchronization=synchronization,
576 aggregation=aggregation,
--> 577 caching_device=caching_device)
578 if regularizer is not None:
579 # TODO(fchollet): in the future, this should be handled at the

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py in _add_variable_with_custom_getter(self, name, shape, dtype, initializer, getter, overwrite, **kwargs_for_getter)
741 dtype=dtype,
742 initializer=initializer,
--> 743 **kwargs_for_getter)
744
745 # If we set an initializer and the variable processed it, tracking will not

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py in make_variable(name, shape, dtype, initializer, trainable, caching_device, validate_shape, constraint, use_resource, collections, synchronization, aggregation, partitioner)
139 synchronization=synchronization,
140 aggregation=aggregation,
--> 141 shape=variable_shape if variable_shape else None)
142
143

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py in call(cls, *args, **kwargs)
257 def call(cls, *args, **kwargs):
258 if cls is VariableV1:
--> 259 return cls._variable_v1_call(*args, **kwargs)
260 elif cls is Variable:
261 return cls._variable_v2_call(*args, **kwargs)

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py in _variable_v1_call(cls, initial_value, trainable, collections, validate_shape, caching_device, name, variable_def, dtype, expected_shape, import_scope, constraint, use_resource, synchronization, aggregation, shape)
218 synchronization=synchronization,
219 aggregation=aggregation,
--> 220 shape=shape)
221
222 def _variable_v2_call(cls,

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py in (**kwargs)
196 shape=None):
197 """Call on Variable class. Useful to force the signature."""
--> 198 previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
199 for _, getter in ops.get_default_graph()._variable_creator_stack: # pylint: disable=protected-access
200 previous_getter = _make_getter(getter, previous_getter)

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in default_variable_creator(next_creator, **kwargs)
2596 synchronization=synchronization,
2597 aggregation=aggregation,
-> 2598 shape=shape)
2599 else:
2600 return variables.RefVariable(

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/variables.py in call(cls, *args, **kwargs)
261 return cls._variable_v2_call(*args, **kwargs)
262 else:
--> 263 return super(VariableMetaclass, cls).call(*args, **kwargs)
264
265

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py in init(self, initial_value, trainable, collections, validate_shape, caching_device, name, dtype, variable_def, import_scope, constraint, distribute_strategy, synchronization, aggregation, shape)
1432 aggregation=aggregation,
1433 shape=shape,
-> 1434 distribute_strategy=distribute_strategy)
1435
1436 def _init_from_args(self,

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, distribute_strategy, shape)
1565 with ops.name_scope("Initializer"), device_context_manager(None):
1566 initial_value = ops.convert_to_tensor(
-> 1567 initial_value() if init_from_fn else initial_value,
1568 name="initial_value", dtype=dtype)
1569 if shape is not None:

~/anaconda3/envs/myenv/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py in ()
119 (type(init_ops.Initializer), type(init_ops_v2.Initializer))):
120 initializer = initializer()
--> 121 init_val = lambda: initializer(shape, dtype=dtype)
122 variable_dtype = dtype.base_dtype
123 if use_resource is None:

~/Downloads/new_downloads/1/keras_retinanet/initializers.py in call(self, shape, dtype)
35 def call(self, shape, dtype=None):
36 # set bias to -log((1 - p)/p) for foreground
---> 37 result = np.ones(shape, dtype=dtype) * -math.log((1 - self.probability) / self.probability)
38
39 return result

~/.local/lib/python3.6/site-packages/numpy/core/numeric.py in ones(shape, dtype, order)
205
206 """
--> 207 a = empty(shape, dtype, order)
208 multiarray.copyto(a, 1, casting='unsafe')
209 return a

TypeError: data type not understood

Hi Priya, I am running the notebook to detect the pedestrians in the examples photos but I am getting this error in the caption:

Do you know what that is?

I used your config with updated sizes of anchors. It went well during first epoch but when the first epoch ended and it came to compute validation loss this exception was raised.

tensorflow.python.framework.errors_impl.InvalidArgumentError: 
Incompatible shapes: [1,147951] vs. [1,197268] [[{{node boxes/mul_7}}]][[{{node filtered_detections/map/while/PadV2_2/paddings}}]]

I have followed all the steps for running the repository , but when I execute keras_retinanet/bin/train.py following error occurs.

File "keras_retinanet/bin/train.py", line 24, in
import keras
ImportError: No module named keras

Although keras==2.3.1 is installed

Hi,

I really appreciate for your work.
Could you please clarify following points w.r.to parameters of config.ini file:
If i change anchor size from [32, 64, 128, 256, 512] to [16, 32, 64, 128, 256] then how would i change the aspect ratio and scale values in config file. Means how can i set the ratio and scale parameters values when i change the anchor size.
I shall be grateful for your generous response.

Thanks,