question 1:

1. I use a machine with NVIDIA gtx960 graphics, 2g memory, oom appears when training 300x300 ssd model.
2. so I modified batch_size = 2, image size = 224, can train, train 60,000 steps and loss = 5.11.
3. then I test ssd_detect.ipynb with image_resize=224, running this ipynb but can not success localizate which person and fish.
4. when test ssd_detect.ipynb with image_resize=300, can success localizate .

question 2:

how to test fps? what should i do?

$ ./data/VOC0712/create_data.sh
Traceback (most recent call last):
File "/home/chuanjun/Documents/caffe/data/VOC0712/../../scripts/create_annoset.py", line 103, in
label_map = caffe_pb2.LabelMap()
AttributeError: 'module' object has no attribute 'LabelMap'
Traceback (most recent call last):
File "/home/chuanjun/Documents/caffe/data/VOC0712/../../scripts/create_annoset.py", line 103, in
label_map = caffe_pb2.LabelMap()
AttributeError: 'module' object has no attribute 'LabelMap'

Is the visualize option in DetectionOutputLayer can not be used yet?
I am confused about the following code in :void DetectionOutputLayer::Forward_cpu(...):
if (visualize_) {

ifdef USE_OPENCV

vector<cv::Mat> cv_imgs;
this->data_transformer_->TransformInv(bottom[3], &cv_imgs);
vector<cv::Scalar> colors = GetColors(label_to_display_name_.size());
VisualizeBBox(cv_imgs, top[0], visualize_threshold_, colors,
    label_to_display_name_);

endif // USE_OPENCV

I am confused about the bottom[3] in DetectionOutputLayer, because the test.prototext indicates that the DetectionOutputLayer only has 3 buttom blob.

Hi, @weiliu89 , I downloaded the code and changed to the ssd branch.
it seems there are compiling errors.

In file included from ./include/caffe/layers/annotated_data_layer.hpp:8,
from src/caffe/layers/annotated_data_layer.cpp:11:
./include/caffe/data_reader.hpp:76: error: type/value mismatch at argument 1 in template parameter list for ‘template struct boost::weak_ptr’
./include/caffe/data_reader.hpp:76: error: expected a type, got ‘caffe::DataReader::Body’
./include/caffe/data_reader.hpp:76: error: template argument 2 is invalid
./include/caffe/data_reader.hpp:76: error: template argument 4 is invalid
make: *** [.build_debug/src/caffe/layers/annotated_data_layer.o] Error 1
make: *** Waiting for unfinished jobs....
and I fount that the data_reader.hpp is different with the original caffe's data_reader.hpp
The original caffe doesn't have this
template
24 class DataReader {

could help me check it?

When I run ssd_pascal_video.py, it reports this error:

Traceback (most recent call last):
File "./examples/ssd/ssd_pascal_video.py", line 90, in
'video_type': P.VideoData.VIDEO,
File "/home/zhaoboya/SSD/caffe/python/caffe/net_spec.py", line 240, in getattr
return getattr(getattr(caffe_pb2, name + 'Parameter'), param_name)
AttributeError: type object 'VideoDataParameter' has no attribute 'VIDEO'

My environment is python3,ubuntu14.04,cuda7.5 and titanx

I want to train SSD on other model,such as Alexnet. And I also try to do this. But I can not train the model successful. How can I train SSD with ALexnet?

I trained the ssd300,but the result only reach 55%~,can you explain the details about how you train the final model?

Hi, Wei. Thank you in advance for your wonderful code :)

Already execution file is in $CAFFE_ROOT/build/examples/ssd/ssd_detect.bin.

I want to execute and integrate the SSD framework with other system on the TX1 board.

Thus, I want to modify and compile the ssd_detect.cpp. But, I'm not familiar with compiling in ubuntu.

Could you explain how to compile with ssd_detect.cpp ?

When I execute the command:

./build/tools/caffe time --model models/VGGNet/VOC0712/SSD_300x300_score/train.prototxt

following error occurred:

I0712 20:17:13.744546 22885 net.cpp:274] Network initialization done.
I0712 20:17:13.744894 22885 caffe.cpp:320] Performing Forward
I0712 20:17:16.929119 22885 caffe.cpp:325] Initial loss: 0
I0712 20:17:16.929173 22885 caffe.cpp:326] Performing Backward
I0712 20:17:20.996608 22885 caffe.cpp:334] *** Benchmark begins ***
I0712 20:17:20.996647 22885 caffe.cpp:335] Testing for 50 iterations.
F0712 20:17:24.123919 22885 prior_box_layer.hpp:62] Not Implemented Yet
*** Check failure stack trace: ***
@ 0x7f60e56addaa (unknown)
@ 0x7f60e56adce4 (unknown)
@ 0x7f60e56ad6e6 (unknown)
@ 0x7f60e56b0687 (unknown)
@ 0x7f60e5eb2feb caffe::PriorBoxLayer<>::Backward_cpu()
@ 0x40976c time()
@ 0x4061dc main
@ 0x7f60e49bbf45 (unknown)
@ 0x406911 (unknown)
@ (nil) (unknown)

I have run ssd_pascal_video.py，and got 12FPS in the video.
My GPU is Tesla K80,and I use cuda-7.0. I think it's too slow compared with 58 FPS in your paper.
Is there something wrong ？

When I run python examples/ssd/ssd_pascal_webcam.py, I Failed to open webcam: 0.
It seems that I need a webcam connnected to SSD,but how do I test a video with SSD named demo.mp4 that already exits in my directory？

I get this error everytime i run ssd_pascal.py . Any idea how to fix this ?

I0712 13:04:50.931237 23455 net.cpp:91] Creating Layer conv1_1
I0712 13:04:50.931251 23455 net.cpp:425] conv1_1 <- data_data_0_split_0
I0712 13:04:50.931257 23455 net.cpp:399] conv1_1 -> conv1_1
F0712 13:04:50.938412 23468 data_transformer.cpp:196] Check failed: datum_height == height (600 vs. 300)
*** Check failure stack trace: ***
@ 0x7fccd7752daa (unknown)
@ 0x7fccd7752ce4 (unknown)
@ 0x7fccd77526e6 (unknown)
@ 0x7fccd7755687 (unknown)
@ 0x7fccd7fe7ec0 caffe::DataTransformer<>::Transform()
@ 0x7fccd7fec0db caffe::DataTransformer<>::Transform()
@ 0x7fccd7fec198 caffe::DataTransformer<>::Transform()
@ 0x7fccd7fec24e caffe::DataTransformer<>::Transform()
@ 0x7fccd7f25cdf caffe::AnnotatedDataLayer<>::load_batch()
@ 0x7fccd7efd4a9 caffe::BasePrefetchingDataLayer<>::InternalThreadEntry()
@ 0x7fccd7fdf530 caffe::InternalThread::entry()
@ 0x7fcccca565aa thread_proxy
@ 0x7fccc85ba184 start_thread
@ 0x7fccd638737d (unknown)
@ (nil) (unknown)
Aborted (core dumped)

Dear Contributor,

1. Thanks for making this wonderful code available for general use.
2. I wanted to train on new set of classes and made my changes for the same in prototxt file in data folder and pascal_multilabel_datalayers file in examples/pycaffe/layers, but my training keeps on returning NaN and then on my test image no segmentation comes up.
3. Would be delighted if you could take some time out and help me with the same.

Thanks.

./data/VOC0712/create_data.sh
does not exist: /home/yunquan/data/VOCdevkit/VOCdevkit_person/Annotations/000001.xml
does not exist: /home/yunquan/data/VOCdevkit/VOCdevkit_person/Annotations/000003.xml

I referenced http://blog.csdn.net/sinat_30071459/article/details/51332084 to create my own dataset , And I modifyed /home/XXX/caffe/data/VOC0712/trainval.txt 、test.txt and test_name_size.txt . when I used ./data/VOC0712/create_data.sh ,it failed. I promise 000001.xml and 000003.xml is existed.

The project compiles fine but runs into a weird error when I start the pascal training using ssd_pascal.py without any change (except data path and gpus).
Context: 1 GPU, cuda 7.0, cudnn v4
Below are the error messages:

...
...
multibox_loss_param {
loc_loss_type: SMOOTH_L1
conf_loss_type: SOFTMAX
loc_weight: 1
num_classes: 21
share_location: true
match_type: PER_PREDICTION
overlap_threshold: 0.5
use_prior_for_matching: true
background_label_id: 0
use_difficult_gt: true
do_neg_mining: true
neg_pos_ratio: 3
neg_overlap: 0.5
code_type: CENTER_SIZE
}
}
I0713 20:19:47.714035 23178 layer_factory.hpp:77] Creating layer data
I0713 20:19:47.714771 23178 net.cpp:91] Creating Layer data
I0713 20:19:47.714797 23178 net.cpp:399] data -> data
I0713 20:19:47.714933 23178 net.cpp:399] data -> label
I0713 20:19:47.716934 23266 db_lmdb.cpp:35] Opened lmdb examples/VOC0712/VOC0712_trainval_lmdb
I0713 20:19:47.742223 23178 annotated_data_layer.cpp:52] output data size: 32,3,300,300
I0713 20:19:47.838522 23178 net.cpp:141] Setting up data
I0713 20:19:47.838660 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.838678 23178 net.cpp:148] Top shape: 1 1 1 8 (8)
I0713 20:19:47.838685 23178 net.cpp:156] Memory required for data: 34560032
I0713 20:19:47.838709 23178 layer_factory.hpp:77] Creating layer data_data_0_split
I0713 20:19:47.838824 23178 net.cpp:91] Creating Layer data_data_0_split
I0713 20:19:47.838837 23178 net.cpp:425] data_data_0_split <- data
I0713 20:19:47.838865 23178 net.cpp:399] data_data_0_split -> data_data_0_split_0
I0713 20:19:47.838891 23178 net.cpp:399] data_data_0_split -> data_data_0_split_1
I0713 20:19:47.838907 23178 net.cpp:399] data_data_0_split -> data_data_0_split_2
I0713 20:19:47.838927 23178 net.cpp:399] data_data_0_split -> data_data_0_split_3
I0713 20:19:47.838939 23178 net.cpp:399] data_data_0_split -> data_data_0_split_4
I0713 20:19:47.838951 23178 net.cpp:399] data_data_0_split -> data_data_0_split_5
I0713 20:19:47.838963 23178 net.cpp:399] data_data_0_split -> data_data_0_split_6
I0713 20:19:47.839156 23178 net.cpp:141] Setting up data_data_0_split
I0713 20:19:47.839172 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839181 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839208 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839220 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839227 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839236 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839244 23178 net.cpp:148] Top shape: 32 3 300 300 (8640000)
I0713 20:19:47.839251 23178 net.cpp:156] Memory required for data: 276480032
I0713 20:19:47.839257 23178 layer_factory.hpp:77] Creating layer conv1_1
I0713 20:19:47.839328 23178 net.cpp:91] Creating Layer conv1_1
I0713 20:19:47.839339 23178 net.cpp:425] conv1_1 <- data_data_0_split_0
I0713 20:19:47.839354 23178 net.cpp:399] conv1_1 -> conv1_1
F0713 20:19:47.872011 23267 math_functions.cpp:250] Check failed: a <= b (1 vs. 1)
*** Check failure stack trace: ***
@ 0x7f1ec1a0f4dd google::LogMessage::Fail()
@ 0x7f1ec1a115ef google::LogMessage::SendToLog()
@ 0x7f1ec1a0f0cc google::LogMessage::Flush()
@ 0x7f1ec1a11e8d google::LogMessageFatal::~LogMessageFatal()
@ 0x7f1ec2290c1f caffe::caffe_rng_uniform<>()
@ 0x7f1ec22911b4 caffe::SampleBBox()
@ 0x7f1ec229165d caffe::GenerateSamples()
@ 0x7f1ec2291861 caffe::GenerateBatchSamples()
@ 0x7f1ec211033d caffe::AnnotatedDataLayer<>::load_batch()
@ 0x7f1ec219d4b1 caffe::BasePrefetchingDataLayer<>::InternalThreadEntry()
@ 0x7f1ec210d176 caffe::InternalThread::entry()
@ 0x7f1eb9331ce9 (unknown)
@ 0x7f1eb4ea1e9a start_thread
@ 0x7f1ec100d36d (unknown)
Aborted

I checked that line 250 of math_functions.cpp is
CHECK_LE(a, b);
Isn't it supposed to be fine when a=1 and b=1 ?

I changed the class_number in the ssd_pacal.py, but the pretrained model provided seems trained over 21 classes,which lead to a shape mismatch problem from mbox_source_layers. How could I modify the training code?

And I have another question about the training image size. The net has forced the input size of 500_500 or 300_300, and the code has warped the input images. If the training images are not square, would this lead to some problems about convergence?

@weiliu89
In the new version which contains ssd_pascal_video.py, there is an error when i import caffe

File "/home/zhaoboya/caffe-ssd/python/caffe/init.py", line 8, in
from .net_spec import layers, params, NetSpec, to_proto
File "/home/zhaoboya/caffe-ssd/python/caffe/net_spec.py", line 244, in
_param_names = param_name_dict()
File "/home/zhaoboya/caffe-ssd/python/caffe/net_spec.py", line 36, in param_name_dict
param_type_names = [type(getattr(layer, s)).name for s in param_names]
File "/home/zhaoboya/caffe-ssd/python/caffe/net_spec.py", line 36, in
param_type_names = [type(getattr(layer, s)).name for s in param_names]
AttributeError: 'LayerParameter' object has no attribute 'transform_param'

But when I change the caffe_pb2.py to the origin version which doesn't contain ssd_pascal_video.py, import caffe is ok and the test of mnist is right.

I compare the two caffe_pb2.py file, the differences are serialized_start, serialized_end and some code about "VIDEO"

my environment is python3, how can I fix this problem...sorry to interrupt you again,thanks

I trained VGG_VOC2012ext with 980ti+cuda7.5+cudnn v3(since cudnn v4 seems not compatible), and I have tested cudnn and non-cudnn version. However, the cudnn version is much slower(40 seconds for 20 iterations, and non-cudnn is 19 seconds for 20 iterations).
So is there something wrong with my compiling or cudnn version, could you please explain it?

Thanks

When I perform command ./data/VOC0712/create_data.sh, it report
Traceback (most recent call last):
File "/root/lishengxi/caffe/data/VOC0712/../../scripts/create_annoset.py", line 103, in
label_map = caffe_pb2.LabelMap()
AttributeError: 'module' object has no attribute 'LabelMap' error

Sorry to bother you again. When i'm training the Net using 'examples/ssd/ssd_pascal.py', an error shows up.

I0630 12:04:19.019912 38219 caffe.cpp:209] Resuming from models/VGGNet/VOC0712/SSD_300x300/VGG_VOC0712_SSD_300x300_iter_60000.solverstate
F0630 12:04:19.292585 38219 net.cpp:765] Cannot copy param 0 weights from layer 'conv4_3_norm_mbox_conf'; shape mismatch. Source param shape is 63 512 3 3 (290304); target param shape is 18 512 3 3 (82944). To learn this layer's parameters from scratch rather than copying from a saved net, rename the layer.

this error always happens in a FC layer because of needing fixed length. But, why it happens here?

Hello, thanks for your sharing.
Now I'm training ssd models on imagenet, could you provide labelmap_ilsvrc.prototxt, moreover, with related data preperation scripts?

Hi, according to:

-#34
-https://github.com/weiliu89/caffe/blob/ssd/examples/ssd/score_ssd_pascal.py

The speed reported, e.g. for ssd500x500 is FPS = no_of_test_imgaes/total_time = 4952/67 = 74.
However, i think this is for "test_batch_size = 8" in score_ssd_pascal.py.

In my own measurement, which use my own timing function:
START_TIMER();
net.Forward();
cudaThreadSynchronize();
time += END_TIMER();

I am getting the below for titan-X:
voc07+12_ssd300x300 : 30 ms or 33.3 fps
voc07+12_ssd500x500 : 121 ms or 8.26 fps

Please let me know if my measurement are reasonable.
Also I think the speed from faster-RCNN website and paper is for batch=1.
YOLO website and paper reports similarly as SSD using batch>=1.

Hi, @weiliu89 , I downloaded the code and changed to the ssd branch.
When I run the ssd_pascal.py, there are some errors about lmdb. I've followed your instructions in readme file. Could you help me check it?

I0624 22:59:21.664676 12142 layer_factory.hpp:77] Creating layer data
I0624 22:59:21.665473 12142 net.cpp:91] Creating Layer data
I0624 22:59:21.665518 12142 net.cpp:399] data -> data
I0624 22:59:21.665637 12142 net.cpp:399] data -> label
I0624 22:59:21.682493 12155 db_lmdb.cpp:35] Opened lmdb /home/caffe_ssd/examples/VOC0712/VOC0712_trainval_lmdb
C: @ 0x7f56d22717d0 cv::InputArray::getMat()
*** SIGSEGV (@0x80030001) received by PID 12142 (TID 0x7f56dddc39c0) from PID 18446744071562264577; stack trace: ***
@ 0x7f56db1ccd40 (unknown)
@ 0x7f56d22717d0 cv::InputArray::getMat()
@ 0x7f56dbcb32b9 cv::imdecode()
@ 0x7f56dcf3c392 caffe::DecodeDatumToCVMatNative()
@ 0x7f56dcfaa086 caffe::DataTransformer<>::InferBlobShape()
@ 0x7f56dce53009 caffe::AnnotatedDataLayer<>::DataLayerSetUp()
@ 0x7f56dceaabe9 caffe::BaseDataLayer<>::LayerSetUp()
@ 0x7f56dceab036 caffe::BasePrefetchingDataLayer<>::LayerSetUp()
@ 0x7f56dcdcc374 caffe::Layer<>::SetUp()
@ 0x7f56dcf000bd caffe::Net<>::Init()
@ 0x7f56dcefe471 caffe::Net<>::Net()
@ 0x7f56dceeaddd caffe::Solver<>::InitTrainNet()
@ 0x7f56dceea600 caffe::Solver<>::Init()
@ 0x7f56dceea09c caffe::Solver<>::Solver()
@ 0x7f56dcfb5bd3 caffe::SGDSolver<>::SGDSolver()
@ 0x7f56dcfc1227 caffe::Creator_SGDSolver<>()
@ 0x421971 caffe::SolverRegistry<>::CreateSolver()
@ 0x41d34c train()
@ 0x41f7df main
@ 0x7f56db1b7ec5 (unknown)
@ 0x41c199 (unknown)
@ 0x0 (unknown)
Segmentation fault (core dumped)

i want to run ssd on windows ,but it have many bugs, especal the detection_output_layer.cu file

the error following:
Traceback (most recent call last):
File "/home/zhaolei/caffe/data/VOC0712/../../scripts/create_annoset.py", line 103, in
label_map = caffe_pb2.LabelMap()
AttributeError: 'module' object has no attribute 'LabelMap'

I want to create data,but i run the script ,it not has LabelMap() function in caffe_pb2. I have import caffe_pb2 file.

Hi,
Can we use other base network than vgg16 which would be fast?
I want to run SSD @30 fps for 1080p video with GPU memory of 2.5 GB.

Thanks,
william.j.

Hi,

I want to use resnet152 model for ssd.
and, i convert your "ssd_pascal.py".
so, this part

VGGNetBody(net, from_layer='data', fully_conv=True, reduced=True, dilated=True, dropout=False, freeze_layers=freeze_layers)
--> ResNet152Body(net, from_layer='data')

pretrain_model = "models/VGGNet/VGG_ILSVRC_16_layers_fc_reduced.caffemodel"
--> pretrain_model = "my_path/ResNet-152-model.caffemodel"

finally, commad : python examples/ssd/ssd_pascal.py

but, i have a error..

File "examples/ssd/ssd_pascal_resnet.py", line 401, in
prior_variance=prior_variance, kernel_size=3, pad=1)
File "../opt/caffe/python/caffe/model_libs.py", line 618, in CreateMultiBoxHead
net[norm_name] = L.Normalize(net[from_layer], scale_filler=dict(type="constant", value=normalizations[i]),
File ..r/opt/caffe/python/caffe/net_spec.py", line 188, in getitem
return self.getattr(item)
File ".../opt/caffe/python/caffe/net_spec.py", line 182, in getattr
return self.tops[name]
KeyError: 'conv4_3'

thank you.

it seems that this project do not provide the code of evaluating.

Hi~ I try my data using SSD, and i want to train a 5 classes detector (my labels are defined as number 1-5). but it comes an error to me when using'./data/VOC0712/create_data.sh' to create lmdb. The error is In io.cpp line 285 : unknow name. Do i have to change my own labels to the VOC labels?

I got this error when i run ./data/VOC0712/create_data.sh and i can't generate lmdb files.

Cannot parse label map file: /home/zhaoboya/SSD/caffe/data/VOC0712/labelmap_voc.prototxt
Cannot parse label map file: /home/zhaoboya/SSD/caffe/data/VOC0712/labelmap_voc.prototxt

The labelmap_voc.prototxt is not modified.
Thanks for helping me.

Hi:
I train new models from new data. the new data has four label(including background 0). but I find it will not detect label 1 and label 2 object, it will only detect label 3 object. And I check the training log. I find in the training log , all the test net will print
I0715 09:58:41.775827 257 solver.cpp:421] Iteration 1000, Testing net (#0)
I0715 09:58:42.032855 257 net.cpp:684] Ignoring source layer mbox_loss
W0715 09:58:44.846182 257 solver.cpp:507] Missing true_pos for label: 1
W0715 09:58:44.846276 257 solver.cpp:507] Missing true_pos for label: 2
I0715 09:58:44.846349 257 solver.cpp:526] Test net output #0: detection_eval = 0.333333

the last layer of loss is
layer {
name: "mbox_loss"
type: "MultiBoxLoss"
bottom: "mbox_loc"
bottom: "mbox_conf"
bottom: "mbox_priorbox"
bottom: "label"
top: "mbox_loss"
include {
phase: TRAIN
}
propagate_down: true
propagate_down: true
propagate_down: false
propagate_down: false
loss_param {
normalization: VALID
}
multibox_loss_param {
loc_loss_type: SMOOTH_L1
conf_loss_type: SOFTMAX
loc_weight: 1.0
num_classes: 4
share_location: true
match_type: PER_PREDICTION
overlap_threshold: 0.5
use_prior_for_matching: true
background_label_id: 0
use_difficult_gt: true
do_neg_mining: true
neg_pos_ratio: 3.0
neg_overlap: 0.5
code_type: CENTER_SIZE
}

And I check the lmdb data, I find the label is right , in the training and testing data, it has label 1 and label 2 object.
Is there any problem in training?
thanks.

[----------] 6 tests from CuDNNConvolutionLayerTest/0, where TypeParam = float
[ RUN ] CuDNNConvolutionLayerTest/0.TestGradientCuDNN
*** Aborted at 1467031849 (unix time) try "date -d @1467031849" if you are using GNU date ***
PC: @ 0x2add859c3897 cudnnAddTensor
*** SIGSEGV (@0x703101e04) received by PID 30140 (TID 0x2add7741f080) from PID 51387908; stack trace: ***
@ 0x2add7e290340 (unknown)
@ 0x2add859c3897 cudnnAddTensor
@ 0x2add7d653873 caffe::CuDNNConvolutionLayer<>::Forward_gpu()
@ 0x48e356 caffe::Layer<>::Forward()
@ 0x491fbf caffe::GradientChecker<>::CheckGradientSingle()
@ 0x49518b caffe::GradientChecker<>::CheckGradientExhaustive()
@ 0x9300da caffe::CuDNNConvolutionLayerTest_TestGradientCuDNN_Test<>::TestBody()
@ 0x9a6813 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x99d4f7 testing::Test::Run()
@ 0x99d59e testing::TestInfo::Run()
@ 0x99d6a5 testing::TestCase::Run()
@ 0x9a09e8 testing::internal::UnitTestImpl::RunAllTests()
@ 0x9a0c77 testing::UnitTest::Run()
@ 0x473d9f main
@ 0x2add7e4bfec5 __libc_start_main
@ 0x47cba9 (unknown)
make: *** [runtest] Segmentation fault (core dumped)

the link of fully convolutional reduced(atrous) VGGNet can not access.

@weiliu89, I was trying to follow your steps for training on mscoco.
As per: https://github.com/weiliu89/caffe/tree/ssd/data/coco
# Check scripts/batch_split_annotation.py and change settings accordingly.
python scripts/batch_split_annotation.py
# Create the minival2014_name_size.txt and test-dev2015_name_size.txt in $CAFFE_ROOT/data/coco
python scripts/batch_get_image_size.py

However I can't find batch_split_annotation.py or batch_get_image_size.py in your repo under scripts folder. Is it stored in a different location?

I'm currently tweaking various hyperparameters to get better results from my data.

examples/ssd/ssd_pascal.py

# Minimum scale of 0.2
min_ratio = 20
max_ratio = 95
step = int(math.floor((max_ratio - min_ratio) / (len(mbox_source_layers) - 2)))
min_sizes = []
max_sizes = []
for ratio in xrange(min_ratio, max_ratio + 1, step):
  min_sizes.append(min_dim * ratio / 100.)
  max_sizes.append(min_dim * (ratio + step) / 100.)
# A box with scale 0.1 is added nonetheless. Why?
min_sizes = [min_dim * 10 / 100.] + min_sizes
max_sizes = [[]] + max_sizes

Magic Number Alert:
I do not understand why a box with ratio 10 is being added to the list of box sizes.
If I were to set min_ratio = 4, would it still make sense to add the min_dim * 10 / 100 box, or should I rather add a min_dim * 2 / 100 box?

In your SSD paper, you write

[sic 3.4 MS COCO] but
now our smallest default box has a scale of 0.1 instead of 0.2, and the scale of the default
box on conv4 3 is 0.07 (e.g. corresponding to 21 pixels for a 300 × 300 image).

Is this cite relevant to this?

I would be happy if you could shed some light on this subject, please.

Thanks in advance for your wonderful code :)

Recently, I tried to change the VGG-16 base network to ResNet-50L & ResNet-101L by using pretrained model from K. He Github repo. and reffering your code 'ResBody' and 'Res152Body' in model_libs.py.

In ResNet-50L version, I used res3d_branch2c and res4f_branch2c for mbox_source_layers.
In ResNet-101L version, I used res3b3_branch2c and res4b22_branch2c for mbox_source_layers.

In addition, I also freezed relatively previous layers.

However, the results on ResNet50 & ResNet101 always were not better than VGG-base.

I read in previous Issues that you already successfully tested Inception-v3 and ResNet101L. I wonder your results.

And Could you uploaded your pretrained model on Inception-v3 and ResNet101L ?

Thank you.

Traceback (most recent call last):
File "/home/ilab/Documents/SSD/caffe/data/VOC0712/../../scripts/create_annoset.py", line 7, in
from caffe.proto import caffe_pb2
ImportError: No module named caffe.proto
Traceback (most recent call last):
File "/home/ilab/Documents/SSD/caffe/data/VOC0712/../../scripts/create_annoset.py", line 7, in
from caffe.proto import caffe_pb2

ImportError: No module named caffe.proto

make test, make runtest are all fine, and i follow the step strictly please help

I have trained the Net on my own data successfully. So i try to test the net using 'examples/ssd/score_ssd_pascal.py' , it runs without any error and output:

I0704 16:32:17.817559 4196 blocking_queue.cpp:50] Data layer prefetch queue empty
I0704 16:32:21.424787 4196 solver.cpp:526] Test net output #0: detection_eval = 0.936136
I0704 16:32:21.424835 4196 solver.cpp:325] Optimization Done.
I0704 16:32:21.424841 4196 caffe.cpp:222] Optimization Done.

In output_result_dir, i.e., "{}/data/VOCdevkit/results/VOC2007/{}_score/Main", there exists some output files which are like 'comp4_det_test_classename.txt'. But there are no words or other information in these txt files. I wonder why? (i check the files in " {}/data/VOCdevkit/results/VOC2007/{}/Main", and they do have results. I think they may be created in training stage not in test stage. )

I don't quite understand how prior_box_layer performs backpropagation if it is not implemented

Hi, @weiliu89 . You did a great job! I have learned much from your work. Thank you.
However, I think the CMake files should be updated. When I tried to build the Caffe-SSD project using CMake, errors occurred which informed me that cv::VideoCapture and boost::regex were undefined references. So I changed the file /cmake/Dependencies.cmake to fix the problems.
In line 5, add regex at the end.

find_package(Boost 1.46 REQUIRED COMPONENTS system thread filesystem regex)

In line 72, add videoio at the end.

find_package(OpenCV QUIET COMPONENTS core highgui imgproc imgcodecs videoio)

Thank you for your work!

Hi there.

The original caffe (1.0.0-rc3) building process run without any problem.
I'm unable to build caffe-ssd and get the following error:

[ 60%] Building CXX object src/caffe/CMakeFiles/caffe.dir/layers/log_layer.cpp.o
In file included from /home/caffe-ssd/Downloads/caffe/include/caffe/common.hpp:19:0,
from /home/caffe-ssd/Downloads/caffe/include/caffe/blob.hpp:8,
from /home/caffe-ssd/Downloads/caffe/include/caffe/layers/multibox_loss_layer.hpp:8,
from /home/caffe-ssd/Downloads/caffe/src/caffe/layers/multibox_loss_layer.cpp:6:
/home/caffe-ssd/Downloads/caffe/include/caffe/util/device_alternate.hpp:15:36: error: no ‘void caffe::MultiBoxLossLayer::Forward_gpu(const std::vectorcaffe::Blob<Dtype_>&, const std::vectorcaffe::Blob<Dtype_>&)’ member function declared in class ‘caffe::MultiBoxLossLayer’
const vector<Blob>& top) { NO_GPU; }
^
/home/caffe-ssd/Downloads/caffe/src/caffe/layers/multibox_loss_layer.cpp:574:1: note: in expansion of macro ‘STUB_GPU’
STUB_GPU(MultiBoxLossLayer);
^
/home/caffe-ssd/Downloads/caffe/include/caffe/util/device_alternate.hpp:19:39: error: no ‘void caffe::MultiBoxLossLayer::Backward_gpu(const std::vectorcaffe::Blob<Dtype>&, const std::vector&, const std::vectorcaffe::Blob<Dtype_>&)’ member function declared in class ‘caffe::MultiBoxLossLayer’
const vector<Blob_>& bottom) { NO_GPU; }
^
/home/caffe-ssd/Downloads/caffe/src/caffe/layers/multibox_loss_layer.cpp:574:1: note: in expansion of macro ‘STUB_GPU’
STUB_GPU(MultiBoxLossLayer);
^
make[2]: *** [src/caffe/CMakeFiles/caffe.dir/layers/multibox_loss_layer.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
make: *** [all] Error 2

I have error when changed cuda version form 7.5 to 8.0
in file ----- detection_output_layer.cu
/usr/include/boost/property_tree/detail/json_parser_read.hpp: In constructor ‘boost::property_tree::json_parser::json_grammar::definition::definition(const boost::property_tree::json_parser::json_grammar&)’:
/usr/include/boost/property_tree/detail/json_parser_read.hpp:257:264: error: ‘type name’ declared as function returning an array
escape
^
/usr/include/boost/property_tree/detail/json_parser_read.hpp:257:264: error: ‘type name’ declared as function returning an array

Im not correctly understand why this error arise?

to start train ssd when i do this command :
python examples/ssd/ssd_pascal.py

The following questions arise:

F0707 18:28:31.445760 14913 parallel.cpp:130] Check failed: error == cudaSuccess (10 vs. 0) invalid device ordinal
*** Check failure stack trace: ***
@ 0x7f65bd791daa (unknown)
@ 0x7f65bd791ce4 (unknown)
@ 0x7f65bd7916e6 (unknown)
@ 0x7f65bd794687 (unknown)
@ 0x7f65bdec65a7 caffe::DevicePair::compute()
@ 0x7f65bdecb33c caffe::P2PSync<>::Prepare()
@ 0x7f65bdecb8bc caffe::P2PSync<>::Run()
@ 0x40843b train()
@ 0x4061dc main
@ 0x7f65bca9ff45 (unknown)
@ 0x406911 (unknown)
@ (nil) (unknown)
Aborted (core dumped)

I've been using the SSD model in python scripts, and it works great and always gives consistent results. But when I try doing the same thing in C++ with Caffe::Net::Forward(), the results are inconsistent; i.e. for the exact same input I get different outputs each time I forward pass. Is there an obvious reason for this?

1. I want to test a video, not just a picture or obtained from the camera, to test a video, what should I do?
2. for the ( from caffe import layers as L ). I find your reference function{ L.Convolution()}. but i can`t find the function definitions in the layers.Where can I find Convolution() function? thank you

Hi, @weiliu89 , I downloaded the code and changed to the ssd branch. But it can not make when I use cudnn v5 or v4.So I want to ask which cudnn version do the code support.Thanks.

Hi,
I would like to use this for 1080p video. what changes I need to do in config file?
I have to train on my own data set as well.
Any idea about fps? assuming 4-5 objects in an image.

Thanks.
william. j.

File "examples/ssd/score_ssd_pascal.py", line 3, in
from caffe.model_libs import *
ImportError: No module named model_libs
I can not find caffe.model_libs, This error will be related to what. thank you,

Hi, @weiliu89,I ran ssd in another dataset and then I found that the loss is always 0. Do you have any idea of such a problem?

Here are some results presented of the first 130 iterations:

I0628 12:15:56.185905 3080 solver.cpp:231] Iteration 0, loss = 0
I0628 12:15:56.185936 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:15:56.185964 3080 sgd_solver.cpp:106] Iteration 0, lr = 0.001
I0628 12:16:04.980218 3080 solver.cpp:231] Iteration 10, loss = 0
I0628 12:16:04.980249 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:05.367525 3080 sgd_solver.cpp:106] Iteration 10, lr = 0.001
I0628 12:16:14.537703 3080 solver.cpp:231] Iteration 20, loss = 0
I0628 12:16:14.537734 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:14.926493 3080 sgd_solver.cpp:106] Iteration 20, lr = 0.001
I0628 12:16:23.177091 3080 solver.cpp:231] Iteration 30, loss = 0
I0628 12:16:23.177120 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:23.557282 3080 sgd_solver.cpp:106] Iteration 30, lr = 0.001
I0628 12:16:32.177230 3080 solver.cpp:231] Iteration 40, loss = 0
I0628 12:16:32.177299 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:32.177320 3080 sgd_solver.cpp:106] Iteration 40, lr = 0.001
I0628 12:16:41.230350 3080 solver.cpp:231] Iteration 50, loss = 0
I0628 12:16:41.230391 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:41.602286 3080 sgd_solver.cpp:106] Iteration 50, lr = 0.001
I0628 12:16:49.292162 3080 solver.cpp:231] Iteration 60, loss = 0
I0628 12:16:49.292201 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:49.687913 3080 sgd_solver.cpp:106] Iteration 60, lr = 0.001
I0628 12:16:57.399756 3080 solver.cpp:231] Iteration 70, loss = 0
I0628 12:16:57.399794 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:16:57.823508 3080 sgd_solver.cpp:106] Iteration 70, lr = 0.001
I0628 12:17:04.994513 3080 solver.cpp:231] Iteration 80, loss = 0
I0628 12:17:04.994612 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:17:05.400712 3080 sgd_solver.cpp:106] Iteration 80, lr = 0.001
I0628 12:17:12.449678 3080 solver.cpp:231] Iteration 90, loss = 0
I0628 12:17:12.449715 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:17:12.865375 3080 sgd_solver.cpp:106] Iteration 90, lr = 0.001
I0628 12:17:20.008211 3080 solver.cpp:231] Iteration 100, loss = 0
I0628 12:17:20.008249 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:17:20.417125 3080 sgd_solver.cpp:106] Iteration 100, lr = 0.001
I0628 12:17:27.465800 3080 solver.cpp:231] Iteration 110, loss = 0
I0628 12:17:27.465837 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:17:27.878662 3080 sgd_solver.cpp:106] Iteration 110, lr = 0.001
I0628 12:17:34.952018 3080 solver.cpp:231] Iteration 120, loss = 0
I0628 12:17:34.952057 3080 solver.cpp:247] Train net output #0: mbox_loss = 0 (* 1 = 0 loss)
I0628 12:17:35.360599 3080 sgd_solver.cpp:106] Iteration 120, lr = 0.001
I0628 12:17:42.428278 3080 solver.cpp:231] Iteration 130, loss = 0