msracver / flow-guided-feature-aggregation Goto Github PK

Could someone make this model run without GPU? I've tried without success.

Does this repo only work with cuda 9.0?
I got OSError: libcudart.so.9.2: cannot open shared object file: No such file or directory, with may caused by the file in /lib/nms/gpu_nms.cu is compiled under cuda 9.0

Can the model from the demo be used for test on the validation set?

MXNet: 1.3.0
Python: 3.6.6

Running 'python ./fgfa_rfcn/demo.py' gave the following error:
infer_shape error. Arguments:
data: (1, 3, 562, 1000)
im_info: (1, 3)
data_cache: (19, 3, 562, 1000)
feat_cache: (1, 3, 562, 1000)

Traceback (most recent call last):
File "./fgfa_rfcn/demo.py", line 259, in
main()
File "./fgfa_rfcn/demo.py", line 207, in main
pred_result = im_detect(aggr_predictors, data_batch, data_names, scales, cfg)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/tester.py", line 146, in im_detect
output_all = predictor.predict(data_batch)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/tester.py", line 43, in predict
self._mod.forward(data_batch)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/module.py", line 1045, in forward
shared_module=self._curr_module)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/module.py", line 401, in bind
state_names=self._state_names)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 191, in init
self.bind_exec(data_shapes, label_shapes, shared_group)
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 277, in bind_exec
shared_group))
File "/home/p1p13/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 494, in _bind_ith_exec
arg_shapes, _, aux_shapes = self.symbol.infer_shape(**input_shapes)
File "/home/p1p13/incubator-mxnet/python/mxnet/symbol/symbol.py", line 996, in infer_shape
res = self._infer_shape_impl(False, *args, **kwargs)
File "/home/p1p13/incubator-mxnet/python/mxnet/symbol/symbol.py", line 1126, in _infer_shape_impl
ctypes.byref(complete)))
File "/home/p1p13/incubator-mxnet/python/mxnet/base.py", line 255, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Error in operator warping_feat: [16:17:36] src/operator/./bilinear_sampler-inl.h:158: Check failed: dshape[0] == lshape[0] (1 vs. 19)

python: 2.7
mxnet: v0.10.0
I run the demo but meet this error (I follow all the steps in the readme file)

Traceback (most recent call last):
File "./fgfa_rfcn/demo.py", line 257, in
main()
File "./fgfa_rfcn/demo.py", line 155, in main
arg_params=arg_params, aux_params=aux_params)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/tester.py", line 32, in init
self._mod.bind(provide_data, provide_label, for_training=False)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/module.py", line 839, in bind
for_training, inputs_need_grad, force_rebind=False, shared_module=None)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/module.py", line 396, in bind
state_names=self._state_names)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 186, in init
self.bind_exec(data_shapes, label_shapes, shared_group)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 272, in bind_exec
shared_group))
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 545, in _bind_ith_exec
context, self.logger)
File "/home/lxt/Github/Flow-Guided-Feature-Aggregation/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 523, in _get_or_reshape
arg_arr = nd.zeros(arg_shape, context, dtype=arg_type)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/ndarray.py", line 1003, in zeros
return _internal._zeros(shape=shape, ctx=ctx, dtype=dtype)
File "", line 15, in _zeros
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/_ctypes/ndarray.py", line 72, in _imperative_invoke
c_array(ctypes.c_char_p, [c_str(str(val)) for val in vals])))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/base.py", line 84, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [13:34:30] src/c_api/c_api_ndarray.cc:392: Operator _zeros cannot be run; requires at least one of FCompute, NDArrayFunction, FCreateOperator be registered

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7f724aa48f2c]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(MXImperativeInvoke+0x8c9) [0x7f724b7f5209]
[bt] (2) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f724deb0e40]
[bt] (3) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f724deb08ab]
[bt] (4) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7f724e0c03df]
[bt] (5) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7f724e0c4d82]
[bt] (6) python(PyObject_Call+0x43) [0x4b0c93]
[bt] (7) python(PyEval_EvalFrameEx+0x602f) [0x4c9f9f]
[bt] (8) python(PyEval_EvalCodeEx+0x255) [0x4c2705]
[bt] (9) python(PyEval_EvalFrameEx+0x686f) [0x4ca7df]

can i use pip to install mxnet like:
pip install mxnet-cu90

In the README it says

Two config files have been provided so far, namely, frame baseline (R-FCN) and the proposed FGFA for ImageNet VID. We use 4 GPUs to train models on ImageNet VID.

I currently find two configs in the experiments/fgfa_rfcn/cfgs/ directory:

1. fgfa_rfcn_vid_dem.yaml, and
2. resnet_v1_101_flownet_imagenet_vid_rfcn_end2end_ohem.yaml

I assume that the second is for training/testing FGFA, but can the first one be used to train/test the baseline? And why is the first one then used in the demo.py?

Hi, the v0.10.0 seems have been deleted. So, I git check v0.10.0-support, but I cannot run the demo. Could you help me to solve it, please?

hello,How to download imagenet 2015 detection and video detection dataset? Can you share the dataset

I try to use do_python_eval() in lib/dataset/imagenet_vid.py to calculate mAP by classes.
But I get a results like :

AP for airplane = 0.0047
AP for antelope = 0.0028
AP for bear = 0.0019
AP for bicycle = 0.0044
AP for bird = 0.0022
AP for bus = 0.0027
AP for car = 0.0021
AP for cattle = 0.0023
AP for dog = 0.0028
AP for domestic_cat = 0.0021
AP for elephant = 0.0024
AP for fox = 0.0018
AP for giant_panda = 0.0023
AP for hamster = 0.0026
AP for horse = 0.0014
AP for lion = 0.0012
AP for lizard = 0.0015
AP for monkey = 0.0023
AP for motorcycle = 0.0194
AP for rabbit = 0.0013
AP for red_panda = 0.0047
AP for sheep = 0.0009
AP for snake = 0.0013
AP for squirrel = 0.0007
AP for tiger = 0.0032
AP for train = 0.0014
AP for turtle = 0.0017
AP for watercraft = 0.0033
AP for whale = 0.0015
AP for zebra = 0.0030
Mean [email protected] = 0.0029

But I can get mAP = 0.7420, when i use do_python_eval_gen.
What the trouble is it？
How can i calculate it correctly？

Thank you！

In your paper you set lr = 0.001 when train on the first 80k in ImageNet DET training set, and set lr = 0.0001 when train on the last 40k in ImageNet DET training set, and so on. so I want to know if the learning rate will change when training the FGFA end-to-end?

I notice that lib/dataset/imagenet_vid_groundtruth_motion_iou.mat contains ious between frames on val set. For some reasons, I want to generate similar file on train set. Any code to generate this file? or it's annotated by hand?

cuda version :8.0
cudnn version:7.0.5
gcc version:4.8.5
what should I do?

ck (most recent call last):
File "setup_linux.py", line 63, in
CUDA = locate_cuda()
File "setup_linux.py", line 58, in locate_cuda
for k, v in cudaconfig.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'

In the paper, the learning rate is 0.001. However, in the source code, the learning rate is 0.00025, as set in the yaml file. Why there is such a difference?

You are referring to two config files that have been provided the frame baseline (R-FCN) and the proposed FGFA for ImageNet VID.

In my folder ./experiments/fgfa_rfcn/cfgs/ I can see 2 config files although the second one being named fgfa_rfcn_vid_demo.yaml. Is this the baseline config file you are referring to? The other one is obviously the proposed FGFA method.

Hi,

for me demo.py wouldn't execute without errors, unless I compiled mxnet with the USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda-8.0 USE_CUDNN=1 flags. Maybe you could/should add this to the README.

Cheers,
Moritz

Hi

I was just wondering the difference in the [email protected] scores for the two files. So in the imagenet_vid_eval you get classwise AP but when all these are averaged we only get 0.6351 but then the mAP from the eval_motion script gives 0.7711. I assume there is some conceptual difference in how the scores are calculated/averaged?

Update: Fixed, it was an issue with the detections I was using.

Thanks,
Hayden

It seems to me the rescale_grad should be set to 1/batch_size. In case of 4 GPUs training rescale_grad should be 0.25. Why is the value set to "1.0" in train_end2end.py?

The provided models were trained on the ImageNet VID dataset, which does not include a 'person' label. Does anyone know where can I find a trained model for persons?

Also, does anyone know of a Tensorflow/Pytorch implementation of this architecture? I'm having too much trouble making it run.

Hello,

I am running demo.py and I am getting for these two variables (they are dict variables in output_all dict in im_detect function in fgfa_rfcn/tester.py file) nan and inf respectively.
This erroneous functionality comes from the Predictor class in fgfa_rfcn/tester.py
Any thoughts?

In 'VID_train_15frames.txt':
It looks like:
'train/ILSVRC2015_VID_train_0000/ILSVRC2015_train_00000000 1 10 300'
In 'VID_val_videos.txt':
It looks like:
'val/ILSVRC2015_val_00000000 1 0 464'
What's thr meaning of the last three numbers?

Thanks for releasing the code. How is motion iou calculated when the length of object instance is less than 10 frames?

importerror: no mudule named dill

There are some trouble to use OneDrive to download the pretrained model in China.
Can you provide another way ( like BaiduYun or Google )to download, please!
Thank you!

I was trying to train to the ILSVRC2015 data and since they data is large I had to put on an external hard drive and point to it.
I have modified the .yaml file as follows:

dataset:
  NUM_CLASSES: 31
  dataset: ImageNetVID
# dataset_path: "./data/ILSVRC2015"
  dataset_path: "/media/stormnet/easystore/ILSVRC2015"
  image_set: DET_train_30classes+VID_train_15frames
#  root_path: "./data"
  root_path: "/media/stormnet/easystore"
  test_image_set: VID_val_videos
  proposal: rpn
  motion_iou_path: './lib/dataset/imagenet_vid_groundtruth_motion_iou.mat'
  enable_detailed_eval: true 

However, I am getting this error:

raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [14:07:18] src/io/local_filesys.cc:154: Check failed: allow_null  LocalFileSystem: fail to open "./output/fgfa_rfcn/imagenet_vid/resnet_v1_101_flownet_imagenet_vid_rfcn_end2end_ohem/VID_val_videos/../DET_train_30classes_VID_train_15frames/fgfa_rfcn_vid-0002.params"

Why is it complaining about this missing file? why is it needed anyways?

Thanks in advance.

Hello. When I run the demo.py, I meet the problem below. My workstation is configured with an NVIDIA GTX 1070(8G). The version of Mxnet is 1.0.0. Looking forward to your reply. Thank you.

[19:45:05] /home/travis/build/dmlc/mxnet-distro/mxnet-build/dmlc-core/include/dmlc/logging.h:308: [19:45:05] src/storage/./pooled_storage_manager.h:107: cudaMalloc failed: out of memory

Stack trace returned 10 entries:
[bt] (0) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x28965c) [0x7fef5294865c]
[bt] (1) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29c846f) [0x7fef5508746f]
[bt] (2) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ca628) [0x7fef55089628]
[bt] (3) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ceb41) [0x7fef5508db41]
[bt] (4) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2a4ec11) [0x7fef5510dc11]
[bt] (5) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x3b6879b) [0x7fef5622779b]
[bt] (6) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x268f834) [0x7fef54d4e834]
[bt] (7) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246960b) [0x7fef54b2860b]
[bt] (8) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246e760) [0x7fef54b2d760]
[bt] (9) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2450f1d) [0x7fef54b0ff1d]

[19:45:05] /home/travis/build/dmlc/mxnet-distro/mxnet-build/dmlc-core/include/dmlc/logging.h:308: [19:45:05] src/engine/./threaded_engine.h:359: [19:45:05] src/storage/./pooled_storage_manager.h:107: cudaMalloc failed: out of memory

Stack trace returned 10 entries:
[bt] (0) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x28965c) [0x7fef5294865c]
[bt] (1) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29c846f) [0x7fef5508746f]
[bt] (2) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ca628) [0x7fef55089628]
[bt] (3) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ceb41) [0x7fef5508db41]
[bt] (4) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2a4ec11) [0x7fef5510dc11]
[bt] (5) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x3b6879b) [0x7fef5622779b]
[bt] (6) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x268f834) [0x7fef54d4e834]
[bt] (7) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246960b) [0x7fef54b2860b]
[bt] (8) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246e760) [0x7fef54b2d760]
[bt] (9) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2450f1d) [0x7fef54b0ff1d]

A fatal error occurred in asynchronous engine operation. If you do not know what caused this error, you can try set environment variable MXNET_ENGINE_TYPE to NaiveEngine and run with debugger (i.e. gdb). This will force all operations to be synchronous and backtrace will give you the series of calls that lead to this error. Remember to set MXNET_ENGINE_TYPE back to empty after debugging.

Stack trace returned 8 entries:
[bt] (0) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x28965c) [0x7fef5294865c]
[bt] (1) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24511c4) [0x7fef54b101c4]
[bt] (2) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2455d5b) [0x7fef54b14d5b]
[bt] (3) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2455f36) [0x7fef54b14f36]
[bt] (4) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x245275b) [0x7fef54b1175b]
[bt] (5) /home/smiles/anaconda2/bin/../lib/libstdc++.so.6(+0xb7260) [0x7fefc3111260]
[bt] (6) /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184) [0x7fefc8747184]
[bt] (7) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fefc7d66ffd]

terminate called after throwing an instance of 'dmlc::Error'
what(): [19:45:05] src/engine/./threaded_engine.h:359: [19:45:05] src/storage/./pooled_storage_manager.h:107: cudaMalloc failed: out of memory

Stack trace returned 10 entries:
[bt] (0) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x28965c) [0x7fef5294865c]
[bt] (1) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29c846f) [0x7fef5508746f]
[bt] (2) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ca628) [0x7fef55089628]
[bt] (3) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x29ceb41) [0x7fef5508db41]
[bt] (4) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2a4ec11) [0x7fef5510dc11]
[bt] (5) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x3b6879b) [0x7fef5622779b]
[bt] (6) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x268f834) [0x7fef54d4e834]
[bt] (7) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246960b) [0x7fef54b2860b]
[bt] (8) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x246e760) [0x7fef54b2d760]
[bt] (9) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2450f1d) [0x7fef54b0ff1d]

A fatal error occurred in asynchronous engine operation. If you do not know what caused this error, you can try set environment variable MXNET_ENGINE_TYPE to NaiveEngine and run with debugger (i.e. gdb). This will force all operations to be synchronous and backtrace will give you the series of calls that lead to this error. Remember to set MXNET_ENGINE_TYPE back to empty after debugging.

Stack trace returned 8 entries:
[bt] (0) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x28965c) [0x7fef5294865c]
[bt] (1) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24511c4) [0x7fef54b101c4]
[bt] (2) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2455d5b) [0x7fef54b14d5b]
[bt] (3) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2455f36) [0x7fef54b14f36]
[bt] (4) /home/smiles/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x245275b) [0x7fef54b1175b]
[bt] (5) /home/smiles/anaconda2/bin/../lib/libstdc++.so.6(+0xb7260) [0x7fefc3111260]
[bt] (6) /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184) [0x7fefc8747184]
[bt] (7) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fefc7d66ffd]

Aborted (core dumped)

Why every file root ends with a one?
like train/ILSVRC2013_train/n02391049/n02391049_8321 1
train/ILSVRC2013_train/n02391049/n02391049_8340 1
train/ILSVRC2013_train/n02391049/n02391049_835 1
Useless? Am i right?
Thanks?

I test an image from my own dataset, and it occurs the following error in demo.py, how to solve it?
Another question is I want to finetune my own dataset, how to generate "groundtruth_motion_iou" file in config.py? Please help me~

get-predictor
Traceback (most recent call last):
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/demo.py", line 259, in
main()
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/demo.py", line 232, in main
pred_result = im_detect(aggr_predictors, data_batch, data_names, scales, cfg)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/tester.py", line 144, in im_detect
output_all = predictor.predict(data_batch)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/tester.py", line 41, in predict
self._mod.forward(data_batch)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/module.py", line 1045, in forward
shared_module=self._curr_module)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/module.py", line 401, in bind
state_names=self._state_names)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 191, in init
self.bind_exec(data_shapes, label_shapes, shared_group)
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 277, in bind_exec
shared_group))
File "/home/sipl03/lx/code/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/core/DataParallelExecutorGroup.py", line 494, in _bind_ith_exec
arg_shapes, _, aux_shapes = self.symbol.infer_shape(**input_shapes)
File "/usr/local/lib/python2.7/dist-packages/mxnet-1.3.0-py2.7.egg/mxnet/symbol/symbol.py", line 996, in infer_shape
res = self._infer_shape_impl(False, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-1.3.0-py2.7.egg/mxnet/symbol/symbol.py", line 1126, in _infer_shape_impl
ctypes.byref(complete)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-1.3.0-py2.7.egg/mxnet/base.py", line 255, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Error in operator concat0: [11:01:47] src/operator/nn/concat.cc:66: Check failed: shape_assign(&(*in_shape)[i], dshape) Incompatible input shape: expected [19,0,794,600], got [10,3,794,600]

运行demo程序的时候正常，但是运行fgfa_rfcn_end2end_train_test.py进行训练的时候就会报错，提示为“段错误（核心已转储），是在./fgfa_rfcn/core文件夹中的module.py文件中fit函数出错，我百思不得其解，希望各位大神能帮我解决疑惑。

As title.

I changed 'SEQ_NMS': True in cfgs, when I tested my model with two gpus and error happened like
#############################
Traceback (most recent call last):

File "experiments/fgfa_rfcn/fgfa_rfcn_end2end_train_test.py", line 20,
in
test.main()

File "experiments/fgfa_rfcn/../../fgfa_rfcn/test.py", line 53,
in main
enable_detailed_eval=config.dataset.enable_detailed_eval)

File "experiments/fgfa_rfcn/../../fgfa_rfcn/function/test_rcnn.py", line 85,
in test_rcnn
pred_eval_multiprocess(gpu_num, feat_predictors, aggr_predictors, test_datas, imdb, cfg, vis=vis, ignore_cache=ignore_cache, thresh=thresh, logger=logger)

File "experiments/fgfa_rfcn/../../fgfa_rfcn/core/tester.py", line 408, in pred_eval_multiprocess
res.append(job.get())

File "/home/xaserver/anaconda3/envs/py2/lib/python2.7/multiprocessing/pool.py", line 572, 6
in get
raise self._value

TypeError: write_vid_results_multiprocess() takes exactly 2 arguments (3 given)
###########################
and I found the related funsions in tester.py and imagenet_vid.py, the related information are:
for i in range(gpu_num):
job = apply_async(pool, pred_eval_seqnms, (i, imdb))
jobs.append(job)
for job in jobs:
res.append(job.get())
def run_dill_encode(payload):
fun,args=dill.loads(payload)
return fun(*args)

def apply_async(pool,fun,args):
payload=dill.dumps((fun,args))
return pool.apply_async(run_dill_encode,(payload,))
def write_vid_results_multiprocess(self, detection, gpu_id):

and who knows how to make it right? thx.

The version of mxnet is v0.10.0
I managed to run the demo, but when I try to train, an error occurs:

Namespace(cfg='experiments/fgfa_rfcn/cfgs/resnet_v1_101_flownet_imagenet_vid_rfcn_end2end_ohem.yaml', ignore_cache=False, shuffle=False, thresh=0.001, vis=False)
{'CLASS_AGNOSTIC': True,
 'MXNET_VERSION': '',
 'SCALES': [(600, 1000)],
 'TEST': {'BATCH_IMAGES': 1,
          'CXX_PROPOSAL': True,
          'HAS_RPN': True,
          'KEY_FRAME_INTERVAL': 9,
          'NMS': 0.3,
          'RPN_MIN_SIZE': 0,
          'RPN_NMS_THRESH': 0.7,
          'RPN_POST_NMS_TOP_N': 300,
          'RPN_PRE_NMS_TOP_N': 6000,
          'SEQ_NMS': False,
          'max_per_image': 300,
          'test_epoch': 2},
 'TRAIN': {'ASPECT_GROUPING': True,
           'BATCH_IMAGES': 1,
           'BATCH_ROIS': -1,
           'BATCH_ROIS_OHEM': 128,
           'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0],
           'BBOX_NORMALIZATION_PRECOMPUTED': True,
           'BBOX_REGRESSION_THRESH': 0.5,
           'BBOX_STDS': [0.1, 0.1, 0.2, 0.2],
           'BBOX_WEIGHTS': array([1., 1., 1., 1.]),
           'BG_THRESH_HI': 0.5,
           'BG_THRESH_LO': 0.0,
           'CXX_PROPOSAL': True,
           'ENABLE_OHEM': True,
           'END2END': True,
           'FG_FRACTION': 0.25,
           'FG_THRESH': 0.5,
           'FLIP': True,
           'MAX_OFFSET': 9,
           'MIN_OFFSET': -9,
           'RESUME': False,
           'RPN_BATCH_SIZE': 256,
           'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'RPN_CLOBBER_POSITIVES': False,
           'RPN_FG_FRACTION': 0.5,
           'RPN_MIN_SIZE': 0,
           'RPN_NEGATIVE_OVERLAP': 0.3,
           'RPN_NMS_THRESH': 0.7,
           'RPN_POSITIVE_OVERLAP': 0.7,
           'RPN_POSITIVE_WEIGHT': -1.0,
           'RPN_POST_NMS_TOP_N': 300,
           'RPN_PRE_NMS_TOP_N': 6000,
           'SHUFFLE': True,
           'begin_epoch': 0,
           'end_epoch': 2,
           'lr': 0.00025,
           'lr_factor': 0.1,
           'lr_step': '1.333',
           'model_prefix': 'fgfa_rfcn_vid',
           'momentum': 0.9,
           'warmup': False,
           'warmup_lr': 0,
           'warmup_step': 0,
           'wd': 0.0005},
 'dataset': {'NUM_CLASSES': 31,
             'dataset': 'ImageNetVID',
             'dataset_path': './data/ILSVRC2015',
             'enable_detailed_eval': True,
             'image_set': 'DET_train_30classes+VID_train_15frames',
             'motion_iou_path': './lib/dataset/imagenet_vid_groundtruth_motion_iou.mat',
             'proposal': 'rpn',
             'root_path': './data',
             'test_image_set': 'VID_val_videos'},
 'default': {'frequent': 100, 'kvstore': 'device'},
 'gpus': '0,1,2,3',
 'network': {'ANCHOR_MEANS': [0.0, 0.0, 0.0, 0.0],
             'ANCHOR_RATIOS': [0.5, 1, 2],
             'ANCHOR_SCALES': [8, 16, 32],
             'ANCHOR_STDS': [0.1, 0.1, 0.4, 0.4],
             'FGFA_FEAT_DIM': 3072,
             'FIXED_PARAMS': ['conv1', 'res2', 'bn'],
             'IMAGE_STRIDE': 0,
             'NORMALIZE_RPN': True,
             'NUM_ANCHORS': 9,
             'PIXEL_MEANS': array([103.06, 115.9 , 123.15]),
             'RCNN_FEAT_STRIDE': 16,
             'RPN_FEAT_STRIDE': 16,
             'pretrained': './model/pretrained_model/resnet_v1_101',
             'pretrained_epoch': 0,
             'pretrained_flow': './model/pretrained_model/flownet'},
 'output_path': './output/fgfa_rfcn/imagenet_vid',
 'symbol': 'resnet_v1_101_flownet_rfcn'}
num_images 555
ImageNetVID_VID_val_videos gt roidb loaded from ./data/cache/ImageNetVID_VID_val_videos_gt_roidb.pkl
[12:10:52] include/dmlc/logging.h:304: [12:10:52] src/io/local_filesys.cc:154: Check failed: allow_null  LocalFileSystem: fail to open "./output/fgfa_rfcn/imagenet_vid/resnet_v1_101_flownet_imagenet_vid_rfcn_end2end_ohem/VID_val_videos/../DET_train_30classes_VID_train_15frames/fgfa_rfcn_vid-0002.params"

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io15LocalFileSystem4OpenERKNS0_3URIEPKcb+0x640) [0x7f9adb858bf0]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc6Stream6CreateEPKcS2_b+0x1c6) [0x7f9adb84a446]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(MXNDArrayLoad+0x4a5) [0x7f9adb7ddc25]
[bt] (3) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f9af9b0de40]
[bt] (4) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f9af9b0d8ab]
[bt] (5) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7f9af9d1d3df]
[bt] (6) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7f9af9d21d82]
[bt] (7) python(PyEval_EvalFrameEx+0x578f) [0x4c15bf]
[bt] (8) python(PyEval_EvalCodeEx+0x306) [0x4b9ab6]
[bt] (9) python(PyEval_EvalFrameEx+0x603f) [0x4c1e6f]

Traceback (most recent call last):
  File "experiments/fgfa_rfcn/fgfa_rfcn_end2end_train_test.py", line 21, in <module>
    test.main()
  File "experiments/fgfa_rfcn/../../fgfa_rfcn/test.py", line 58, in main
    enable_detailed_eval=config.dataset.enable_detailed_eval)
  File "experiments/fgfa_rfcn/../../fgfa_rfcn/function/test_rcnn.py", line 83, in test_rcnn
    arg_params, aux_params = load_param(prefix, epoch, process=True)
  File "experiments/fgfa_rfcn/../../fgfa_rfcn/../lib/utils/load_model.py", line 57, in load_param
    arg_params, aux_params = load_checkpoint(prefix, epoch)
  File "experiments/fgfa_rfcn/../../fgfa_rfcn/../lib/utils/load_model.py", line 23, in load_checkpoint
    save_dict = mx.nd.load('%s-%04d.params' % (prefix, epoch))
  File "/usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/ndarray.py", line 2100, in load
    ctypes.byref(names)))
  File "/usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/base.py", line 84, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [12:10:52] src/io/local_filesys.cc:154: Check failed: allow_null  LocalFileSystem: fail to open "./output/fgfa_rfcn/imagenet_vid/resnet_v1_101_flownet_imagenet_vid_rfcn_end2end_ohem/VID_val_videos/../DET_train_30classes_VID_train_15frames/fgfa_rfcn_vid-0002.params"

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc2io15LocalFileSystem4OpenERKNS0_3URIEPKcb+0x640) [0x7f9adb858bf0]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc6Stream6CreateEPKcS2_b+0x1c6) [0x7f9adb84a446]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.0-py2.7.egg/mxnet/libmxnet.so(MXNDArrayLoad+0x4a5) [0x7f9adb7ddc25]
[bt] (3) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f9af9b0de40]
[bt] (4) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f9af9b0d8ab]
[bt] (5) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7f9af9d1d3df]
[bt] (6) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7f9af9d21d82]
[bt] (7) python(PyEval_EvalFrameEx+0x578f) [0x4c15bf]
[bt] (8) python(PyEval_EvalCodeEx+0x306) [0x4b9ab6]
[bt] (9) python(PyEval_EvalFrameEx+0x603f) [0x4c1e6f]

How can I solve this problem?

how to discriminate slow, medium and fast motions

MXNet ： 1.0
Python ; 2.7.14
I try to run the demo, but an error like following:

Traceback (most recent call last):
File "./fgfa_rfcn/demo.py", line 257, in
main()
File "./fgfa_rfcn/demo.py", line 99, in main
aggr_sym = aggr_sym_instance.get_aggregation_symbol(cfg)
File "/home/liyang/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/symbols/resnet_v1_101_flownet_rfcn.py", line 1081, in get_aggregation_symbol
aggregated_conv_feat += tiled_weight * warp_list[i]
File "/home/liyang/anaconda2/lib/python2.7/site-packages/mxnet-1.0.0-py2.7.egg/mxnet/symbol/symbol.py", line 104, in iadd
raise NotImplementedForSymbol(self.iadd, '+=', other, 1)
mxnet.base.NotImplementedForSymbol: Function iadd (namely operator "+=") with arguments (<class 'mxnet.symbol.symbol.Symbol'>, <type 'int'>) is not implemented for Symbol and only available in NDArray.

How can I fix that issue?
Thank you!

Hi, @authors,
Thanks for your works. Now I want to use my own datasets to train. So first I maybe to do my datasets as ILSVRC2015. But I cannot download the ILSVRC2015 for some reason. So can you tell me details about ILSVRC2015? (1) "ILSVRC2015/Annotations/VID", I know this folder have groundtruth information about every video in xml. But I do not know the content about the xml . So can you give me a xml as example. (2)If a video has 30 frames, is it has 30 xmls?
My email is : [email protected]. Thank you very much!

I read the code and find that, the training and the test networks are different in structure.
In particular, in training network, the embedding layers are added after the concatenation of the output of ResNet-101 and FlowNet, as is shown in the following code:

# In function get_train_symbol()
# conv_feat is output of ResNet, warp_conv_feat_1{or 2} is output of FlowNet
concat_embed_data = mx.symbol.Concat(*[conv_feat[0], warp_conv_feat_1, warp_conv_feat_2], dim=0) 
embed_output = self.get_embednet(concat_embed_data) 

However, in the test network, the structure becomes very different. The embedding layers are added only to the output of ResNet-101. And then the output of the embedding layers are concatenated with the output of FlowNet, as is shown in the following code:

# In function get_feat_symbol()
conv_feat = self.get_resnet_v1(data)
embed_feat = self.get_embednet(conv_feat) # embedding network is added right after ResNet
conv_embed = mx.sym.Concat(conv_feat, embed_feat, name="conv_embed")
... # some codes
# In function get_aggregation_symbol()
# flow is the output of FlowNet, feat_cache is the output of embedding layers
flow_grid = mx.sym.GridGenerator(data=flow, transform_type='warp', name='flow_grid')
conv_feat = mx.sym.BilinearSampler(data=feat_cache, grid=flow_grid, name='warping_feat')
embed_output = mx.symbol.slice_axis(conv_feat, axis=1, begin=1024, end=3072)

Why are the networks so different between training and test?

Moreover, when I'm training the network, the training Acc stays high, like 0.98, but when I test the model on demo images, I get a terrible result.

I clone the project and configure all the data and model paths following the instructions. I tried to continue to train from the model for demo, i.e.

./model/rfcn_fgfa_flownet_vid-0000.params

which is downloaded from the OneDrive url provided in the README.

The logs looks normal, as follows:

Epoch[0] Batch [100]    Speed: 1.04 samples/sec Train-RPNAcc=0.938544,  RPNLogLoss=0.207737,    RPNL1Loss=0.354940,     RCNNAcc=0.842822,       RCNNLogLoss=1.380372,   RCNNL1Loss=0.245006,
Epoch[0] Batch [200]    Speed: 1.04 samples/sec Train-RPNAcc=0.954699,  RPNLogLoss=0.151813,    RPNL1Loss=0.258118,     RCNNAcc=0.815532,       RCNNLogLoss=1.306230,   RCNNL1Loss=0.368251,
Epoch[0] Batch [300]    Speed: 1.05 samples/sec Train-RPNAcc=0.963273,  RPNLogLoss=0.122710,    RPNL1Loss=0.220691,     RCNNAcc=0.805233,       RCNNLogLoss=1.272574,   RCNNL1Loss=0.387604,
Epoch[0] Batch [400]    Speed: 1.07 samples/sec Train-RPNAcc=0.967464,  RPNLogLoss=0.108166,    RPNL1Loss=0.203036,     RCNNAcc=0.800499,       RCNNLogLoss=1.245192,   RCNNL1Loss=0.370216,
Epoch[0] Batch [500]    Speed: 1.05 samples/sec Train-RPNAcc=0.971533,  RPNLogLoss=0.095719,    RPNL1Loss=0.188637,     RCNNAcc=0.796875,       RCNNLogLoss=1.218545,   RCNNL1Loss=0.347318,
Epoch[0] Batch [600]    Speed: 1.07 samples/sec Train-RPNAcc=0.974242,  RPNLogLoss=0.087136,    RPNL1Loss=0.177525,     RCNNAcc=0.797629,       RCNNLogLoss=1.187403,   RCNNL1Loss=0.326912,
Epoch[0] Batch [700]    Speed: 1.07 samples/sec Train-RPNAcc=0.976512,  RPNLogLoss=0.079193,    RPNL1Loss=0.166786,     RCNNAcc=0.799628,       RCNNLogLoss=1.147833,   RCNNL1Loss=0.308086,
Epoch[0] Batch [800]    Speed: 1.06 samples/sec Train-RPNAcc=0.977943,  RPNLogLoss=0.074186,    RPNL1Loss=0.164597,     RCNNAcc=0.800503,       RCNNLogLoss=1.122886,   RCNNL1Loss=0.292854,
Epoch[0] Batch [900]    Speed: 1.07 samples/sec Train-RPNAcc=0.978635,  RPNLogLoss=0.071338,    RPNL1Loss=0.158227,     RCNNAcc=0.798418,       RCNNLogLoss=1.113350,   RCNNL1Loss=0.283827,
Epoch[0] Batch [1000]   Speed: 1.05 samples/sec Train-RPNAcc=0.979403,  RPNLogLoss=0.068680,    RPNL1Loss=0.151551,     RCNNAcc=0.800332,       RCNNLogLoss=1.086266,   RCNNL1Loss=0.271887,
Epoch[0] Batch [1100]   Speed: 1.07 samples/sec Train-RPNAcc=0.980064,  RPNLogLoss=0.066033,    RPNL1Loss=0.149408,     RCNNAcc=0.798940,       RCNNLogLoss=1.076426,   RCNNL1Loss=0.264026,
Epoch[0] Batch [1200]   Speed: 1.06 samples/sec Train-RPNAcc=0.980735,  RPNLogLoss=0.063761,    RPNL1Loss=0.144844,     RCNNAcc=0.797968,       RCNNLogLoss=1.063309,   RCNNL1Loss=0.260105,
Epoch[0] Batch [1300]   Speed: 1.05 samples/sec Train-RPNAcc=0.981171,  RPNLogLoss=0.062085,    RPNL1Loss=0.142027,     RCNNAcc=0.798472,       RCNNLogLoss=1.046781,   RCNNL1Loss=0.254449,
Epoch[0] Batch [1400]   Speed: 1.07 samples/sec Train-RPNAcc=0.981467,  RPNLogLoss=0.061773,    RPNL1Loss=0.138603,     RCNNAcc=0.801297,       RCNNLogLoss=1.017234,   RCNNL1Loss=0.248268,
Epoch[0] Batch [1500]   Speed: 1.06 samples/sec Train-RPNAcc=0.981986,  RPNLogLoss=0.060457,    RPNL1Loss=0.135045,     RCNNAcc=0.803818,       RCNNLogLoss=0.991378,   RCNNL1Loss=0.243015,
Epoch[0] Batch [1600]   Speed: 1.05 samples/sec Train-RPNAcc=0.982362,  RPNLogLoss=0.059222,    RPNL1Loss=0.132028,     RCNNAcc=0.805146,       RCNNLogLoss=0.970428,   RCNNL1Loss=0.242290,
Epoch[0] Batch [1700]   Speed: 1.05 samples/sec Train-RPNAcc=0.982717,  RPNLogLoss=0.058142,    RPNL1Loss=0.131539,     RCNNAcc=0.808362,       RCNNLogLoss=0.943044,   RCNNL1Loss=0.237796,
Epoch[0] Batch [1800]   Speed: 1.07 samples/sec Train-RPNAcc=0.982972,  RPNLogLoss=0.057485,    RPNL1Loss=0.130012,     RCNNAcc=0.810964,       RCNNLogLoss=0.919282,   RCNNL1Loss=0.234166,
Epoch[0] Batch [1900]   Speed: 1.07 samples/sec Train-RPNAcc=0.983284,  RPNLogLoss=0.056381,    RPNL1Loss=0.127654,     RCNNAcc=0.813182,       RCNNLogLoss=0.898093,   RCNNL1Loss=0.231330,
Epoch[0] Batch [2000]   Speed: 1.06 samples/sec Train-RPNAcc=0.983577,  RPNLogLoss=0.055477,    RPNL1Loss=0.125975,     RCNNAcc=0.816303,       RCNNLogLoss=0.874262,   RCNNL1Loss=0.228761,
Epoch[0] Batch [2100]   Speed: 1.06 samples/sec Train-RPNAcc=0.983429,  RPNLogLoss=0.055541,    RPNL1Loss=0.124450,     RCNNAcc=0.818520,       RCNNLogLoss=0.854937,   RCNNL1Loss=0.227023,
Epoch[0] Batch [2200]   Speed: 1.06 samples/sec Train-RPNAcc=0.983559,  RPNLogLoss=0.055248,    RPNL1Loss=0.123187,     RCNNAcc=0.821203,       RCNNLogLoss=0.835192,   RCNNL1Loss=0.223909,
Epoch[0] Batch [2300]   Speed: 1.06 samples/sec Train-RPNAcc=0.983856,  RPNLogLoss=0.054266,    RPNL1Loss=0.122544,     RCNNAcc=0.822937,       RCNNLogLoss=0.818458,   RCNNL1Loss=0.222563,
Epoch[0] Batch [2400]   Speed: 1.06 samples/sec Train-RPNAcc=0.983945,  RPNLogLoss=0.054052,    RPNL1Loss=0.120982,     RCNNAcc=0.823651,       RCNNLogLoss=0.804805,   RCNNL1Loss=0.222604,
Epoch[0] Batch [2500]   Speed: 1.06 samples/sec Train-RPNAcc=0.984273,  RPNLogLoss=0.053099,    RPNL1Loss=0.119158,     RCNNAcc=0.824958,       RCNNLogLoss=0.790649,   RCNNL1Loss=0.220705,
Epoch[0] Batch [2600]   Speed: 1.07 samples/sec Train-RPNAcc=0.984386,  RPNLogLoss=0.052425,    RPNL1Loss=0.118785,     RCNNAcc=0.827017,       RCNNLogLoss=0.774911,   RCNNL1Loss=0.218256,
Epoch[0] Batch [2700]   Speed: 1.05 samples/sec Train-RPNAcc=0.984553,  RPNLogLoss=0.051816,    RPNL1Loss=0.117867,     RCNNAcc=0.827972,       RCNNLogLoss=0.762744,   RCNNL1Loss=0.217780,
Epoch[0] Batch [2800]   Speed: 1.06 samples/sec Train-RPNAcc=0.984407,  RPNLogLoss=0.052380,    RPNL1Loss=0.118125,     RCNNAcc=0.829447,       RCNNLogLoss=0.749248,   RCNNL1Loss=0.216280,
Epoch[0] Batch [2900]   Speed: 1.06 samples/sec Train-RPNAcc=0.984538,  RPNLogLoss=0.051778,    RPNL1Loss=0.116990,     RCNNAcc=0.831085,       RCNNLogLoss=0.736197,   RCNNL1Loss=0.214425,
Epoch[0] Batch [3000]   Speed: 1.07 samples/sec Train-RPNAcc=0.984686,  RPNLogLoss=0.051383,    RPNL1Loss=0.116145,     RCNNAcc=0.832022,       RCNNLogLoss=0.726220,   RCNNL1Loss=0.214046,
Epoch[0] Batch [3100]   Speed: 1.05 samples/sec Train-RPNAcc=0.984793,  RPNLogLoss=0.051247,    RPNL1Loss=0.115204,     RCNNAcc=0.833453,       RCNNLogLoss=0.715207,   RCNNL1Loss=0.212304,
Epoch[0] Batch [3200]   Speed: 1.06 samples/sec Train-RPNAcc=0.984917,  RPNLogLoss=0.050870,    RPNL1Loss=0.114867,     RCNNAcc=0.834683,       RCNNLogLoss=0.703953,   RCNNL1Loss=0.210789,
Epoch[0] Batch [3300]   Speed: 1.06 samples/sec Train-RPNAcc=0.984955,  RPNLogLoss=0.050687,    RPNL1Loss=0.114899,     RCNNAcc=0.835670,       RCNNLogLoss=0.694319,   RCNNL1Loss=0.209543,
Epoch[0] Batch [3400]   Speed: 1.06 samples/sec Train-RPNAcc=0.984918,  RPNLogLoss=0.050745,    RPNL1Loss=0.114710,     RCNNAcc=0.836985,       RCNNLogLoss=0.684625,   RCNNL1Loss=0.208034,
Epoch[0] Batch [3500]   Speed: 1.07 samples/sec Train-RPNAcc=0.984995,  RPNLogLoss=0.050442,    RPNL1Loss=0.113783,     RCNNAcc=0.838238,       RCNNLogLoss=0.675153,   RCNNL1Loss=0.206618,
Epoch[0] Batch [3600]   Speed: 1.07 samples/sec Train-RPNAcc=0.985046,  RPNLogLoss=0.050316,    RPNL1Loss=0.113588,     RCNNAcc=0.839296,       RCNNLogLoss=0.666437,   RCNNL1Loss=0.205760,
Epoch[0] Batch [3700]   Speed: 1.07 samples/sec Train-RPNAcc=0.984950,  RPNLogLoss=0.050420,    RPNL1Loss=0.113458,     RCNNAcc=0.840301,       RCNNLogLoss=0.658199,   RCNNL1Loss=0.204153,
Epoch[0] Batch [3800]   Speed: 1.07 samples/sec Train-RPNAcc=0.985015,  RPNLogLoss=0.050012,    RPNL1Loss=0.113324,     RCNNAcc=0.841688,       RCNNLogLoss=0.649121,   RCNNL1Loss=0.202450,
Epoch[0] Batch [3900]   Speed: 1.07 samples/sec Train-RPNAcc=0.985168,  RPNLogLoss=0.049591,    RPNL1Loss=0.112202,     RCNNAcc=0.842604,       RCNNLogLoss=0.642186,   RCNNL1Loss=0.202128,
Epoch[0] Batch [4000]   Speed: 1.06 samples/sec Train-RPNAcc=0.985318,  RPNLogLoss=0.049009,    RPNL1Loss=0.111678,     RCNNAcc=0.843418,       RCNNLogLoss=0.635388,   RCNNL1Loss=0.201368,
Epoch[0] Batch [4100]   Speed: 1.06 samples/sec Train-RPNAcc=0.985317,  RPNLogLoss=0.048885,    RPNL1Loss=0.111040,     RCNNAcc=0.844182,       RCNNLogLoss=0.628305,   RCNNL1Loss=0.200417,
Epoch[0] Batch [4200]   Speed: 1.06 samples/sec Train-RPNAcc=0.985169,  RPNLogLoss=0.049311,    RPNL1Loss=0.111755,     RCNNAcc=0.844617,       RCNNLogLoss=0.622948,   RCNNL1Loss=0.200082,
Epoch[0] Batch [4300]   Speed: 0.72 samples/sec Train-RPNAcc=0.985110,  RPNLogLoss=0.049333,    RPNL1Loss=0.112316,     RCNNAcc=0.845354,       RCNNLogLoss=0.616863,   RCNNL1Loss=0.199422,

But when I run the model checkpoint on the demo images, as used in the demo.py, I get tens of boxes overlapping each other, and I can hardly figure out which one is good. But as in groundtruth, there are only 7 planes in each image. This is strange because even if I use the initial checkpoint provided for demo, I get good detection results, with 7 bounding boxes each covering a plane perfectly.

What's wrong with my training?

The only changes I make to the code are the model and data paths, as well as gpu ids.

I run the demo.py. It generates the result.But When I combined all frames into video.It is out of order.Why? Thanks for your help.

I run python ./fgfa_rfcn/demo.py
then have a problem,
Traceback (most recent call last):
File "./fgfa_rfcn/demo.py", line 21, in
from utils.image import resize, transform
File "/home/grade3_weiyun/projects/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/../lib/utils/image.py", line 6, in
from bbox.bbox_transform import clip_boxes
File "/home/grade3_weiyun/projects/Flow-Guided-Feature-Aggregation-master/fgfa_rfcn/../lib/bbox/bbox_transform.py", line 3, in
from bbox import bbox_overlaps_cython
ImportError: cannot import name bbox_overlaps_cython

how should i do?

flake8 testing of https://github.com/msracver/Flow-Guided-Feature-Aggregation on Python 2.7.13

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./lib/bbox/setup_linux.py:46:48: F821 undefined name 'CUDA'
            self.set_executable('compiler_so', CUDA['nvcc'])
                                               ^

./lib/nms/seq_nms.py:50:26: F821 undefined name 'areas2'
                areas1 = areas2
                         ^

./lib/rpn/rpn.py:72:39: F821 undefined name 'get_pair_image'
    imgs, ref_imgs, eq_flags, roidb = get_pair_image(roidb, cfg)
                                      ^

hi, this is NO aggregation for current frame when training. but when testing, there is aggregation for current frame. how to understand this design? thank you.

Hi,
I'm reading the code to calculate mAP. In the code:
https://github.com/msracver/Flow-Guided-Feature-Aggregation/blob/master/lib/dataset/imagenet_vid_eval.py#L58
When calculating the AUC of precision over recall on one class, it seems like the code enforces the precision to be monotonic by bumping up some precision.

Is this the general method to evaluate mAP?

Hi, thanks for the nice code.
I got a small question when read code and paper. Only one training command mod.fit in train_end2end.py.
Which file should I refer to for the Two Phase Training procedure mentioned in paper? Specifically, the first phase is about training R-FCN with DET data, and the 2nd stage is about training FGFA with VID data.
Or only the 2nd training phase is implemented in this code?
Thanks in advance.

@scharifa: Actually, it is ok when I train the code. But test phase, the error is throw out. I change the KEY_FRAME_INTERVAL in config.py and min_diff and max_diff. Hope it is helpful for you.

Originally posted by @Feywell in #16 (comment)

Can someone share the caffe implementation of a warping layer?
Thank you so much!

Hello guys I'm trying to install FGFA but when I try to clone the MXnet v0.10.0 and execute the "git submodule update" command I get the following error:

Cloning into '/home/ghost/projects/incubator-mxnet/cub'...
error: Server does not allow request for unadvertised object 89de7ab20167909bc2c4f8acd397671c47cf3c0d
Fetched in submodule path 'cub', but it did not contain 89de7ab20167909bc2c4f8acd397671c47cf3c0d. Direct fetching of that commit failed.

I would appreciate some help,
Thanks

i have been following the installation instructions but at the 2nd steo whn i run th init file i get this issue
i would really appericate any help!

[19:25:41] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[19:25:41] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.
[19:25:42] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied.

I meet a trouble when i run demo.py