Hello,

I'm using ubuntu 16.04, gcc 5.4, and libboost1.58-dev:amd64. I followed this link to modify the Makefile in custom_ops folder. And I got the undefined symbol error as below. I don't know if my boost version is not correct? Or there could be some other reasons? I have confirmed that my boost libraries are installed in "/usr/lib/x86_64-linux-gnu". And boost header files are in "/usr/include/boost" (of course in my Makefile, BOOST_INC_DIR := $/usr/include, as in the aforementioned issue post). Thanks!

Traceback (most recent call last):
File "../train.py", line 16, in
from datasets import dataset_factory
File "/home/local/research/DL/AttentionalPoolingAction/src/datasets/dataset_factory.py", line 8, in
from datasets import hmdb51
File "/home/local/research/DL/AttentionalPoolingAction/src/datasets/hmdb51.py", line 8, in
from datasets.video_data_utils import gen_dataset
File "/home/local/research/DL/AttentionalPoolingAction/src/datasets/video_data_utils.py", line 18, in
from custom_ops.custom_ops_factory import read_json_pose, read_sparse_label
File "/home/local/research/DL/AttentionalPoolingAction/src/custom_ops/custom_ops_factory.py", line 16, in
os.path.join(ROOT_PATH, 'render_pose.so')).render_pose
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/local/research/DL/AttentionalPoolingAction/src/custom_ops/render_pose.so: undefined symbol: _ZN5boost13thread_detail20rollback_once_regionERNS_9once_flagE

The error displayed when I tried to evaluate the model:
tensorflow.python.framework.errors_impl.notfounderror: libopencv_ocl.so.2.4: cannot open shared object file: No such file or directory.

Does it mean I have to create the libopencv_ocl.so.2.4 file or need to modify some configuration file?

~/AttentionalPoolingAction-master/src$ python eval.py --cfg ../experiments/003_MPII_ResNet_withPoseAttention_pretrained.yaml
Traceback (most recent call last):
File "eval.py", line 28, in
from preprocess_pipeline import get_input
File "/home/cyf/AttentionalPoolingAction-master/src/preprocess_pipeline.py", line 2, in
from custom_ops.custom_ops_factory import pose_to_heatmap, render_pose,
File "/home/cyf/AttentionalPoolingAction-master/src/custom_ops/custom_ops_factory.py", line 12, in
os.path.join(ROOT_PATH, 'pose_to_heatmap.so')).pose_to_heatmap
File "/home/cyf/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/cyf/AttentionalPoolingAction-master/src/custom_ops/pose_to_heatmap.so: undefined symbol: _ZN2cv3Mat10deallocateEv

After compiling some custom_ops, I got this error when i try to eval.
What should i do?
Is there anything i should edify in src/cumtom_ops/Makefile?
Thanks a lot

1. It seems like dropout layer before the topdown attention logits has missed the is_training argument (line 283 in nets_factory.py). Kindly check this

2. In vgg.py, the vgg16 function (line 166-170) uses the relued output of conv5_2 (not pre-relu conv5_3) as the final convolutional endpoint. And the relu activation of conv5_3 is missing from the architecture. Since we are loading the imagenet weights, the architecture has to be consistent. Please share your thoughts.

Hi I'm trying to reproduce the the performance on HICO dataset. Using ResNet only you reported the performance of 30.2 mAP. But I only got 26.3. Could you please share the parameters you use?
These parameters are:

1. How the image is pre-processed for training/validating
2. Are there any other information used (pose?text?)
3. Is the loss function softmax cross entropy or sigmod binary cross entropy for multi-class?
4. what's the optimizer? (adam or SGD?)
5. what's the training scheduler? (learning rate change, freezing any layers?)

Thanks a lot!

So, the algorithm seems to replace the average pooling with a weighted pooling using bottom up saliency as the weight?

How do I infer in a realtime video?

When I was going to run python eval.py --cfg ../experiments/001_MPII_ResNet_pretrained.yaml and I found that this file was not exist. So where can I find this .so file? Thank you !
‘’‘
Traceback (most recent call last):
File "eval.py", line 20, in
from datasets import dataset_factory
File "/home/gzx/AttentionalPoolingAction/src/datasets/dataset_factory.py", line 8, in
from datasets import hmdb51
File "/home/gzx/AttentionalPoolingAction/src/datasets/hmdb51.py", line 8, in
from datasets.video_data_utils import gen_dataset
File "/home/gzx/AttentionalPoolingAction/src/datasets/video_data_utils.py", line 18, in
from custom_ops.custom_ops_factory import read_json_pose, read_sparse_label
File "/home/gzx/AttentionalPoolingAction/src/custom_ops/custom_ops_factory.py", line 12, in
os.path.join(ROOT_PATH, 'pose_to_heatmap.so')).pose_to_heatmap
File "/home/gzx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
File "/home/gzx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: /home/gzx/AttentionalPoolingAction/src/custom_ops/pose_to_heatmap.so: cannot open shared object file: No such file or directory‘
’’‘’’

Hi @rohitgirdhar ,
Thanks for your great work! It's really helpful. Can you give your pose data and feature data of HICO dataset and HMDB51 dataset? Thank you in advance. 👍

Thanks a lot for the paper and sharing the code.
It seems that for CIFAR10 dataset, the result is very similar with and without attentional pooling.

With: 92.87%
Without: 92.86%

ResNet 32 (residual blocks: 5-5-5, output_channels: 16-32-64 , number of parameters: 0.46 ) is used for this test.

Is this attentional pooling dependent on special datasets?

@rohitgirdhar Hi, can you give some advice about how to get top-down map or bottom-up map introduced in your paper? And is there a file in this repository I can use to get top-down map or bottom-up map ?
Thanks a lot!

dir of annotation and image files are set correctly, but when I run "python gen_tfrecord_mpii.py", the following error occured:

Traceback (most recent call last):
File "gen_tfrecord_mpii.py", line 259, in
main()
File "gen_tfrecord_mpii.py", line 182, in main
with open(_SPLITS_PATH.format(spl), 'r') as fin:
IOError: [Errno 2] No such file or directory: '../../src/data/mpii/lists/'

The link of rec file is also incorrect

Hi,

I am trying to run the evaluation script. However, I got the following error message.

undefined symbol: _ZN2cv8fastFreeEPv

Could you please let me know how to solve this issue?

Thanks.

When evaluating

python eval.py --cfg ../experiments/001_MPII_ResNet_pretrained.yaml

It outputs

INFO:tensorflow:Restoring parameters from pretrained_models/mpii_baseline/model.ckpt-12000
  0%|                                               | 0/6988 [00:00<?, ?it/s]Exception KeyError: KeyError(<weakref at 0x7ff7ad282f70; to 'tqdm' at 0x7ff7ad2e1f10>,) in <bound method tqdm.__del__ of   0%|                                               | 0/6988 [00:00<?, ?it/s]> ignored
Traceback (most recent call last):
  File "eval.py", line 335, in <module>
    main()
  File "eval.py", line 267, in main
    all_labels = np.concatenate(all_labels)
ValueError: need at least one array to concatenate

Something is wrong with

Hi, I read your paper Attentional Pooling for Action Recognition and feels it great for my network pooling (a C3D network for video recognition). However, in your code I did not find clear clues about a_k, b, x_a, x_b and the corresponding pooling module in the paper. All I can see is about "POSE_ATTENTION_LOGITS".

Can you give me a more brief instruction? so that I can use your attention pooling module to pool a [bsz, 128, 16, 32, 32] feature into [bsz, 128, 1, 32, 32]

Thanks a lot for sharing the code.

Will attentional pooling do better on UCF101?
It seems that the performance is almost the same without using pre-trained model. Both are around 40%.

Any suggestions?

Thanks a lot!

Hello,

I'm trying to compile the custom_ops. I got the following error at the "make" step. I'm using the Ubuntu 16.04.

/usr/bin/ld: cannot find -lIlmImf
/usr/bin/ld: cannot find -lextra_component-NOTFOUND
collect2: error: ld returned 1 exit status
Makefile:10: recipe for target 'pose_to_heatmap.so' failed
make: *** [pose_to_heatmap.so] Error 1

I found that ilmimf is a part of the openexr library (http://www.openexr.com/). But it looks like installing the openexr cannot resolve the problem.

Thank you!

Thanks a lot for the paper and sharing the code.

In your paper, you have mentioned that HMDB51 in your work. Could you please release the code of this part!

Regards

Hi @rohitgirdhar ,
Thanks for you nice work
When I ran

 python train.py --cfg ../experiments/002_MPII_ResNet_withAttention.yaml

and then

python eval.py --cfg ../experiments/002_MPII_ResNet_withAttention.yaml --gpu 0,1,2,3

using default parameters in the yaml file, I got a result of Mean AP: 0.277, Accuracy: 0.335, which is lower than results reported in paper (30.3,37.2). Then I run Attention + Pose in this way:

 python train.py --cfg ../experiments/003_MPII_ResNet_withPoseAttention.yaml

and then

python eval.py --cfg ../experiments/003_MPII_ResNet_withPoseAttention.yaml --gpu 0,1,2,3

and I got results Mean AP: 0.292，Accuracy: 0.367, also lower than expected results 30.6, 37.8.
I am using a machine with Ubuntu 16.04 and 4 1080Ti GPUs.
Could you please tell me if anything is wrong in the running procedures? Thanks sincerely.

Sorry, nevermind. Completely unrelated issue.

@rohitgirdhar Thanks for your excellent code! I tested the trained models on the validation set, but the result of attentional model is: Mean AP: 0.0408322168145 Accuracy: 0.0651116199199; result of the pose attentional model is: Mean AP: 0.0203709144522 Accuracy: 0.0435031482541. These two models are got after 12000 training steps without pretrained models. For attentional model, the loss decreased from 32.7 to 27.2, and for pose attentional model, the loss decreased from 32.7 to 30.2. Can you share the final loss of these two models? Maybe I need train more steps to decrease the loss.

Hello rohit,
Thanks for your algorithm.
I try to use your pretrained model for my video.
Here is my Makefile:

No compilation error, but be blocked for evaluation.
I run " python eval.py --cfg ../experiments/002_MPII_ResNet_withAttention_pretrained.yaml"
And got the error:

OS: Ubuntu-18
Python: 3.6
CUDA: 9.0
Tensorflow: 1.12
Sorry for disturbance.
Thanks for help!
Best regards!

In nets_factory.py

sys.path.append('libs/tensorflow_compact_bilinear_pooling/')
from compact_bilinear_pooling import compact_bilinear_pooling_layer

But there's no libs folder in the repo. Can you provide some info?

Thanks.

Hi, when I try to run python train.py --cfg ../experiments/002_MPII_ResNet_withAttention_train+val.yaml, it can not find the file. Can you please provide it? Thanks!

BTW, do we need compact_bilinear_pooling in this project? Because this file does not exist either.

I've installed the version of tensorflow which is mentioned. When execute the Makefile I'm running into this error.

tensorflow-1.1.0-rc2/tensorflow/core/framework/op.h: No such file or directory
#include "tensorflow-1.1.0-rc2/tensorflow/core/framework/op.h"

Hello, can you write an Attentional Pooling version that caffe can use?

When I try to run the python3 train.py --cfg ../experiments/002_MPII_ResNet_withAttention.yaml,it cannot find the file resnet_v1_101.ckpt.Can you please provide it? Thanks!

Thanks a lot for sharing the code!

When run $python eval.py --cfg ../experiments/001_MPII_ResNet_pretrained.yaml

It has the following error. Any suggestions?
Thanks a lot!

Traceback (most recent call last):
File "eval.py", line 21, in
from datasets import dataset_factory
File "/media/lci/storage/AttentionalPoolingAction/src/datasets/dataset_factory.py", line 8, in
from datasets import hmdb51
File "/media/lci/storage/AttentionalPoolingAction/src/datasets/hmdb51.py", line 8, in
from datasets.video_data_utils import gen_dataset
File "/media/lci/storage/AttentionalPoolingAction/src/datasets/video_data_utils.py", line 18, in
from custom_ops.custom_ops_factory import read_json_pose, read_sparse_label
File "/media/lci/storage/AttentionalPoolingAction/src/custom_ops/custom_ops_factory.py", line 12, in
os.path.join(ROOT_PATH, 'pose_to_heatmap.so')).pose_to_heatmap
File "/home/lci/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /media/lci/storage/AttentionalPoolingAction/src/custom_ops/pose_to_heatmap.so: undefined symbol: _ZN2cv3Mat10deallocateEv

I am getting the majority of the images classified as two lables only. The evaluation accuracy is 36% as mentioned and I am using my latest checkpoint only for classification. Is someone else also facing the same/similar issue?
And can anyone help with it?

Thanks.

When make in folder "custom_ops", I got error:

/usr/bin/ld: cannot find -lboost_system
/usr/bin/ld: cannot find -lboost_filesystem
/usr/bin/ld: cannot find -lboost_thread
collect2: error: ld returned 1 exit status
Makefile:11: recipe for target 'pose_to_heatmap.so' failed
make: *** [pose_to_heatmap.so] Error 1

I think maybe it's because "BOOST_DIR := /home/rgirdhar/Software/basic/boost/install2/" in Makefile file. How should I change the dir?

Hi,

Could you tell me which file has the attention pooling layer? Is it integrated with the nets files?