Pose Estimation Similarity With Tensorflow

This repository consists of all the code required for similar 2-D pose detection in dance videos. This can used for any type of pose estimation application to find the similarity.

We will be using tensorflow for our position estimation using openpose for tensorflow.

Outputs:

For Wrong Dance:

For Right Dance:

Credits:

Ildoo Kim: https://github.com/ildoonet
GitHub Repo Link: https://github.com/ildoonet/tf-pose-estimation

Gunjan Seth: https://github.com/gsethi2409
GitHub Repo Link: https://github.com/gsethi2409/tf-pose-estimation

Ipython Notebook File for Pose Estimation Similarity:

2D-Pose-Similarity.ipynb

Requirements:

Python 3.7
Tensorflow 2.0+
OpenCV

Installation Steps:

1. Git Clone:

Clone this repository into your local machine.
git clone https://github.com/mohit9949/Pose-Estimation-Similarity-With-TensorFlow.git

2. Installing the requirements:

Install all the requirements provided in the requirements.txt
cd Pose-Estimation-Similarity-With-TensorFlow
pip install -r requirements.txt
If any problem with downloading pycocotools follow this link at Step 4: https://github.com/markjay4k/Mask-RCNN-series/blob/master/Mask_RCNN%20Install%20Instructions.ipynb

3. Install SWIG:

conda install swig
or
Download Link: http://www.swig.org/survey.html

4. Build C++ library for post-processing:

cd tf_pose/pafprocess
swig -python -c++ pafprocess.i && python setup.py build_ext --inplace

5. Installing tf_slim:

cd ../../
pip install git+https://github.com/adrianc-a/tf-slim.git@remove_contrib
or
git clone https://github.com/google-research/tf-slim.git
and copy the folder tf_slim in it to our repository Pose-Estimation-Similarity-With-TensorFlow

6. Download Tensorflow Graph File(pb file).

cd models/graph/cmu
bash download.sh
cd ../../..

Conclusion:

We have developed a pose estimation similarity pipeline to compare similarity between two poses from the given feed of videos or live cam.
Flaws:
This approach fails when the trainer is far or the user is near to the camera or vise-versa. This happens because there is a scale variation between the keypoints of the image.
Solution:
We can eleminate this problem by croping out the image of a person using a CNN architecture like Yolo or anything that could detect the bounding boxes of a person.
This image then can be fed to the openpose model to estimate keypoints for both the sources.
Scope of improvement:
The accuracy of the model for keypoint prediction can be increased by taking a much powerful pretrained model architecture than mobilenet.

Keep getting the error "Expected in: flat namespace" when trying to run the Jupyter notebook.

The full error is:

ImportError                               Traceback (most recent call last)
<ipython-input-1-ceb20aa8d14b> in <module>
      6 
      7 import numpy as np
----> 8 from tf_pose import common
      9 from tf_pose.estimator import TfPoseEstimator
     10 from tf_pose.networks import get_graph_path, model_wh

~/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/__init__.py in <module>
      3 from __future__ import print_function
      4 
----> 5 from tf_pose.runner import infer, Estimator, get_estimator

~/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/runner.py in <module>
      6 
      7 from tf_pose import common
----> 8 from tf_pose import eval
      9 from tf_pose.estimator import TfPoseEstimator
     10 from tf_pose.networks import get_graph_path, model_wh

~/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/eval.py in <module>
     11 
     12 from tf_pose.common import read_imgfile
---> 13 from tf_pose.estimator import TfPoseEstimator
     14 from tf_pose.networks import model_wh, get_graph_path
     15 

~/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/estimator.py in <module>
     19 #from tensorflow.python.compiler.tensorrt import trt
     20 try:
---> 21     from tf_pose.pafprocess import pafprocess
     22 except ModuleNotFoundError as e:
     23     print(e)

~/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/pafprocess/pafprocess.py in <module>
     11 # Import the low-level C/C++ module
     12 if __package__ or "." in __name__:
---> 13     from . import _pafprocess
     14 else:
     15     import _pafprocess

ImportError: dlopen(/Users/username/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/pafprocess/_pafprocess.so, 2): Symbol not found: _PyCObject_Type
  Referenced from: /Users/username/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/pafprocess/_pafprocess.so
  Expected in: flat namespace
 in /Users/username/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/pafprocess/_pafprocess.so

I followed the installation instructions as written in the Readme. I don't get any error when doing:

swig -python -c++ pafprocess.i && python setup.py build_ext --inplace
I get only 2 warnings generated:

cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I. -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pafprocess_wrap.cpp -o build/temp.macosx-11.2-x86_64-2.7/pafprocess_wrap.o
c++ -bundle -undefined dynamic_lookup -Wl,-F. -L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.2.Internal.sdk/usr/local/libressl/lib build/temp.macosx-11.2-x86_64-2.7/pafprocess.o build/temp.macosx-11.2-x86_64-2.7/pafprocess_wrap.o -o /Users/prasantasaikia/Downloads/Pose-Estimation-Similarity-With-TensorFlow/tf_pose/pafprocess/_pafprocess.so
ld: warning: directory not found for option '-L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.2.Internal.sdk/usr/local/libressl/lib'

I thought maybe this happens because the swig command says python, and not python3, which is what I am using on my MacBook to run OpenPose. So, I tried replacing the python in the swig command with python3, but I get the following error then: swig error : Unrecognized option -python3.

I've also uninstalled and reinstalled the whole thing multiple times. But this error keeps occurring. How do I fix it?

mohit9949 / pose-estimation-similarity-with-tensorflow Goto Github PK