Code Monkey home page Code Monkey logo

vnect-tensorflow's Introduction

VNect -- Tensorflow version

This project is the tensorflow implementation of VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera, SIGGRAPH 2017.

This is not an official implementation. Please contact paper author for related model.

Environments

  • Ubuntu 16.04
  • Python 2.7
  • Tensorflow 1.3.0
  • OpenCV 3.3.0
  • OpenGL (optional)

Inference

  • 1.Download model, put them in folder models/weights
  • 2.Edit demo settings in shell script, --device --demo_type --model_file --test_img --plot_2d --plot_3d
  • 3.If you have OpenGL, you can run run_demo_tf_gl.sh for faster rendering of 3d joints. Otherwise, run run_demo_tf.sh

TODO

  • Some bugs in detected 3D joint locations.
  • Training part of model.

vnect-tensorflow's People

Contributors

cjue avatar timctho avatar wangqr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vnect-tensorflow's Issues

Is the vnect_model has mistake?

Hi @timctho
I had read the vnect_model.py code, and I think maybe a mistake at line 68.
self.res3d = tf.add(self.res3d_branch2c, self.res3b, name='res3d_add')

the add function second param 'self.res3b', maybe is 'self.res3c'?

Where can I download the weights (vnect_tf)?

Hello

Thank you for your implementation.

I am trying to run the demo, but getting errors because of the absence of model_file.

Is there anywhere that I can download the pre-trained weight file?

Thank you

CPU mode gives wrong 3D pose results

python demo_tf.py --device=cpu \
		   --demo_type=image \
		   --test_file=test_imgs/yuniko.jpg \
		   --model_file=models/weights/vnect_tf \
		   --plot_2d=True \
		   --plot_3d=True

figure_1
figure_2

caffe issues

I'm having a really hard time to get caffe running on my system.
Is there a way to not use caffe?
Would it be possible to convert the model to a tensorflow model and save it's weights (through serialisation in tensorflow)?

kinect?

How can I do this with kinect V1? I know it's for kinect alternative

issue using converted vnect.pkl

i converted the caffemodel using the caffe_weights_to_pickle.py.
however, when i want to load the vnect.pkl with tensorflow, i get:
DataLossError (see above for traceback): Unable to open table file models/weights/vnect_tf: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

thanks for any help

I am getting this error . aseOperation = baseOperation, OpenGL.error.GLError: GLError( err = 1282, description = b'invalid operation', baseOperation = glMatrixMode, cArguments = (GL_PROJECTION,) ) ? please help me

Config:

  • Win 10
  • python 3.7
  • Logitech C270 webcam
PS C:\Users\TechRova\Downloads\VNect-tensorflow-master> python demo_tf_gl.py --device=gpu --demo_type=webcam --test_img=
test_imgs/yuniko.jpg --model_file=models/weights/vnect_tf  --plot_2d=True --plot_3d=True
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
  File "demo_tf_gl.py", line 247, in <module>
    glMatrixMode(GL_PROJECTION)
  File "C:\Users\TechRova\AppData\Local\Programs\Python\Python37\lib\site-packages\OpenGL\platform\baseplatform.py", line 409, in __call__
    return self( *args, **named )
  File "C:\Users\TechRova\AppData\Local\Programs\Python\Python37\lib\site-packages\OpenGL\error.py", line 232, in glCheckError
    baseOperation = baseOperation,
OpenGL.error.GLError: GLError(
        err = 1282,
        description = b'invalid operation',
        baseOperation = glMatrixMode,
        cArguments = (GL_PROJECTION,)
)

Issue Webcam heatmaps

Hey,

I've been trying out this project and managed to get the demo working for pictures without any issues, however if I try to run the demo with a webcam instead of a picture I get weird results.

No matter what is the font or if there's a subject in the frame, I get false heatmap (garbage) all over the picture. Did anyone manage to work with a video feed from a camera or is it a known issue? I compared the code of the image and video demo and they are both pretty much identical treatment wise.

I added a picture just to show what I mean by garbage, this is not static it changes at each iteration.

heatmapissue

How can i get the right vnect_tf.~~ files

Hello
I am Junkyung Kim.

I had tried your code.

  1. download weights from the author of paper
  2. change weights with using './caffe_weight_to_pickle.py'
  3. run '/models/vnect_model.py' to get
    vnect_tf.data-00000-of-00001
    vnect_tf.index
    vnect_tf.meta
  4. run '/demo_tf.py'

But I got really bad estimation results with transformed weights from original parameter.
I guess the transformed weights are not exactly same as the original caffe weights.

Can somebody help me..?

about VNect-tensorflow models

hi timctho
i test VNECT-tensorflow , i find Vnect-tf very faster but less accurate . i want to know about your tensorflow model come from . how i do train myself mode .?

About 3D labels

Thanks you for afford TF version project. The 3D label shape formulate : (batch, W, H, nums_points), how to generate this labels?

Real time estimation

It is amazing project! Thanks for sharing!

I want to ask is it good for real time pose detection?

Bounding Box tracker!!!

@timctho As explained in the VNect official paper about Bounding Box tracker and utilizing the previously detected key points to determine the Bounding Box for the new frame. Have you implemented this idea in your code.
I am not able to understand the way you have did it. Could you please explain or refer me any article or any other resource where I could learn more about it..
Thanks in Advance

about ImportError: No module named models.nets

hi timctho
/run_demo_tf.sh but i got an error
File "demo_tf.py" . line 10, in
from models.nets import vnect_model_bn_folded as vnect_model
ImportError: No module named models.nets
i download the tensorflow model abd then save models/weights/vnect_tf
my system:
ubuntu 16.04 64bit gtx1080ti cuda-8.0 python2.7 opencv 3.3.0
i don't know why .pls help me thanks

What's the point of applying different scale values in demo.py?

Hi! I am a bit of confused while reading your admirable work that I can't catch the point of the following part in demo.py:

VNect-tensorflow/demo.py

Lines 32 to 33 in 9aa423e

# input scales
scales = [1.0, 0.7]

The original code meets errors when running under two scale values. And I simply try deleting either of these two values in order to fix those errors, which successfully generates relatively good results, too.
I wonder the idea of applying two kinds of scale values in this work. Could you tell me the motivation and the advantages of this strategy? Thank you very much!

How to properly run ?

I find that the documentation is lacking and haven't really been able to run it.

I've done the following

  • Contact and get the author to send me the model and weights in the caffemodel format
  • Convert it to a pickle format using caffe_weights_to_pickle.py
  • Create an event file running the model file and renamed it vnect_tf and saved it in the weights folder

Yet I can't run any of the demos, I'm getting a dataLoss error, more specifically:

DataLossError (see above for traceback): Unable to open table file models/weights/vnect_tf: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

Is there something I need to do beforehand or did I do something wrong? I'd be willing to update the readme or even create a script to automate the process if I get this running.

Solution for Smartphones

Awesome solution! Do you know the minimum configuration for a pc to work with it? Do you think it would work on smartphones?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.