lyuj1998 / handtailor Goto Github PK

Hi, I try to estimate the 3D hand in the camera coordinate. I use demo.py with known intrinsic, the render result seems to be OK, but the 3D hand is not aligned with the GT.

请问能提供一下训练的参考代码吗？非常感谢

Hey, this is very awesome work. Will there any training code be available soon?

I'm trying to understand where you got common.DEPTH_RANGE and common. DEPTH_MIN that are used here. In your paper you specify that you scale normalized the predicted root depth when using the perspective projection. Are these values what are used to unnormalized the predicted root depth? I may have incorrectly thought that normalization had to do with the length from the root to the MCP joint (or whatever the chosen joint was), is that not the case when predicting root depth

As you have mentioned in the Hand Scale Optimization part.

你好，请问可以用您论文中去除 IkNet 的Hand Module去做三维姿态的估计吗，我尝试使用您论文中提到 公式（6）损失去训练网络，用热力图监督二维坐标，用21点3D pose监督uvd2xyz后的3D坐标，用根关节点的值监督root，最终效果不好，请问我这样的作法对吗？

Is it possible to run app_in_the_wild.py without using a Real Sense camera? As in is it possible to simply connect the webcam of the laptop?

hi nice work！i think the neural network output predicts then using the 3D model to correct , can it be understood as a post-processing operation?

我想请问为什么我在运行demo.py 之后没有任何反应呢

I believe this is a typo

it should be

s = torch.sqrt((deltaj[0,0]**2 + deltaj[0,1]**2)/(1 - deltaj[0,2]**2))

Hello, how to solve the installation problem of jaxlib on Windows?

Great work you have done with this research @LyuJ1998 . I just have a question on the joint's location converted here from uvd to xyz;

• What is the index for each joint in the converted locations?

Hi, I was wondering that why we use align_idx=9, instead of root_idx 0. Is there any reason? If we will align by root before computing loss anyway, why don't we align by root in the beginning?

as the titled said. Thanks !

Hi, When I use python demo.py, this error came up

ImportError: cannot import name 'cusolver' from 'jaxlib'

@lixiny I got pretty weird results, when used logitech c270 camera with it's intrisics.

I used this command,
python demo.py --fx 1078.03779 --fy 1084.50988 --cx 580.850545 --cy 245.959325

I have checked the camera matrix multiple times, and it seems okay.
Output Video link: https://drive.google.com/file/d/10oH3cL92bcvFLr1HTEfp5QOTn2WvBpVM/view?usp=sharing

Hiya, I had a few questions about the tailor module

Just as a high-level overview (just to make sure I correctly understand the approach). It seems that you want to refine the predicted pose parameters from the inverse kinematics network with the insight that the 2d keypoints will generally have higher accuracy/ reliability allowing for not only a more accurate prediction but also helping with bad initializations and occlusion problems in typical optimization-based techniques face (along with fewer iterations due to a good initial condition)

1. Did you experiment with different optimizers in jax other than Adam? Such as the Levenberg-Marquat solver (not sure if this is implemented in jax) used in papers like MegaTrack/RGB2Hands/Online Optical Marker-based Hand Tracking with Deep Labels
2. For the in-the-wild examples, I've noticed certain poses lead to failure cases for estimating the scale camera parameter, is this something you've noticed as well? usually when the hand is horizontal rather than vertical

I enjoyed reading your paper greatly! Thank you so much for the contribution and I look forward to your response