stevenlsw / semi-hand-object Goto Github PK

Hello! It's a great project. I am wondering if you could provide the code or some instructions for preprocessing. So that I could use it in other tasks? Thanks in advance!

Thanks for your great work!
I want to know how to draw joint figures in your paper.

When I ran your code, I find the 2d joints result (your model output "pred_joints") is below/

It seems like it's not in the pixel frame. I don't know how to visualize it (like joint figures in your paper).
If it's not convenient to show me your code, please just tell me the procedure to do it!
Thanks a lot!

Hi! I was wondering if the pseudo labeled dataset (including the selected frames and labels from the something-something dataset) are available for training. I assume that the model trained by running traineval.py and following the steps in the readme gives the baseline supervised model? If so, is the combined dataset with pseudo labels available for download? Also does the current loss.py implement the binary mask for object pose loss?

Thanks!

Thank you for your excellent work!
There some diffculties that I can't get the same or even closed results provided by your paper in Table 4(supervised with CR).And I find the hyper-parameters in the code is not configed as your paper.Do your mand providing your hyper-parameters to help me ? Thank you.
Here my logs.

====== Options ======
HO3D_root: /datassd/huanyao/semi-hand/Semi-Hand-Object-master/assets/data/ho3d_v2
blocks: 1
channels: 256
epochs: 60
evaluate: False
host_folder: /datassd/huanyao/semi-hand/Semi-Hand-Object-master/host_folder
inp_res: 512
lambda_joints2d: 100.0
lambda_objects: 500.0
lr: 0.0001
lr_decay_gamma: 0.7
lr_decay_step: 10
mano_lambda_joints3d: 10000.0
mano_lambda_manopose: 10
mano_lambda_manoshape: 0.1
mano_lambda_regulpose: 1
mano_lambda_regulshape: 100.0
mano_lambda_verts3d: 10000.0
mano_neurons: [1024, 512]
mano_root: assets/mano_models
manual_seed: 0
momentum: 0.9
network: honet_transformer
obj_model_root: assets/object_models
resume: /datassd/huanyao/semi-hand/Semi-Hand-Object-master/model.pth
save_results: True
snapshot: 5
stacks: 1
test_batch: 16
test_freq: 10
train_batch: 24
transformer_depth: 1
transformer_head: 1
use_cuda: 1
weight_decay: 0.0005
workers: 16

Hi! Thank you for the amazing work.
I'm trying to train the model, but the provided download link for the preprocessed files gives me an access denied error.
Would it be possible to check this?

Hi,
I am trying to perform the evaluation on SSV2 (sth-sth) dataset, I've successfully evaluated the trained model on HO3D, however, I am working on a small project for a project at the university and would like to run it on SSV2.
Would you be able to point me in the right direction - what part of the dataloader should be changed to be able work with SSV2 and how can I do that?
Amazing work.

Hi, your work is inspiring me a lot.

I found that you are using different a pointcloud from the original data for YCB objects. How did you get it?

Did you random sample from the original point cloud?

Hi @stevenlsw ,

I've been trying to evaluate the trained model, but got result shown as below:

 Traceback (most recent call last):

    File "traineval.py", line 104, in <module>
        main(args)
    File "traineval.py", line 35, in main
        train_dat = get_dataset(args, mode="train")
    File "/home/yilin/Hand_est/Semi-Hand-Object/utils/utils.py", line 133, in get_dataset
        train_label_root="ho3d-process", mode=mode, inp_res=args.inp_res)
    File "/home/yilin/Hand_est/Semi-Hand-Object/dataset/ho3d.py", line 55, in __init__
        self.set_list = ho3d_util.load_names(os.path.join(train_label_root, "train.txt"))
    File "/home/yilin/Hand_est/Semi-Hand-Object/dataset/ho3d_util.py", line 14, in load_names
        with open(image_path) as f:
    FileNotFoundError: [Errno 2] No such file or directory: 'ho3d-process/train.txt'

And by tracing back to ho3d.py, it seems that train.txt in the HO3D dataset some preprocessed files train_K.json, train_joint.json, train_mano.json and train_obj.json are supposed to be loaded to the ho3d-process path. Am I missing some preprocessing process? Also, is it ok to use HO3D_v3 for training and testing?

Thank you for the amazing work! I wanted to ask if there was an update on the demo script of the model? Thanks

Thank you for your work. I have used your code for training on RTX 3090, and the parameters have not changed. However, I cannot obtain experimental results similar to those in the paper(hand), even if I have trained more epochs.
here is my results:

If you could provide some details of the training, I would greatly appreciate it.

Hi @stevenlsw

Great work!!! I have some trouble using your code. The result of the 2D joint prediction is weird. You can see the image and simple demo code on https://github.com/JiaheXu/Semi-Hand-Object test.jpg is the code I used for testing, hand_detect_result.png is the result of hand_detection. 2d_joints.jpg is the result of the 2D joint prediction. you can run the code with python3 demo.py --evaluate --HO3D_root=../dataset/HO3D_v2 --resume=./model.pth.tar --test_batch=1 --host_folder=exp_results .Please take a look, thanks!!!!

red dots are the joint preditions

When I run the code as written in the readme.md, the result is different from trained model.
Why is it different and how to get trained model?

How do you get full segmentation from the dataset when the occluded part is included in the figure in the data pre-process?

thanks in advance.