stevenlsw / hoi-forecast Goto Github PK
View Code? Open in Web Editor NEW[CVPR 2022] Joint hand motion and interaction hotspots prediction from egocentric videos
License: MIT License
[CVPR 2022] Joint hand motion and interaction hotspots prediction from egocentric videos
License: MIT License
I believe there is a small bug in the code at this line
The function should return outside the inner if conditions.
Hi @stevenlsw ,
Thanks for your fantastic work again! Could you tell me where is the ground-truth annotation file (containing such as 'norm_contacts') in EGTEA Gaze+ dataset similar to the ek100_eval_labels.pkl
in EK-100 dataset?
Looking forward to your reply~
Thank you for sharing this great work!
According to the REAME instruction, it seems that the link to the pretrained models is invalid/missing here. Are you planning to release them in the near future?
Hi Shaowei,
Many thanks for sharing this great work! I noticed there is no LICENSE under this repo. I would like to use some pieces of your code, primarily the basic Transformer components from networks/embedding.py, networks/layer.py, networks/net_utils.py, and networks/transformer.py.
To respect and protect your copyright, I'm wondering if these codes are developed by yourself or referred to some licensed open-source softwares (e.g., PyTorch, huggingface, etc). If developed by yourself, would you mind providing a license (MIT, BSD 3.0, or Apache 2.0)? If modified from other sources, could you tell where are they originally from?
Thank you!
Hi,
Thanks for sharing your source code.
I am getting "EOFError: Ran out of input" error while evaluating future hand trajectory or future interaction hotspots. I have followed the steps on the README. All of the error trace is following:
Using 1 GPUs !
=> loading checkpoint 'model.pth.tar'
Loaded checkpoint from epoch 35, starting from there
Loading validation samples: 100%|██████████████████████████████████████████████████████████████████████████████████| 3513/3513 [00:00<00:00, 8801.58it/s]
Loading eval samples: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 401/401 [00:00<00:00, 5768.16it/s]
C:\Users\mesad\miniconda3\lib\site-packages\torch\utils\data\dataloader.py:563: UserWarning: This DataLoader will create 16 worker processes in total. Our suggested max number of worker in current system is 12 (`cpuset` is not taken into account), which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
evaluation dataset size: 401
evaluate epoch 35
Traceback (most recent call last):
File "C:\Users\mesad\projects\bitirme\hoi-forecast\traineval.py", line 125, in <module>
main(args)
File "C:\Users\mesad\projects\bitirme\hoi-forecast\traineval.py", line 76, in main
epoch_pass(Exception ignored in: <function lmdbdict.__del__ at 0x000002084936B0D0>
File "C:\Users\mesad\projects\bitirme\hoi-forecast\netscripts\epoch_feat.py", line 31, in epoch_pass
Traceback (most recent call last):
File "C:\Users\mesad\miniconda3\lib\site-packages\lmdbdict\lmdbdict.py", line 184, in __del__
for batch_idx, sample in enumerate(loader):
File "C:\Users\mesad\miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 444, in __iter__
if self.mode == 'w':
AttributeError return self._get_iterator():
File "C:\Users\mesad\miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 390, in _get_iterator
'lmdbdict' object has no attribute 'mode'
Traceback (most recent call last):
File "<string>", line 1, in <module>
return _MultiProcessingDataLoaderIter(self)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
File "C:\Users\mesad\miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 1077, in __init__
exitcode = _main(fd, parent_sentinel)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\spawn.py", line 126, in _main
w.start()
File "C:\Users\mesad\miniconda3\lib\multiprocessing\process.py", line 121, in start
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
self._popen = self._Popen(self)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\mesad\miniconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'get_features_loader.<locals>.<lambda>'```
Hey Steven,
When trying to access the google drive file under Evaluation on EK100 i dont have permission to access these files.
Are the files restricted for some reason?
Hi,Steven. I meet the problem when trying the train mode. I extract the features like the RULSTM, but I only get the data.mdb and lock.mdb. I found the data.lmdb in the eval mode, and the value include global features, hand features ,hand bbox. It seems that the method in LSTM cannot provide us the correct lmdb for training. Could you please tell me how can I get the whole LMDB including hand&object features? Also, I wonder how can I make .lmdb type document(not .mdb).
Thanks!
Hi,
I have a custom dataset that contains rgb frames of an egocentric video like EPIC KITCHEN. What steps do I need to take in order to predict hand trajectories?
Thanks.
Hi Steven,
Thank you so much for sharing the implementation of this interesting paper.
If it's not too much trouble and if you still have the lmdb file of EGTEA Gaze+ Dataset on your local system, could you consider sharing it? This will immensely aid my progress and save me a big amount of time.
Thank you in advance.
Hello @stevenlsw ,
Thanks for sharing your nice work! Could you please post your device setup for training and evaluation (e.g. what GPU and how many) ?
Hi Steven
Thanks for sharing this interesting work. In the paper, you have extracted hand and object bounding boxes for the datasets.
Is it possible to share it for EGTEA dataset as it is not publicly available?
Thanks and Regards
Debaditya
Hi, I found that you only uploaded the ek100_eval_labels.pkl, could you please share the ek55_eval_labels.pkl?
Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.