I'm a Ph.D. Candidate at the Department of Computing, The Hong Kong Polytechnic University.
- High-Level Video Content Analytics
- Visual Knowledge Learning
- Foundation Models
๐ดโโ๏ธ ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection (MM 2020)
Home Page: https://arxiv.org/abs/2008.06254
License: GNU General Public License v3.0
thank you very much.
Does you API visualize BB? How can I visualize these bounding boxes for the image img_name = 'data/hico_20160224_det/images/test2015/HICO_test2015_00007935.jpg' I have 4 bounding boxes for this image
print(anno[anno[:, 0] == img_id])
yields:
tensor([[7935., 133., 566., 128., 594., 190., 533., 159., 628., 217.],
[7935., 133., 349., 108., 425., 233., 233., 134., 484., 354.],
[7935., 139., 563., 126., 602., 207., 519., 154., 635., 260.],
[7935., 139., 334., 98., 421., 241., 222., 127., 484., 350.]])
Hi could you please make this accessible?
Hi,
Thanks for the excellent work. I wonder if you could release the lists of unseen objects and actions you used in the paper or where I could find them.
Thanks!
Thanks for you work! I got some questions:
torch.cat(o_blob,h_blob)
to concatenate?Hi. Thanks for your excellent work!
I wait for long time when I execute wget https://catcatserver.xyz/models/faster_rcnn_r50_fpn_3x_coco-26df6f6b.pth
.
Is that address work? Or could you provide the URL that can be used in China?
Hello, there is a table of results on HICO-DET dataset under zero-shot settings (Table 3 in paper), but no model for it provided. Could you please provide these models? Thanks!
hi, my nncore is 0.2.4, but
File "/dev/shm/L/ConsNet/consnet/models/blocks/fusion.py", line 9, in
from nncore.nn import build_mlp
ImportError: cannot import name 'build_mlp' from 'nncore.nn' (/dev/shm/L/ConsNet/nncore/nncore/nn/init.py)
hoping for your reply~
Hi,in the zero sample setting, do you set all pictures containing unseen hoi to invisible, regardless of whether the picture also contains a seen hoi sample. Where's the corresponding code. Thanks
build = dict(
train=dict(
max_per_img=dict(hum=-1, obj=-1),
score_thr=dict(hum=0.1, obj=0.1),
iou_thr=dict(pos=0.6, neg=0.4),
max_h_as_o=-1,
neg_pos_ub=50),
test=dict(
max_per_img=dict(hum=10, obj=20),
score_thr=dict(hum=0.5, obj=0.1),
iou_thr=dict(pos=0.5, neg=0.5),
max_h_as_o=3))
Hi. In this snippet from configs/build.py
, would you please explain why are different hyperparameter values used in train and test? (for eg. score thr for hum is 0.1 in train and 0.5 in test). What advantage is achieved by doing so?
Also, what does neg_pos_ub
mean in train?
Thanks for you work! I got an error:
(conse) LSX@csm:/disk1/LSX/ConsNet$ python tools/build_dataset.py
/disk1/LSX/ConsNet/mmdetection/mmdet/core/anchor/builder.py:15: UserWarning: build_anchor_generator
would be deprecated soon, please use build_prior_generator
warnings.warn(
/disk1/LSX/ConsNet/mmdetection/mmdet/datasets/api_wrappers/coco_api.py:19: UserWarning: mmpycocotools is deprecated. Please install official pycocotools by "pip install pycocotools"
warnings.warn(
loading annotations into memory...
Done (t=0.82s)
creating index...
index created!
detecting objects in train split
[ ] 0/37633, 0.0 task/s, elapsed: 0s, ETA: 0sTraceback (most recent call last):
File "/disk1/LSX/ConsNet/tools/build_dataset.py", line 288, in
main()
File "/disk1/LSX/ConsNet/tools/build_dataset.py", line 278, in main
blobs[split] = detect_objects(model, data_loader, annos, split)
File "/disk1/LSX/ConsNet/tools/build_dataset.py", line 37, in detect_objects
gt_blobs, dt_blobs = model(return_loss=False, **data)
File "/home/LSX/anaconda3/envs/conse/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/LSX/anaconda3/envs/conse/lib/python3.9/site-packages/mmcv/parallel/data_parallel.py", line 42, in forward
return super().forward(*inputs, **kwargs)
File "/home/LSX/anaconda3/envs/conse/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 165, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/LSX/anaconda3/envs/conse/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/LSX/anaconda3/envs/conse/lib/python3.9/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/disk1/LSX/ConsNet/mmdetection/mmdet/models/detectors/base.py", line 173, in forward
return self.forward_test(img, img_metas, **kwargs)
File "/disk1/LSX/ConsNet/mmdetection/mmdet/models/detectors/base.py", line 146, in forward_test
return self.simple_test(imgs[0], img_metas[0], **kwargs)
File "/disk1/LSX/ConsNet/consnet/models/detectors/hooked_rcnn.py", line 29, in simple_test
gt_blobs, _ = self.roi_head.simple_test(
File "/disk1/LSX/ConsNet/consnet/models/detectors/hooked_roi_head.py", line 20, in simple_test
return self.simple_test_bboxes(
File "/disk1/LSX/ConsNet/mmdetection/mmdet/models/roi_heads/test_mixins.py", line 126, in simple_test_bboxes
det_bbox, det_label = self.bbox_head.get_bboxes(
File "/disk1/LSX/ConsNet/consnet/models/detectors/hooked_bbox_head.py", line 70, in get_bboxes
gt_blob, dt_blob = _hooked_nms(*blob, feat, **cfg)
File "/disk1/LSX/ConsNet/consnet/models/detectors/hooked_bbox_head.py", line 17, in _hooked_nms
scores = scores[:, :-1]
IndexError: too many indices for tensor of dimension 1
Technically, it is not totally unseen scenarios because consistency graph already has both visual and semantic features from training and testing datasets. I don't know if I understand correctly?
thanks a lot for great documentation. got an error in this step
(consnet) mona@goku:~/research/code/ConsNet$ python tools/build_dataset.py --checkpoint checkpoints/
Traceback (most recent call last):
File "tools/build_dataset.py", line 280, in <module>
main()
File "tools/build_dataset.py", line 259, in main
model = build_detector(cfg.model, args.checkpoint)
File "/home/mona/venv/consnet/lib/python3.8/site-packages/consnet/models/builder.py", line 50, in build_detector
load_checkpoint(model, checkpoint, map_location='cpu')
NameError: name 'load_checkpoint' is not defined
Hi. Thanks for the great documentation on the README.
I went through your code for extracting bounding box features through RoIAlign. But, I am still confused. Lets say we already have bounding boxes in our hand which we got from faster-rcnn. Now, I would like to understand the process for a given single bounding box - what is the sequence of layers a given bounding box goes through inorder to arrive at the final 1024 dimensional feature vector for it? Would you please help me with the sequence of layers responsible for feature extraction process?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.