Comments (13)
Please ensure you used the config of kpt point number in here
from gsnet.
When you run bash run.sh, has this Line752-754 been executed?
from gsnet.
from gsnet.
I have figure out that this problem is caused by the roi_head.py you provide is not used actually in my detectron2. but when i use the roi_head.py your provide in my environment(cuda11.1+pytorch1.8), i meet a lot of problem with the version of detecrton2.
i have solve most of the problem, but one:
i change the line 640: selected_keypoint_logits_heatmap = self.keypoint_head(keypoint_features)
to :selected_keypoint_logits_heatmap = self.keypoint_head(keypoint_features, pred_instances)
because missing argument.
but the output of 'self.keypoint_head' is not the same, my 'selected_keypoint_logits_heatmap' is a list, it can not put into:
(line 642)selected_keypoint_res = keypoint_rcnn_inference(selected_keypoint_logits_heatmap, pred_instances);
i find in my 'selected_keypoint_logits_heatmap' has a key named pred_keypoint_heatmaps
so i change (line 642)selected_keypoint_res = keypoint_rcnn_inference(selected_keypoint_logits_heatmap, pred_instances);
to:
selected_keypoint_res = keypoint_rcnn_inference(selected_keypoint_logits_heatmap[0].pred_keypoint_heatmaps, pred_instances);
but the selected_keypoint_logits_heatmap[0].pred_keypoint_heatmaps is shape of [9,17,56,56], i think the correct shape is [9.66,56,56]
can you print the shape of selected_keypoint_logits_heatmap in your environment, or do you have any idea about this problem.
thanks a lot
from gsnet.
Hi, please use the keypoint head in this file, which has only keypoint_features passed in.
from gsnet.
Thanks for you reply, i have use the file you provide then i can use:
(line 640)selected_keypoint_logits_heatmap = self.keypoint_head(keypoint_features)
to get tensor, but the shape of selected_keypoint_logits_heatmap is [9,17,56,56]
but i have successfully install your project on my cuda 10 environment the shape of selected_keypoint_logits_heatmap is [9,66,56,56]
i have check the shape of keypoint_features in both environments , they are[9,256,14,14]
why the input is same and function is same, but i got different output
from gsnet.
this is the warning information when run:
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (316, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (4,) in the checkpoint but (316,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.keypoint_head.score_lowres.weight' to the model due to incompatible shapes: (512, 66, 4, 4) in the checkpoint but (512, 17, 4, 4) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.keypoint_head.score_lowres.bias' to the model due to incompatible shapes: (66,) in the checkpoint but (17,) in the model! You might want to double check if this is expected.
Some model parameters or buffers are not found in the checkpoint:
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.keypoint_head.score_lowres.{bias, weight}
The checkpoint state_dict contains keys that are not used by the model:
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
there is also the value '17' , does it mean i use the incorrect mode, but how can i change the model?
from gsnet.
Thanks for you reply, i have use the file you provide then i can use:
(line 640)selected_keypoint_logits_heatmap = self.keypoint_head(keypoint_features)
to get tensor, but the shape of selected_keypoint_logits_heatmap is [9,17,56,56]but i have successfully install your project on my cuda 10 environment the shape of selected_keypoint_logits_heatmap is [9,66,56,56]
i have check the shape of keypoint_features in both environments , they are[9,256,14,14]
why the input is same and function is same, but i got different output
I think this 66 or 17 difference may due to config file setting.
from gsnet.
thank you very much! i have solve this problem successfully!
i meet the very similar problem then:
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (316, 1024) in the model! You might want to double check if this is expected.
it is the problem about 4 and 316, i cant find this in the default file, can you help me fine how to fix the config file
from gsnet.
316/4 = 79. I think the 79 may be the number of we provided car meshes.
from gsnet.
Thank you for your guidance and patience!
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (316, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (4,) in the checkpoint but (316,) in the model! You might want to double check if this is expected.
Some model parameters or buffers are not found in the checkpoint:
roi_heads.box_predictor.bbox_pred.{bias, weight}
The checkpoint state_dict contains keys that are not used by the model:
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
0%| | 0/26 [00:00<?, ?it/s]['input/']
keypoint_features_forward_box torch.Size([27, 256, 14, 14])
torch.Size([27, 66, 56, 56])
0%| | 0/26 [00:19<?, ?it/s]
Traceback (most recent call last):
File "demo2/demo.py", line 90, in
predictions, visualized_output = demo.run_on_image(img, save_name)
File "D:\ApolloCar3D\gsnet-master\reference_code\GSNet-release\demo2\predictor.py", line 54, in run_on_image
predictions = self.predictor(image)
File "d:\apollocar3d\detectron2-master\detectron2\engine\defaults.py", line 318, in call
predictions = self.model([inputs])[0]
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "d:\apollocar3d\detectron2-master\detectron2\modeling\meta_arch\rcnn.py", line 146, in forward
return self.inference(batched_inputs)
File "d:\apollocar3d\detectron2-master\detectron2\modeling\meta_arch\rcnn.py", line 209, in inference
results, _ = self.roi_heads(images, features, proposals, None)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "d:\apollocar3d\detectron2-master\detectron2\modeling\roi_heads\roi_heads.py", line 568, in forward
pred_instances = self._forward_3d_pose_inference(selected_roi_features, selected_boxes, selected_kpt_pos, selected_heatmap, pred_instances)
File "d:\apollocar3d\detectron2-master\detectron2\modeling\roi_heads\roi_heads.py", line 737, in _forward_3d_pose_inference
new_box_pos = self.relu(self.fuse_box_pos(new_box_pos))
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\nn\modules\linear.py", line 94, in forward
return F.linear(input, self.weight, self.bias)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\nn\functional.py", line 1753, in linear
return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 dim 1 must match mat2 dim 0
This is the whole error, I think is the same as the 66 or 17 difference, but I don't have any idea about it. I have check the code line you point out, i'm sure i use the 79 in here.
from gsnet.
For i successfully build your work on my 2080 environment,I also have another question. For the generated results, how to visualize them into a 3D mesh. Do you modify the render_car_instance [here]([https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance] to achieve it.
THANK YOU VERY MUCH. I've just started my PhD. I think your work is excellent and I want to follow your work. Thanks again!
from gsnet.
yes, you could use render_car_instance.py to help visualize the projections.
from gsnet.
Related Issues (19)
- Is there any trained model for evaluation or any training script available? HOT 1
- About 3D semantic keypoints on the predicted mesh HOT 3
- pytorch version for detectron2 under gsnet/reference_code/GSNet-release HOT 7
- Geometric consistency loss HOT 1
- Abrut kill happening when trying to run the demo.py HOT 3
- Demo failed because of FCOS HOT 1
- No code about loss function HOT 1
- how to use this project only as keypoint detector HOT 1
- kpts_mapping HOT 6
- AttributeError: Cannot find field 'predict_trans' in the given Instances! HOT 1
- Many warnings while loading the pretrained model, and no predictions
- Training codes HOT 1
- About kpts_mapping.npy
- the pretrained model has been removed HOT 5
- 66 3D vertices HOT 1
- Missing attributes HOT 7
- CUDA11.1+Torch 1.8 HOT 1
- Using SoftRas to generate deformed car models HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gsnet.