Comments (11)
Hi Cataldo, the lib folder is for the tensorflow implementation of the pose estimators (e.g., Mobile-Deconv, CPN, MSRA-Deconv). If you replace pose estimator with OpenPose, I believe you could just ignore this folder and not compile it.
from lighttrack.
Thank you for your reply, Guanghan.
I am trying to replace the pose estimator and the detector (it is embedded in the pose estimator) but I can't figure out the structure of keypoints returned by inference_keypoints(...)
in demo_camera_mobile.py
. Being not able to run the code, since my system does not support CUDA, is not helping :
Right now my module outputs an array like this: [x_1, y_1, score_1, ..., x_17, y_17, score_17].
What structure does the demo expect?
Thank,
Cataldo
from lighttrack.
@caloc As you can see in line 64 of keypoint_visualizer.py, the keypoints' order are given. In the provided code, I used PoseTrack order because I trained with this dataset. If you replace the pose estimator, you can actually use any order with an arbitrary number of keypoints too. The tracker can take a list [x_1, y_1, score_1, ..., x_n, y_n, score_n]. Currently, it is [x_1, y_1, score_1, ..., x_15, y_15, score_15]. The (x,y) coordinate is the absolute pixel location in the original image.
from lighttrack.
Thank you, then I will try to use the keypoints list as it is.
from lighttrack.
@Guanghan, I have another doubt regarding the notation of keypoints.
In the definition of get_bbox_from_keypoints(keypoints_python_data)
you check that vis
is different from 0 and from 3:
vis = keypoints_python_data[3 * keypoint_id + 2] if vis != 0 and vis!= 3: x_list.append(x) y_list.append(y)
In my case I have the the third (in modulo) position of the array is a score, with 0 meaning undected keypoint. Here what is the meaning of value 3?
from lighttrack.
@caloc Sorry for the untidiness of the code. I believe comparing value 3 is unnecessary. I may have conducted some experiments or debugged something but failed to clean up all unnecessary code afterwards.
from lighttrack.
@Guanghan I had the doubt that it was a debugging line, no problem.
I think I am almost there, now I got the mismatch error during the inference using the SGCN.
File "/home/cataldo/.local/lib/python3.5/site-packages/torchlight-1.0-py3.5.egg/torchlight/io.py", line 82, in load_weights __doc__ = _io._TextIOBase.__doc__ File "/home/cataldo/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 777, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for Model: size mismatch for A: copying a param with shape torch.Size([3, 15, 15]) from checkpoint, the shape in current model is torch.Size([3, 18, 18]). size mismatch for data_bn.running_var: copying a param with shape torch.Size([30]) from checkpoint, the shape in current model is torch.Size([36]). size mismatch for data_bn.weight: copying a param with shape torch.Size([30]) from checkpoint, the shape in current model is torch.Size([36]). size mismatch for data_bn.running_mean: copying a param with shape torch.Size([30]) from checkpoint, the shape in current model is torch.Size([36]). size mismatch for data_bn.bias: copying a param with shape torch.Size([30]) from checkpoint, the shape in current model is torch.Size([36]). size mismatch for edge_importance.1: copying a param with shape torch.Size([3, 15, 15]) from checkpoint, the shape in current model is torch.Size([3, 18, 18]). size mismatch for edge_importance.0: copying a param with shape torch.Size([3, 15, 15]) from checkpoint, the shape in current model is torch.Size([3, 18, 18]).
How do I retrain the SGCN with COCO 18 notation? Or is there an already trained net?
Thank you for your support!
from lighttrack.
@caloc Unfortunately, I have not trained SGCN with COCO. Training SGCN requires data in pairs (or triplets if necessary), where the image pairs are usually within a few frames in a video sequence. COCO dataset seems to provide independent images, thus hard for SGCN training.
One suggestion is to generate synthetic data with COCO notation and use it for training.
from lighttrack.
@caloc For now, you can set the threshold of SGCN to be 0, which turns it off. It sacrifices the MOTA score a bit, but it does not matter too much in demo. In this way, you can at least check whether your CPU version will run successfully.
from lighttrack.
@caloc, It will be great if you could expose the CPU version, if you managed to get it to work
from lighttrack.
@caloc hi, im interested in using this method with bottom up methods like openpose too. Since openpose is bottom up, did you use the keypoints from openpose to construct the bbox of each person?
from lighttrack.
Related Issues (20)
- Detection result of PoseTrack17 validation set
- Can I use it to calculate the number of visitors with a video?
- How to get the file under the detections_openSVAI path
- How can I get the key points in the inference phase? HOT 1
- Where is evaluate.py for evaluation on PoseTrack 2018? HOT 2
- Evaluation on PoseTrack 2018 ? HOT 1
- dockerfile HOT 1
- Dose this support MacOS CPU version and ignore the inference speed? HOT 1
- 在运行demo_video_mobile.py时报错 HOT 3
- How to merge annotation json file? HOT 2
- OpenSVAI can't be opened in China, how can I get it? HOT 1
- Any recommendation to make it on Keras? HOT 2
- Error in running with flag_nms= True HOT 1
- ImportError: gpu_nmscpython-36m-x86_64-linux-gnu.so: undefined symbol: __cudaPopCallConfiguration HOT 2
- AttributeError: 'MSVCCompiler' object has no attribute 'compiler_so'
- Issue while Running this command '!python /content/lighttrack/demo_video_mobile.py'
- PoseTrack download
- undefined symbol: __cudaRegisterFatBinaryEnd HOT 1
- ModuleNotFoundError: No module named 'tfflat'
- ModuleNotFoundError: No Module named nms.gpu_nms
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lighttrack.