Comments (42)
Yes, you are right.
You can use some tricks of image augmentation (like random horizontal flip, change their light contrast, resize then random crop and etc.) to prevent your model from overfitting.
OR, you can take a look at MARS dataset which is the superset of Market1501.
from deep_sort_pytorch.
The folder name is the label of images in that folder. Duplicating images could introduces risks of overfitting. Split images of each folder to train/test set.
from deep_sort_pytorch.
Yes, it is possible.
I have released the training scripts.
You shall make your own dataset first like Market with each image resized to 64x128.
Then, put images of one ID into individual folders.
Good luck!
from deep_sort_pytorch.
Hello ZQPei,
While training using market dataset, I am getting 0 loss and 100% accuracy in each step since the beginning. I have shared a screenshot below. I used train.py inside deep folder in your repository as you mentioned. Could you please tell me where the problem is?
Also, I'm not getting any curve for training. Only validation curve is being saved.
from deep_sort_pytorch.
I not sure exactly where the problem is. You need to tell me more details.
1.What is your dataset folder like?
2.Which python do you use? python2 python3?
3.You can debug it by adding a breakpoint in train() function. Check the inputs is correct or not.
from deep_sort_pytorch.
-
My dataset is in the following order:
-data
-train
-1
-.jpg
-.jpg
...........
-.jpg
-test
-1
-.jpg
-.jpg
...........
-.jpg
In the data folder there are test and train folders. Inside test and train folder, there are subfolders called 1. Inside the folder 1, there are .jpg images of size 64x128. -
I am using python 3.7.
The training completes successfully and the checkpoint is also saved but the loss is 0 and accuracy 100% on each iteration. When i run test.py, the following error occurs:
from deep_sort_pytorch.
The reason why your loss is zero, is your dataset folders config is not right.
And this also lead to your test.py error.
You shall re-organize your dataset like this as is shown below.
In train/
In test/
from deep_sort_pytorch.
Oh. How many images shall i put in each folder? Do i have to make one folder to put one image or i can put multiple images in one folder.
for e.g.
insider the folder 0002, i have to put only one image or i can put more than one image?
from deep_sort_pytorch.
It has no limit.
You can put as many as images as you want.
But you'd better balance your image numbers in each folder.
from deep_sort_pytorch.
thank you so much. i will try that.
i have another question to you. is your deepsort pytorch model exactly same as the original deep sort? or did u make any changes? what are the changes you made in your pytorch implementation of deepsort?
from deep_sort_pytorch.
Not exactly the same. Because It is not that easy to convert a tf model parameters to a pytorch model.
I have changed the original CNN model a little bit, mainly adding more channels in conv layers. You can also customize these hyper parameters on your models.
The other difference is the dataset I use for training is Martet1501, while the auther was using Mars dataset. Mars dataset is much bigger than Market1501. I suggest you try that later.
If you feel this repository is helpful, please give this repo a star, thank you!
from deep_sort_pytorch.
sure, thanks a lot for your help.
from deep_sort_pytorch.
Hello ZQPei,
thanks for your co-operation and help.
I followed your instructions to configure the dataset and successfully trained the model. But when i try to run it using test.py or demo_yolo3_deepsort.py..the following erros occurs:
Also, the loss graph is quite strange with increasing validation loss.
what could be the issue?
from deep_sort_pytorch.
Loss on training set is really decreasing, which means the network is converging.
But, loss on val set is going the opposite way.
This can happens this model is overfitting on your trainset.
A model will run into overfitting for many reasons.
From your terminel, i can tell there are only 29 classes in your dataset. So I assume that you dataset is really small, isnt it? Too small dataset will lead to overfitting.
I suggest you to finetune your model on my pretrained parameter to solve this problem.
Also, you can google how to solve overfitting problem on a CNN model.
When you run demo.py, the error occured because your model has only 29 outputs in the final fully connected layer. You can solve that by modifing feature_extractor.py in line 10.
self.net = Net(num_classes=29, reid=True)
from deep_sort_pytorch.
Can u please tell me how can i fine tune using your pretrained parameter?
from deep_sort_pytorch.
I recommend you the official transfer learninng tutorial on ants and bees.
It is not that hard, you will get how to fineturn a model with pretrained parameters.
This is good for you.
from deep_sort_pytorch.
The tutorial you suggested focuses on Resnet model. I searched other tutorials but they perform fine tuning on other popular models like VGG, Inception, Xception. I couldnt get any idea how to fine tune other models including your model. Could you please suggest something else?
Also, Could you please tell me the loss function that you used for training?
from deep_sort_pytorch.
Sorry for repling so late.
Yes, the tutorial is basicly showing you how to transfer a model based on a pretrained model on IMAGENET to your own classification task.
The deep sort CNN model is to get the similarity of two person. How?
It is because CNN model's output without the last fully connected layer can get you the feature of a person image, then you can compute the cos similarity(or something else) of two person features to get whether these two are or aren't the same person. It is the main idea of a simple reid model.
So how do we train a CNN model to get a nice person feature that can show the differences of different individuals? By training a CNN model that can classify different person, e.g. in Market1501, there are 1500 individual class of different person. This is basicly a image classification task. I hope you can get that.
I use cross entropy loss for training. It is a classic loss function in classification task.
Also, you can try focal loss or something else from the newest paper.
from deep_sort_pytorch.
Thank you very much ZQPei for your explanation and help.
But I am a little confused about training market1501. Do I have to make separate 1501 folders inside the training folder and put images of each individual inside each folder and same for test? If I do that will there be enough data to train properly?
I mean for 1501 classes, the structure should be:
- train
-0001
-0002
.
.
.
.
-1501
And put the images of each person into each folder?
Or am I getting it wrong?
Because if I do that, there will be only around 30-40 images per class and would the training data be enough?
from deep_sort_pytorch.
And may I ask what is the 'top1err' graph in the train chart about? What does it signify?
from deep_sort_pytorch.
After your CNN's last fully connected layer, you will do softmax to get the proballity of each class and choose the class with the top1 prob(max prob) as your predict class. With this prediction, your error is named top1error.
from deep_sort_pytorch.
Thanks again for your great help.
from deep_sort_pytorch.
Hello ZQPei,
What changes do I have to make in your code if I want to display all the classes with their class name? I have 5 classes. E.g. human, tv on, tv off, fan on and fan off. I need to display their names along with their respective object id. In your code, only persons are displayed by bounding box and it is labelled as 'object 1', 'object 2' and so on. If need to display as ' human 1', 'human 2', 'fan on 1', 'fan on 2' etc..... How can I do that?
from deep_sort_pytorch.
To trains a image classifier, the dataset should use the same set of img label in both training and test set. The market1501 dataset doesn't organized in that way, since it's designed for re-ID problems.
Just reorganize the dataset as ZQPei mentioned~
from deep_sort_pytorch.
To trains a image classifier, the dataset should use the same set of img label in both training and test set. The market1501 dataset doesn't organized in that way, since it's designed for re-ID problems.
Just reorganize the dataset as ZQPei mentioned~
@mljack Hello, do you mean the same number of datasets in both train and test ? I know that the test dataset is only 20% of all datasets ! So, I don't understand how to dataset should use the same set of img label in both training and test set.
Thank you in advance,
from deep_sort_pytorch.
In short, a classifier is first trained to output object id from cropped object image, then the last fully-connected layer is removed and the neural network is re-targeted to output object appearance features which is used to compute cosine distances.
To train the classifier, train and test sets should share the same set of object id which are labels.
from deep_sort_pytorch.
In short, a classifier is first trained to output object id from cropped object image, then the last fully-connected layer is removed and the neural network is re-targeted to output object appearance features which is used to compute cosine distances.
To train the classifier, train and test sets should share the same set of object id which are labels.
@mljack thank you for your response, I don't really understand, do you lean that in test and train folders, I put labels or images ? I mean in 0010, 0011.. I put labels ? And the same images/labels are in both folders ( train and test) a copy ?
During my last succefull trainig of deepsort, I copy subfolders from train to test and it works but I want to better understand.
I have this curves when I made the same images (without labels) in test and train folders.
So it is clearly false what I am doing ! PLease help.
Thank you
from deep_sort_pytorch.
The folder name is the label of images in that folder. Duplicating images could introduces risks of overfitting. Split images of each folder to train/test set.
@mljack Thank you very much,
to train deepSort I don't need xml/.txt labels, that's right ? Only the pretrained weight is needed ?
Regards,
from deep_sort_pytorch.
Did you succeed to train deep sort?
If yes, What is the form of dataset architecture?
from deep_sort_pytorch.
Hi @hedi1920, I used this format in both test and train filder, putting inside inly one picture (different in each folder (test/train)).
But I keep the same number as recommanded above.
from deep_sort_pytorch.
Thank you
Just i would now if the images are cropped (just contains the object) or no?
from deep_sort_pytorch.
Thank you
Just i would now if the images are cropped (just contains the object) or no?
@hedi1920 I put only the image in the folder, without any information ( my image contains 5 classes) without bbox, and it was my last question about cropped data, so I don't know if it is right or no, may be @mljack could answer to this question. thanks
from deep_sort_pytorch.
Tje folder data contain 80% of train and 20% of test
Test and train contains both inputs and labels
from deep_sort_pytorch.
Tje folder data contain 80% of train and 20% of test
Test and train contains both inputs and labels
@hedi1920 May be I didn't understand the answers above, but they say the same number in train as in test, that weird because your answer right know is right, did you try it ?
from deep_sort_pytorch.
If you use the same set for train and test the accuracy will be equal 100% from the first epoch!!!!! It's weird
So the sets will be different to avoid the overfitting
from deep_sort_pytorch.
If you use the same set for train and test the accuracy will be equal 100% from the first epoch!!!!! It's weird
So the sets will be different to avoid the overfitting
@hedi1920 I agree, but if you read my previous messages you will see that I don't really understand how it can work without cropped images and with the same amount of test/train dataset, so if you try It with 20% test and it works could you please leave a message, thanks
from deep_sort_pytorch.
@hedi1920 I agree, but if you read my previous messages you will see that I don't really understand how it can work without cropped images and with the same amount of test/train dataset, so if you try It with 20% test and it works could you please leave a message, thanks
from deep_sort_pytorch.
from deep_sort_pytorch.
@hedi1920 My last result implementing the same amount of data in both test and train I obtained this result
from deep_sort_pytorch.
from deep_sort_pytorch.
In train.py Net =net(num_classes=num_classes, reid=True) Le mar. 22 juin 2021 à 13:22, Sarouch @.***> a écrit :
…
@hedi1920 https://github.com/hedi1920 My last result implementing the same amount of data in both test and train I obtained this result [image: image] https://user-images.githubusercontent.com/40342672/122923465-0ab93680-d365-11eb-8672-bbe4a682d18f.png — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMLLQP35TA4HA26PLT34KVLTUB56XANCNFSM4GZIBL4A .
@hedi1920 Hello, did you find out the problem and train the deepsort ?
from deep_sort_pytorch.
from deep_sort_pytorch.
Related Issues (20)
- from fastreid.config import get_cfg ModuleNotFoundError: No module named 'fastreid' HOT 3
- Can not detect with my own trained weights HOT 1
- cv2.imshow freezes using deep_sort HOT 1
- About the indicator of reid model in evaluate.py
- Deepsort Training
- How to disable fastreid and mmdet? HOT 4
- A not aligned error when calculating the _cosine_distance
- Why here do you delete a convolutional layer?
- reid模型从torch转为onnx,精度损失。
- softmax vs cosine softmax ? HOT 1
- Error while compiling NMS module. Please help! HOT 1
- deep_sort/sort/tracker.py函数疑惑
- 如果需要训练自己的数据集,我该如何重新训练reID部分。我的YOLO目标检测部分已经训练好了。 HOT 2
- What is the threshold range of cosine similarity of reid feature?
- deep_sort/deep/feature_extractor.py Extractor net.eval()
- Does it support oriented bounding boxes' tracking?
- How to replace the detector?
- How to exchange Mask R-CNN from YOLOv3
- pip install -r requirements.txt have error.plz help! HOT 3
- little question about resnet.py
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deep_sort_pytorch.