auduno / clmtools Goto Github PK

tools for building face-models for clmtrackr

CSS 0.08% JavaScript 90.92% Python 7.61% HTML 1.39%

clmtools's Introduction

clmtools

This is a collection of python tools for building models for clmtrackr. This includes:

Model builder / trainer
Annotations for some images in the MUCT dataset
Annotations for various images found online
An annotater for new images
Model viewer

Note the annotations included here and the ones used in clmtrackr are slightly different from the ones included in the MUCT dataset. The difference is mainly in the region around the eyes and nose.

Downloading training data

Images from the MUCT database can be downloaded by running:

cd pdm_builder/data
python download_muct.py

The images will be placed in the folder ./data/images.

A set of facial images found online can be downloaded by running:

cd pdm_builder/data
python download_more.py

Please note that these images are not public domain, and this set of images should therefore not be shared or reproduced anywhere without prior consent from the copyright owners.

Training a model

To train a model, make sure you have numpy, scipy, scikit-learn, scikit-image and PIL installed. If you have pip installed, install them by running:

pip install numpy scipy scikit-learn scikit-image pillow

Note: On OSX, you may have to run as sudo in order to install into the /Library/Python folder.

To train a model, run pdm_builder.py. From the main folder type:

cd pdm_builder
python pdm_builder.py

Model will be output as model.js. Some configuration settings for the model training can be set in ./buildlib/config.py.

Annotating your own images

Open ./annotater/main.html with a browser. Load images to annotate by pushing "Choose Files" and select the image(s) you wish to annotate. The annotater will attempt to fit an annotation, but if it fails miserably, manually select the face by clicking "manually select face" and click and drag a box around the face. From the rough annotated face, modify the annotation by clicking and dragging the points.

Points that are obstructed (by hair, clothes, etc.) should not be used for training the classifiers. To avoid this, hold down shift while clicking the points, so that the point turns red. This means that the point will be used only for creating the facial model, not the classifiers.

Note that annotations will automatically be saved in html5 local storage, so you don't need to save the data between sessions or images. To write out the annotated data, click "save file". The browser will save a file called annotations.csv to the default download folder. Note that this file will include all annotations currently in the local storage.

annotations.csv is a semicolon-separated value file of the following format :

i000qa-fn.jpg;186;346;false;187;382;false;190;410;false;203;440;....
i000qb-fn.jpg;145;357;false;144;389;false;148;422;false;161;451;....

The first column is the name of the image file, followed by three fields for each point, where the first field is x-coordinate of the point, second field is the y-coordinate, and the third field is whether or whether not the point can be used in training classifiers.

To train a model from these annotated images, modify the variables images and annotations in ./buildlib/config.py to point to the folder containing your images and the csv file with annotations respectively.

The placement of the points used in the annotations for the models in clmtrackr look roughly like this:

clmtools's People

Contributors

Stargazers

Watchers

Forkers

saifurrahman 9kopb musajoemo imclab chu888chu888 samuel--hu why-not-sky cvml omergolan keichacon borabilgin zhoushiwei fanqiehc mcanthony swang gigakiller anjith2006 ivanjacobs thebinoman dstnation ayush15k1994 andersonoliveira saibabanadh codeaudit optionalg yuccai gamezpedia j0k mmmika ztane sterotto kate-ivanova aahedi yanshui177 kmiyamoto0105 danielapt pradyuz3rocool moriedan liamwhan adamhipster guyperl supertopoz imageasinput tianyu06030020 edmig nstkuig hyb1234hi manu0377 antoniotorresm jclimma jlpp2006 oxyhyxo gypsybud quijoteshin w0lramd

clmtools's Issues

I don't get something in the fitting faces article, does an SVM kernel when doing logistic regression really exist? Isn't it just a linear kernel?

In https://www.auduno.com/2014/01/05/fitting-faces/ it states the following

in our case we implemented a logistic regression classifier with an SVM kernel (which is what the original paper suggests)

My knowledge on classification is limited. I know what SVMs are on a high level, I know what logistic regression is. But I didn't know what a kernel was. So I looked it up and learned what it more or less is (if I remember correctly it is a dot product that somehow equates to intersecting transposed data points in an arbitrary higher dimensional space). But in all those explanations the words "SVM kernel" do not exist, since support vector machines use of kernel themselves such as a linear or gaussian kernel, or you name it. I searched on via Google for quite a while.

So I took a look in the source code and the word "SVM kernel" does not seem to exist. I just queried on the word "kernel".

https://github.com/auduno/clmtools/search?utf8=%E2%9C%93&q=kernel

To me it seems that the implementation in actuality is a logistic regression classifier with a linear kernel.

My question is: is this the case? Or is there something inherent SVM-like about this particular kernel? When I look at Wikipedia, it does not mention SVM kernels: https://en.wikipedia.org/wiki/Support_vector_machine (it mentions linear kernels, etc.). But when I look at http://scikit-learn.org/stable/modules/svm.html it does mention SVM kernels but not in conjunction with logistic regression.

sklearn DeprecationWarning: Passing 1d arrays...

c:\Miniconda2\lib\site-packages\sklearn\utils\validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample. DeprecationWarning)

I am using scikit-learn 0.17 on Python 2.7.11 on Windows.

The quality of detection.

Hello.
I generated the model but her accuracy much worse the original model (https://github.com/auduno/clmtools/blob/master/annotater/js/model_pca_20_svm.js). I used numpy 1.9.3, scikit-image 0.11.3, pillow 3.0.0 and scikit-learn 0.15.2.
How can I increase accuracy learning model?

In the archive are results works learning model and original model for compare.

report 13.12.2015.zip
log.txt

PCA Error

File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\sklearn\decomposition\pca.py", line 425, in _fit_full
% (n_components, min(n_samples, n_features)))
ValueError: n_components=20 must be between 0 and min(n_samples, n_features)=2 with svd_solver='full'

Where can I download the negatives images?

Hi I am trying to run through your steps to build out a new model. There are a couple of errors but I am now stuck on the step where it cannot find anything in the "negatives" image folder. What should I put in there? Anything that is not a face or is there a dataset I have to download?

Thanks!

Python Help document

me config.py
data_folder = "./data/"
images ="./data/images/"
annotations = "./data/annotations.csv"
message:
No images were found for training. Please make sure that folders in config.py are correct, and that images for training are downloaded.

I want to do batch execution.
Do you have a working document?
Help me

Problem solved

Sorry again, I fixed all issues by myself, thank you for clmtools! :D

Annotation and training for any landmark number.

I need to detect more and fewer landmarks on the face according to functions. İs it possible? Can I use clmtools and clmtracker for this purpose? Thanks.

Can model training only be performed on version 2.7? 3.7 No?

High angular face matching

I'm trying to match faces at high angle (> 30°),
so I load the image and try to annotate it using the annotater, but it doesn't manage to converge so it gives its best estimate that is quite wrong. Nothing unexpected here, but moving every 70pts one by one is such a chore, is there a way to select multiple points at once?
Also, is there a way to modify the model PCA parameters on the given best estimate? As the annotater does not converge on it's own I'd like to do it myself.

use of clmtools

I want to use clmtracker for my project. the problem is that how i can fix models on face. You have suggested clmtools for it but the problem is that i have no experience in python nor i can understand it's document. Is there any other way to make image appear on face.

If you can help me it will be of great help.

Getting Error in clmtools

I want to use clmtools to train the model with existing data , but The clmtool not working.
its giving this error.
Can you please help regarding this issue.

getting negative examples from landscape images
C:\Users\anwaar\Anaconda2\lib\site-packages\sklearn\preprocessing\data.py:160: UserWarning: Numerical issues were encountered when centering the data and might not be solved. Dataset may contain too large values. You may need to prescale your features.
warnings.warn("Numerical issues were encountered "
C:\Users\anwaar\Anaconda2\lib\site-packages\sklearn\preprocessing\data.py:177: UserWarning: Numerical issues were encountered when scaling the data and might not be solved. The standard deviation of the data is probably very close to 0.
warnings.warn("Numerical issues were encountered "
training patch:0
0
C:\Users\anwaar\Anaconda2\lib\site-packages\sklearn\cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
C:\Users\anwaar\Anaconda2\lib\site-packages\sklearn\grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
DeprecationWarning)
Traceback (most recent call last):
File "pdm_builder.py", line 54, in
patchModel = build_patches(data_patches, gradient=True, lbp=True, weights=weights, optimize_params=True)
File "c:\clm\clmtools\pdm_builder\buildlib\buildpatch.py", line 169, in build_patches
clfg = GridSearchCV(SVR(kernel="linear"), {'C':[0.1], 'epsilon' : [0.4, 0.3, 0.2, 0.1]}, loss_func=mean_squared_error, verbose=100)
TypeError: init() got an unexpected keyword argument 'loss_func'

"Numerical issues" Warning building model, then very bad accuracy

I added some new points to the face training images and configured the tool to take them into account.
I get the following Wargning:

/Library/Python/2.7/site-packages/sklearn/preprocessing/data.py:153: UserWarning: Numerical issues were encountered when centering the data and might not be solved. Dataset may contain too large values. You may need to prescale your features.
  warnings.warn("Numerical issues were encountered "
/Library/Python/2.7/site-packages/sklearn/preprocessing/data.py:169: UserWarning: Numerical issues were encountered when scaling the data and might not be solved. The standard deviation of the data is probably very close to 0.

And the precision of the tracker is horrible now.

Any tips? I started with the provided anotations.csv and adapted the model_viewer and pdm_builder to account for the new points.

hot link protection

When I download this image, it gives me an image with an error "hot link protection activated", so I am not sure this is the right image

http://blog.54ka.org/wp-content/uploads/2008/11/accidental_street_portrait_01_by_54ka.jpg