Nice model you have got! Thanks for sharing it.

I wonder what kind of data you fed into the network in order to train, I mean, What database do you use to get so many keypoints and also 3d pose?

As you use a face detector, do you perform face cropping before feeding the data into the network?

Could I get more information about the training stage? Would it work for vgg-network?

Another last thing, why did you transform the prediction into this: predictpoints = predictpoints * vgg_height/2 + vgg_width/2 How the labels were normalized?

Hi, thanks for your works!
I'm confused that once I have annotated facial landmarks(e.g. 68 pts from 300W dataset), how can I transform(or use some algorithms) to get head-pose labels from them. I found from other issues you used Intraface to get head pose from facial landmark points. But is the way correct enough to get pose labels as ground truth. For when we train CNNs, the final accuracy will be limited to the labels. Could you give me some advice or are there any pose dataset? Wish for your reply!

Your pre-trained model performance is quite good. Can you provide info of how you train your net?

Thanks

您好，请问网络文件https://github.com/qiexing/face-landmark-localization/blob/master/train/train_val.prototxt中source: "/home/hkk/DATACENTER/hdf5/box_train_bgr_data_list.txt"里面的内容是什么，是.h5文件的路径吗？
另外，.h5文件中data是图像数据？label是(x1,y1),(x2,y2)...这种形式的68个点的坐标吗？pose的格式是
Pitch, Yaw, Roll顺序的角度吗？
最后一个问题，训练的话，在终端执行"..\..\\Build\x64\Release\caffe.exe" train --solver=ImageNet_solver.prototxt --gpu=0,1,2,3这种类似的命令吗？

Hello,
Thank you for the model and codes. I am a bit confused. There are two deploy files provided. The model/deploy.prototxt consists of a model which gets divided at the fully connected layer fc7 to two parts: 68-point and poselayer. However, the train/deploy.prototxt consists of a model which gets divided after the relu4 layer to be passed onto conv5 and conv5_b, and so is the train/train_val.prototxt. What are the differences in results for these and which one do you recommend? Also, we just have one train_val.prototxt. Is the caffemodel trained on a file similar to model/deploy.prototxt?
Thank you in advance.

作者您好，您的工作很棒，我想跟着你用你train文件夹里面的网络结构自己重新训练一下，但是出现了了loss下降到0.7左右时候就没法下降，不论人脸的姿势是怎么样的，训练检测的每张图片的脸的68个点几乎都是端正分布在图中构成一个正面的脸。

请问您训练的过程中有遇到过这个问题吗？针对这个问题您有没有什么意见或建议以便我能寻找这个问题的根源呢，谢谢！期待您的回复

Traceback (most recent call last):
File "landmarkPredict.py", line 239, in
func(*sys.argv[2:])
File "landmarkPredict.py", line 218, in predictImage
vgg_point_net.set_input_arrays(faces.astype(np.float32),data4DL.astype(np.float32))
TypeError: _Net_set_input_arrays() takes exactly 4 arguments (3 given)

Thanks for author excellent work!

In my research, I need train the other net of face landmark detection. However, only training the single label dataset, I don't know how augment the landmark label data to the format requested by caffe. Can anyone provide the asked dataset , only need little data, to help me learn how to augment multi labels data for caffe.

Thank you very much!

can it be used in video?

Hi, Is it possible to change the batch size for caffe in your prediction script? Will it be faster than running the net once for every image?

I use the cpu only caffe to predict image.got the errror like this:

I0718 11:43:22.602149 1904979968 net.cpp:274] Network initialization done.
I0718 11:43:23.239444 1904979968 net.cpp:752] Ignoring source layer MyData
I0718 11:43:23.320868 1904979968 net.cpp:752] Ignoring source layer loss
I0718 11:43:23.320929 1904979968 net.cpp:752] Ignoring source layer poseloss
F0718 11:43:23.346355 1904979968 common.cpp:66] Cannot use GPU in CPU-only Caffe: check mode.
*** Check failure stack trace: ***
Abort trap: 6

I have modified the landmarkpredict.py:

def predictImage(filename):
    vgg_point_MODEL_FILE = 'model/deploy.prototxt'
    vgg_point_PRETRAINED = 'model/68point_dlib_with_pose.caffemodel'
    mean_filename='model/VGG_mean.binaryproto'
    vgg_point_net=caffe.Net(vgg_point_MODEL_FILE,vgg_point_PRETRAINED,caffe.TEST)
    caffe.set_mode_cpu()
    # caffe.set_mode_gpu()
    caffe.set_device(0)
    f = open(filename)

谢谢！

看你的介绍说采用了人工增加样本的方法，可否分享一下代码呢？

Hello, I have a face data with 194 face key points. I want to use your network to train my data so that I can predict 194 face key points, but there is no explanation in your explanation. Please tell me how should I start?

what's the problem?

I found you submitted a poster in PCM2016 “Facial Landmark Localization by Part-Aware Deep Convolutional Network”

Could you share it to us? I think it could be relevant with this. My email address is: [email protected]

And also, could you share more about your training progress, like your prototxt, slover, log files...

Thank you.

Dear qiexing,
Could you please provide the train_val.prototxt and solver.prototxt of Caffe? Many thanks.

Hi

is it possible to use a video file for head pose estimation ?

How to demarcate the "Roll","Yaw"and "Pitch" in the training set? How many samples in your training set? Thank you!

Hello,
thanks for your excellent job! I want to train the data by myself.

What is your final poseLoss value?

你好！非常感谢你的程序，让我在人脸特征点检测上面得到很多启发。关于多任务学习现有如下疑问：
1.看到你有两个cov4，之后分别处理了68个特征点的线性回归，和3Dpose线性回归，关于标签在最后损失层具体是怎么处理的？因为没有看到你加slice层，所以有些困惑~
2.能否告知一下你数据内容划分？标签那里是68个点的坐标+3个方向值么？

再次感谢你的研究，对我的帮助非常大。若有时间还望告知解答谢谢！

Does this give 3D face landmarks?

Hey its quite an interesting project. Would you mind sharing sample training data & label for us to replicate the results.

Traceback (most recent call last):
File "landmarkPredict.py", line 232, in
func(*sys.argv[2:])
File "landmarkPredict.py", line 201, in predictImage
colorface = getRGBTestPart(bbox,M_left,M_right,M_top,M_bottom,colorImage,vgg_height,vgg_width)
File "landmarkPredict.py", line 78, in getRGBTestPart
face=img[retiBBox[2]:retiBBox[3],retiBBox[0]:retiBBox[1],:]
TypeError: slice indices must be integers or None or have an index method

python landmarkPredict.py img/1.jpg testList.txt typed this in the command line.and got Segmentation fault: 11 . Caffe and dlib is installed fine.

请问制作hdf5数据的时候，1）、像素减均值了吗？
2）、另外特征点的坐标需要归一化吗？

可以方便给一个制作源码吗，谢谢！

Thanks for sharing the code and pre-trained model .
Could you tell me how to get the 68point_dlib_with_pose.caffemodel by intraface？
thank you very much

hellow,i met a error when using landmarkPredict_webcam.py for video. as follow:

File "****/face-landmark-localization/landmarkPredict_webcam.py", line 148, in detectFace
dets = detector(img, 1)
TypeError: call(): incompatible function arguments. The following argument types are supported:

1. (self: _dlib_pybind11.fhog_object_detector, image: array, upsample_num_times: int=0) -> _dlib_pybind11.rectangles

Invoked with: <_dlib_pybind11.fhog_object_detector object at 0x7f859f659c70>, None, 1

总共八层，前五层是卷积层，后三层是全连接层，vgg不是16或者19层吗？

现有的68个landmarks的数据集大多使用300-W，但此数据集中并不包含3D pose，所以我想问下这个3D pose 具体是怎么获得的？

再次请教大神！
我只是训练特征点定位，加上您的模型finetune，只可以把loss降到0.05.请问您在做实验时特征点的loss最低能降到多少呢？
谢谢！

另外请问网络结构是改自经典网络么？
还有3个pose角度的准确性用什么标准评测？

hi, i see your code.
but, i find not related paper.

i wonder related paper about your code.

Thanks for sharing the code! very good job!

You know, dlib provides good implementation for face landmarks estimation.
In order to make it faster, I want to predict only the pose (yaw,roll,pitch).
So if I want to predict the pose angles only, what alterations on model do you suggest in order to make it faster.

Thanks.

It's very kind of you to share the model.

But I wonder some details about the training set. Did you use external dataset apart from IBUG 300W? If used, what's the size of your private dataset? If not, how did you avoid overfitting while using such small data set?

我想得到一张人脸的 这三个参数

when i run code it gives error,please help me
error is
print faces[i].shape
^
SyntaxError: Missing parentheses in call to 'print'

Traceback (most recent call last):
File "landmarkPredict.py", line 252, in
func(*sys.argv[2:])
File "landmarkPredict.py", line 231, in predictImage
vgg_point_net.set_input_arrays(faces.astype(np.float32),data4DL.astype(np.float32))
File "/home/ggj/lch/caffe/python/caffe/pycaffe.py", line 245, in _Net_set_input_arrays
return self._set_input_arrays(data, labels)
RuntimeError: data array has wrong number of channels
Segmentation fault (core dumped)

如果不用预训练模型，初始化也设成了
weight_filler {
type: "xavier"
variance_norm: AVERAGE
}
但是loss降到2再也不降，对测试集来说输出的人脸特征点全部一样。

请问这是什么原因呢？谢谢！

Can't display the image in testlist.

0
img/test0.jpg
1
img/1.jpg
2
img/2.jpg
Traceback (most recent call last):
  File "landmarkPredict.py", line 230, in <module>
    func(*sys.argv[2:])
  File "landmarkPredict.py", line 220, in predictImage
    show_image(colorImage, level1Point, bboxs, predictpose)
  File "landmarkPredict.py", line 48, in show_image
    cv2.circle(img,(int(round(facepoint[faceNum,i*2])),int(round(facepoint[faceNum,i*2+1]))),1,(0,255,0),2)
OverflowError: Python int too large to convert to C long

I'm not familiar with python.What's that mean?

Hey, once again congratz on the project.
I was wondering if it is possible to know the locations of some landmarks on the images.
My goal is to find out if they eye's are closed or open, I figure the best way to do this is knowing the distance between one landmark above the eye and one below.
Thx in advance!