Hey there, I am wondering how I would go about connecting the mouth points to each mask so that when I open my mouth it opens the mouth of the mask?

Can you please tell the execution steps required? And how to link OpenCV

why the result is bad when I use your code to test the before.jpg ?

I am trying to build this using cmake on linux but I got this error:

$ make
[ 93%] Built target dlib
Scanning dependencies of target faceswap
[ 94%] Building CXX object CMakeFiles/faceswap.dir/src/main.cpp.o
FaceSwap/src/main.cpp: In function ‘int main()’:
/home/maria/Code/FaceSwap/src/main.cpp:29:61: error: invalid initialization of non-const reference of type ‘std::vector<cv::Rect_ >&’ from an rvalue of type ‘std::vector<cv::Rect_ >’
std::vectorcv::Rect &cv_faces = detector.faces();
~~~~~~~~~~~~~~^~
make[2]: *** [CMakeFiles/faceswap.dir/build.make:63: CMakeFiles/faceswap.dir/src/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:68: CMakeFiles/faceswap.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

Any idea why?
I am compiling with g++ 6.3.1

Hi,
thanks for sharing this nice face swapping example!
Looking through the code I spotted a little bug in the "Normalize CDF" for-loop in FaceSwapper.cpp:261-267. I think you meant to test [255] for zero instead of [i] in these lines.

Regards,
matthias

When I run this, it launches and then crashes out with:

OpenCV(3.4.2) /tmp/opencv-20180704-42729-197okqt/opencv-3.4.2/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

I see that in Snapchat, they can get more than 68 points on face like the picture below:

Do you know how to get the number of points like Snapchat do? Thanks

Can this program run with a single image input (w/two faces) and just provide an image output?

building this on ubuntu is pretty easy, but it took several tries to figure it out, so here are the instructions. i don't have a camera on my desktop (i'll test it on my laptop next week maybe)

other linux should be similar, just replace apt install as appropriate

could you add these instructions to the readme ?

# ubuntu 16.04
sudo apt install libopencv-dev liblapack-dev libdlib-dev
wget http://sourceforge.net/projects/dclib/files/dlib/v18.10/shape_predictor_68_face_landmarks.dat.bz2
bunzip2 *.bz2
ln -s /usr/share/opencv/haarcascades/haarcascade_frontalface_default.xml .

g++ -std=c++1y *.cpp $(pkg-config --libs opencv lapack) -ldlib 
./a.out

In FaceDetectorAndTracker.cpp
void FaceDetectorAndTracker::track()

	if (m_faceTemplates[i].cols <= 1 || m_faceTemplates[i].rows <= 1)  
	{
		m_facesRects.clear();
		m_tracking = false;
		return;
	}

I want to know why the figure is 1, and is it based on experience?
When I delete this statement and tried to remove one of the detected face from the frame, there will be some delay.
When I change the figure into 2 or 3, there also will be some delay, and soemtimes the two face even couldn't swap (although they are of proper size).

Hope for answer sincerely!!!!

macOS installation

Here was a build on macOS Sierra 10.12.6. Dependencies were installed using the Homebrew package manager.

brew install lapack
brew install openblas
brew install opencv
brew install dlib --with-openblas
git clone https://github.com/hrastnik/FaceSwap.git
cd FaceSwap
wget http://sourceforge.net/projects/dclib/files/dlib/v18.10/shape_predictor_68_face_landmarks.dat.bz2
bunzip2 *.bz2
ln -s /usr/local/share/opencv/haarcascades/haarcascade_frontalface_default.xml .
export PKG_CONFIG_PATH=/usr/local/opt/lapack/lib/pkgconfig:/usr/local/opt/openblas/lib/pkgconfig:$PKG_CONFIG_PATH
g++ -std=c++1y *.cpp $(pkg-config --libs opencv lapack openblas) -ldlib
mkdir bin
mv a.out bin
cd bin
./a.out

Hey there, thank you for this source code. Will this work on OSX using XCode?

Hi!
I was able to run the code. How can I achieve an effect not that one face is cloned to another but follow the emotion? I guess I would need to take the keypoints from one face and apply them to the second.

EXAMPLE
If I have person 1 that is not smiling and person 2 that is smiling and opening eyes, I would like to see the person 1 having the face of person 2 that is not smiling and person 2 having face of person 1 smiling and opening the eyes.

Is it a simple fix to getTransformationMatrices() ? or should I rather fix getFacePoints(...)? Could you possibly guide me on how to achive this simple fix?

Thank you for your excellent work. Can you swap face in real time on the desktop?look forward to your reply

it would be great to be able to run the same algorithm on still images, in addition to on input from a camera

Great project, it is working really well. But I see color artifacts when the lighting of the faces is not the same. There is a paper which addresses this problem in histogram matching: https://www.researchgate.net/profile/Frederik_Zilly/publication/243457773_PRESERVING_DYNAMIC_RANGE_BY_ADVANCED_COLOR_HISTOGRAM_MATCHING_IN_STEREO_VISION/links/0deec51d1b71be896a000000.pdf Maybe you can have a look at it.

I am having the following problem:

DLIB_NO_GUI_SUPPORT is defined so you can't use the GUI code. Turn DLIB_NO_GUI_SUPPORT off if you want to use it." #error "DLIB_NO_GUI_SUPPORT is defined so you can't use the GUI code. Turn DLIB_NO_GUI_SUPPORT off if you want to use it." ^ /usr/include/dlib/gui_core/gui_core_kernel_2.h:12:2: error: #error "Also make sure you have libx11-dev installed on your system" #error "Also make sure you have libx11-dev installed on your system"

How can I turn off DLIB_NO_GUI_SUPPORT ?
Libx11-dev is already installed.

Hi
I build the codes on windows successfully, but when I Run the exe, there is nothing. How can I input videos to the exe ?
I'm a new bee here, any helps will be appreciated.

I'll be leaving instructions here as I figure them out.

STEP 1 -- install gcc++ 4.9 (14.04LTS has 4.8 only)
Instructions via: https://askubuntu.com/questions/428198/getting-installing-gcc-g-4-9-on-ubuntu
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install g++-4.9

STEP 2 -- download dlib
While you are inside Faceswap-master directory:
wget http://dlib.net/files/dlib-19.4.tar.bz2
tar xvjf dlib-19.4.tar.bz2

STEP 3 -- install opencv
apt-get install libopencv-dev python-opencv

STEP 4 -- compile
cd into the source dir, and run g++-4.9 -std=c++14 -o faceswapper *.cpp 'pkg-config opencv --cflags --libs' -Idlib-19.4 dlib-19.4/dlib/all/source.cpp -lpthread -lX11 (IMPORTANT NOTE: the single quote marks around pkg-config opencv --cflags --libs should be BACKTICKS but the code editor won't allow me to put backticks inside backticks)

STEP 5 -- run
./faceswapper

If you get this error: libdc1394 error: Failed to initialize libdc1394 then run this command:
ln /dev/null /dev/raw1394

../src/FaceSwapper.cpp: In member function 'void FaceSwapper::swapFaces(cv::Mat&, cv::Rect&, cv::Rect&)':
../../src/FaceSwapper.cpp:48:53: error: no matching function for call to 'FaceSwapper::featherMask(cv::Mat)'
featherMask(refined_masks(big_rect_ann));
^
../../src/FaceSwapper.cpp:48:53: note: candidate is:
In file included from ../../src/FaceSwapper.cpp:1:0:
../../include/FaceSwapper.h:57:10: note: void FaceSwapper::featherMask(cv::Mat&)
void featherMask(cv::Mat &refined_masks); /// TODO

Hi Mateo,

Your work is great! I have one improvement for you. Face mask could be much better extracted using convexHull. This is my code where I use the convex hull method from OpenCV library to extract face mask and results are excellent.

cv::Mat FaceSwap::getFaceMask(cv::Mat &face, std::vector<cv::Point> &facePoints)
{
    cv::Mat mask;
    mask.create(face.size(), CV_8UC1);
    mask.setTo(cv::Scalar::all(0));

    // Find convex hull
    std::vector<int> hullPtsIndex;
    cv::convexHull(facePoints, hullPtsIndex, false, false);

    // Create convex polygon based on hull points
    cv::Point2i maskPoints[68];

    for(int i=0;i<hullPtsIndex.size();i++)
    {
        maskPoints[i] = facePoints[hullPtsIndex[i]];
    }

    // Fill mask polygon
    cv::fillConvexPoly(mask, maskPoints,hullPtsIndex.size(),cv::Scalar(255));

    return mask;
}

Results: