Reproducing Faceforensics++: Learning to detect manipulated facial images (Github Source Paper )
To reproduce the same results, the dataset needs to be downloaded using this form by agreeing to their terms & conditions.
Repository Descprition:
This repostitory is a major project ,part of the course unit COMP8240 - Applications of Data Science
- Sai Diwakar Bhrugubanda (45852189)
- Saiprudhvi Meda (45725535)
- Rahul Sonkusare (45779880)
- Shruti Tyagi (45831815)
There are two main stages of implemenation
- Replication of original work
- Creating new data & applying on the original work
Dataset used: link
- Data Extraction
- Using the script to download
- Using the script to extract frames from the videos downloaded.
- Data Pre-Processing
- First faces are extracted from video frames using bounding boxes.
- Splitting the data and creating Train,Test and CV data.
- Shuffling the data and assigning class labels to every image 1 or 0 (Fake =1 and Real =0).
- Pickling the data to create .pkl files.
- Xception Modelling
- Import the pre-processed data i.e. Pickle files.
- Feed it into the model to pre-train and then test the accuracies using 2 variation of epochs.
- Evaluate the model on Train,CV and Test data to find accuracy of the model.
python download-Faceforensics.py -d all -c c23 -t videos
python extracted_compressed_videos.py -d <"all" or single dataset via "Face2Face" or "original"> -c c23
File: Original Work/ Data Pre-Processing.ipynb
File: Original Work/ Xceptionnet_Modelling.ipynb
- Creating New Data
- First Order Motion for images animation (Deep fakes) Source Github
- Deepfakes based on front facing images which create perfect deepfakes.
- Deepfakes based on side angle images which create deepfakes that are little easy to differentiate between fake and real.
- Acquired faceswap and face2face from thirdparty sources i.e youtube
- Xception Modelling
File: New Data/Creating-Deepfakes.ipynb
Since the model is performing too well and creating perfect deepfakes, to test the XceptionNet model accuracy we are creating two types of deepfake videos.
File: New Data/ Xceptionnet_Modelling(Front Facing Data).ipynb
File: New Data/ Xceptionnet_Modelling(Side Facing Data).ipynb
Application of XceptionNet of 2 Variations of data i.e Front-Facing and 2. Side-Facing images
- Rössler, Andreas & Cozzolino, Davide & Verdoliva, Luisa & Riess, Christian & Thies, Justus & Nießner, Matthias. (2019). FaceForensics++: Learning to Detect Manipulated Facial Images.
- Siarohin, Aliaksandr & Lathuilière, Stéphane & Tulyakov, Sergey & Ricci, Elisa & Sebe, Nicu. (2020). First Order Motion Model for Image Animation.